WatsonxTextEmbedder
在执行嵌入检索时,您可以使用此组件将查询转换为向量。然后,嵌入检索器会查找相似或相关的文档。
| pipeline 中的最常见位置 | 在查询/RAG 管道中的嵌入 检索器 之前 |
| 必需的初始化变量 | "api_key": An IBM Cloud API key. Can be set with环境变量 WATSONX_API_KEY。"project_id": An IBM Cloud project ID. Can be set with环境变量 WATSONX_PROJECT_ID。 |
| 强制运行变量 | "text": 字符串 |
| 输出变量 | "embedding": 浮点数列表 "meta": 元数据字典 |
| API 参考 | Watsonx |
| GitHub 链接 | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx |
概述
To see the list of compatible IBM watsonx.ai embedding models, head over to IBM documentation. The default model forWatsonxTextEmbedder isibm/slate-30m-english-rtrvr. You can specify another model with themodel 参数指定其他模型。
使用WatsonxTextEmbedder to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the WatsonxDocumentEmbedder, which enriches the document with the computed embedding, also known as vector.
该组件使用默认情况下,使用 WATSONX_API_KEY 和WATSONX_PROJECT_ID environment variables by default. Otherwise, you can pass API credentials at initialization withapi_key 和project_id:
embedder = WatsonxTextEmbedder(
api_key=Secret.from_token("<your-api-key>"),
project_id=Secret.from_token("<your-project-id>")
)
用法
安装使用 watsonx-haystack 包来使用WatsonxTextEmbedder:
pip install watsonx-haystack
单独使用
以下是独立使用该组件的方法:
from haystack_integrations.components.embedders.watsonx.text_embedder import WatsonxTextEmbedder
from haystack.utils import Secret
text_to_embed = "I love pizza!"
text_embedder = WatsonxTextEmbedder(
api_key=Secret.from_env_var("WATSONX_API_KEY"),
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
model="ibm/slate-30m-english-rtrvr"
)
print(text_embedder.run(text_to_embed))
# {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
# 'meta': {'model': 'ibm/slate-30m-english-rtrvr',
# 'truncated_input_tokens': 3}}
We recommend setting WATSONX_API_KEY and WATSONX_PROJECT_ID as environment variables instead of setting them as parameters.
在 pipeline 中
from haystack import Document
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.watsonx.text_embedder import WatsonxTextEmbedder
from haystack_integrations.components.embedders.watsonx.document_embedder import WatsonxDocumentEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [Document(content="My name is Wolfgang and I live in Berlin"),
Document(content="I saw a black horse running"),
Document(content="Germany has many big cities")]
document_embedder = WatsonxDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)['documents']
document_store.write_documents(documents_with_embeddings)
query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", WatsonxTextEmbedder())
query_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query = "Who lives in Berlin?"
result = query_pipeline.run({"text_embedder":{"text": query}})
print(result['retriever']['documents'][0])
# Document(id=..., mimetype: 'text/plain',
# text: 'My name is Wolfgang and I live in Berlin')
更新于 3 个月前
