OpenAI encoder class for generating embeddings using OpenAI API.
The OpenAIEncoder class is a subclass of DenseEncoder and utilizes the OpenAI API to generate embeddings for given documents. It requires an OpenAI API key and supports customization of the score threshold for filtering or processing the embeddings.
default value, should be replaced by config
Initialize the OpenAIEncoder.
Arguments:
name
(str
): The name of the embedding model to use.openai_base_url
(str
): The base URL for the OpenAI API.openai_api_key
(str
): The OpenAI API key.openai_org_id
(str
): The OpenAI organization ID.score_threshold
(float
): The score threshold for the embeddings.str
0 (str
1): The dimensions of the embeddings.str
2 (str
1): The maximum number of retries for the OpenAI API call.Encode a list of text documents into embeddings using OpenAI API.
Arguments:
docs
: List of text documents to encode.truncate
: Whether to truncate the documents to token limit. If
False and a document exceeds the token limit, an error will be
raised.Returns:
List of embeddings for each document.
Encode a list of text documents into embeddings using OpenAI API asynchronously.
Arguments:
docs
: List of text documents to encode.truncate
: Whether to truncate the documents to token limit. If
False and a document exceeds the token limit, an error will be
raised.Returns:
List of embeddings for each document.
OpenAI encoder class for generating embeddings using OpenAI API.
The OpenAIEncoder class is a subclass of DenseEncoder and utilizes the OpenAI API to generate embeddings for given documents. It requires an OpenAI API key and supports customization of the score threshold for filtering or processing the embeddings.
default value, should be replaced by config
Initialize the OpenAIEncoder.
Arguments:
name
(str
): The name of the embedding model to use.openai_base_url
(str
): The base URL for the OpenAI API.openai_api_key
(str
): The OpenAI API key.openai_org_id
(str
): The OpenAI organization ID.score_threshold
(float
): The score threshold for the embeddings.str
0 (str
1): The dimensions of the embeddings.str
2 (str
1): The maximum number of retries for the OpenAI API call.Encode a list of text documents into embeddings using OpenAI API.
Arguments:
docs
: List of text documents to encode.truncate
: Whether to truncate the documents to token limit. If
False and a document exceeds the token limit, an error will be
raised.Returns:
List of embeddings for each document.
Encode a list of text documents into embeddings using OpenAI API asynchronously.
Arguments:
docs
: List of text documents to encode.truncate
: Whether to truncate the documents to token limit. If
False and a document exceeds the token limit, an error will be
raised.Returns:
List of embeddings for each document.