Skip to main content

TrustNest Azure OpenAI via APIM Documentation for Langchain

In this document, you will find basic examples that uses langchain connected to the TrustNest API Manager.

Prerequisites​

Ensure you have the langchain, openai and tiktoken Python libraries installed. If not, install them using pip:

pip install langchain
pip install openai
pip install tiktoken

Usage​

APIM JWT token and Azure APIM API key​

For security and tracking purposes, requests to OpenAI API must include a JWT token and a APIM subscription key.

To do so, we use the custom headers TrustNest-Apim-Subscription-Key and Authorization.

trustnest_apim_access_token = "your-trustnest-apim-access-token"
trustnest_apim_subscription_key = "your-trustnest-api-subscription-key"

Common variables​

base_url = "https://trustnest.azure-api.net/genai-aoai-inference/v1"
api_key = trustnest_apim_access_token
api_version = "2023-05-15"

Create a Chat Completion​

openai.ChatCompletion.azure_api_prefix = "" # Leave empty. This is to prevent `openai` from adding "openai" in the url thus causing a 404 error.
chat_model = AzureChatOpenAI(
    openai_api_base=base_url,
    openai_api_version=api_version,
    deployment_name="gpt35-4k",
    openai_api_key=api_key,
    openai_api_type="azure",
    model="gpt-35-turbo",
    headers = {
    "Authorization": f"Bearer {trustnest_apim_access_token}",
    "TrustNest-Apim-Subscription-Key": trustnest_apim_subscription_key,
    "Content-Type': 'application/json"
    },
)

response = chat_model(
    [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content="Implement basic merge sort algorithm using python.")
    ]
)

Get Embeddings​

openai.Embedding.azure_api_prefix = "" # Leave empty. This is to prevent `openai` from adding "openai" in the url thus causing a 404 error.
embeddings_model = OpenAIEmbeddings(
    openai_api_base=base_url,
    openai_api_version=api_version,
    model="text-embedding-ada-002",
    deployment="text-embedding-ada-002",
    openai_api_key=api_key,
    openai_api_type="azure",
    headers = {
    "Authorization": f"Bearer {trustnest_apim_access_token}",
    "TrustNest-Apim-Subscription-Key": trustnest_apim_subscription_key,
    "Content-Type': 'application/json"
    }
)

embeddings = embeddings_model.embed_query("This is a test")