TrustNest Azure OpenAI via APIM Documentation for Python

In this document, you will find basic examples that uses python OpenAI client connected to the TrustNest API Manager.

Prerequisites

Ensure you have the openai Python library installed. If not, install it using pip:

pip install openai

Usage

APIM JWT token and Azure APIM API key

For security and tracking purposes, requests to OpenAI API must include a JWT token and a APIM subscription key.

To do so, we use the custom headers TrustNest-Apim-Subscription-Key and Authorization.

Configure openai

import openai

trustnest_apim_access_token = "your-trustnest-apim-access-token"
trustnest_apim_subscription_key = "your-trustnest-api-subscription-key"

openai.api_key = "" #Leave empty
openai.api_type = "azure"
openai.api_base = "https://trustnest.azure-api.net/genai-aoai-inference/v1"
openai.api_version = "2023-05-15"

Create a Chat Completion

openai.ChatCompletion.azure_api_prefix = "" #Leave empty. This is to prevent `openai` from adding "openai" in the url thus causing a 404 error.
chat_completion = openai.ChatCompletion.create(
  model="gpt-35-turbo",
  engine="gpt35-4k",
  headers={
          'Authorization': f"Bearer {trustnest_apim_access_token}",
          'TrustNest-Apim-Subscription-Key': trustnest_apim_subscription_key,
          'Content-Type': 'application/json'
        },
  messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Implement basic merge sort algorithm using python."}
      ]
)

Get embeddings

openai.Embedding.azure_api_prefix = ""
embeddings = openai.Embedding.create(
  model="text-embedding-ada-002",
  deployment_id="text-embedding-ada-002",
  headers={
          'Authorization': f"Bearer {trustnest_apim_access_token}",
          'TrustNest-Apim-Subscription-Key': trustnest_apim_subscription_key,
          'Content-Type': 'application/json'
        },
  input="This is a test.",
)