​
Create an API Key

To get started, you will first need to get your API key from the Uptrain Dashboard.

  1. Login with Google
  2. Click on “Create API Key”
  3. Copy the API key and save it somewhere safe

​
Install the Uptrain Python Package

pip install uptrain

​
Create an API Client

from uptrain import APIClient, Settings

settings = Settings(
    uptrain_access_token=YOUR_API_KEY,
    uptrain_server_url="https://demo.uptrain.ai"
)

client = APIClient(settings)

​
Check if you are authenticated

client.check_auth()

​
Running Evaluations

​
Step 1: Define the dataset

Your dataset should contain the following columns:

  • context: The context for the question
  • question: The question to be answered
  • response: The correct answer to the question

Go through the code below to learn how to create a dataset for your evaluation.

import polars as pl

data = pl.DataFrame(
    {
        "context": [
            "Lolita is a 1962 psychological comedy-drama film directed by Stanley Kubrick. The film follows Humbert Humbert, a middle-aged literature lecturer who becomes infatuated with Dolores Haze, a young adolescent girl. It stars Sue Lyon as the titular character.",
            "William Shakespeare was an English playwright and poet, widely regarded as the world's greatest dramatist. He is often called the Bard of Avon. His works consist of some 39 plays, 154 sonnets and a few other verses.",
            "Sachin Tendulkar is a former international cricketer from India. He is widely regarded as one of the greatest batsmen in the history of cricket. He is the highest run scorer of all time in International cricket and played until 16 May 2013.",
            "Python is a high-level general-purpose programming language. Its design philosophy emphasizes code readability. Its language constructs aim to help programmers write clear, logical code for both small and large-scale projects.",
            "The Apollo program was a human spaceflight program carried out by NASA. It accomplished landing the first humans on the Moon from 1969 to 1972. The program was named after Apollo, the Greek god of light, music, and the sun. The first mission flown was dubbed as Apollo 1.",
        ],
        "question": [
            "What was the age of Sue Lyon when she played Lolita?",
            "How many sonnets did Shakespeare write?",
            "When did Sachin Tendulkar retire from cricket?",
            "Who created the Python language?",
            "Which was the first manned Apollo mission?",
        ],
        "response": [
            "The actress who played Lolita, Sue Lyon, was 14 at the time of filming.",
            "Shakespeare wrote 154 sonnets.",
            "Sachin Tendulkar retired from cricket in 2013.",
            "Python language was created by Guido van Rossum.",
            "The first manned Apollo mission was Apollo 1.",
        ],
    }
)

​
Step 2: Run the evaluation

Now that we have our data, we can log it and evaluate it using UpTrain. We use the log_and_evaluate method to do this. This method takes the following arguments:

  • project_name: The name of your project
  • data: The data you want to log and evaluate
  • evals: The evaluations you want to perform on your data

You can find the list of all available evaluations here.

results = client.log_and_evaluate(
    project_name="Sample-Project",
    data=data,
    checks=[Evals.CONTEXT_RELEVANCE, Evals.FACTUAL_ACCURACY, Evals.RESPONSE_RELEVANCE, CritiqueTone(persona="teacher")]
)

​
Step 3: View the results

print(json.dumps(results, indent=3))

Was this page helpful?

Open Source EvaluatorExperiments Evaluation Demo