Tutorials
Uptrain API Client
How to run your evaluations remotely using the Uptrain API Client
Create an API Key
To get started, you will first need to get your API key from the Uptrain Dashboard.
- Login with Google
- Click on “Create API Key”
- Copy the API key and save it somewhere safe
Install the Uptrain Python Package
pip install uptrain
Create an API Client
from uptrain import APIClient, Settings
settings = Settings(
uptrain_access_token=YOUR_API_KEY,
uptrain_server_url="https://demo.uptrain.ai"
)
client = APIClient(settings)
Check if you are authenticated
client.check_auth()
Running Evaluations
Step 1: Define the dataset
Your dataset should contain the following columns:
context
: The context for the questionquestion
: The question to be answeredresponse
: The correct answer to the question
Go through the code below to learn how to create a dataset for your evaluation.
import polars as pl
data = pl.DataFrame(
{
"context": [
"Lolita is a 1962 psychological comedy-drama film directed by Stanley Kubrick. The film follows Humbert Humbert, a middle-aged literature lecturer who becomes infatuated with Dolores Haze, a young adolescent girl. It stars Sue Lyon as the titular character.",
"William Shakespeare was an English playwright and poet, widely regarded as the world's greatest dramatist. He is often called the Bard of Avon. His works consist of some 39 plays, 154 sonnets and a few other verses.",
"Sachin Tendulkar is a former international cricketer from India. He is widely regarded as one of the greatest batsmen in the history of cricket. He is the highest run scorer of all time in International cricket and played until 16 May 2013.",
"Python is a high-level general-purpose programming language. Its design philosophy emphasizes code readability. Its language constructs aim to help programmers write clear, logical code for both small and large-scale projects.",
"The Apollo program was a human spaceflight program carried out by NASA. It accomplished landing the first humans on the Moon from 1969 to 1972. The program was named after Apollo, the Greek god of light, music, and the sun. The first mission flown was dubbed as Apollo 1.",
],
"question": [
"What was the age of Sue Lyon when she played Lolita?",
"How many sonnets did Shakespeare write?",
"When did Sachin Tendulkar retire from cricket?",
"Who created the Python language?",
"Which was the first manned Apollo mission?",
],
"response": [
"The actress who played Lolita, Sue Lyon, was 14 at the time of filming.",
"Shakespeare wrote 154 sonnets.",
"Sachin Tendulkar retired from cricket in 2013.",
"Python language was created by Guido van Rossum.",
"The first manned Apollo mission was Apollo 1.",
],
}
)
Step 2: Run the evaluation
Now that we have our data, we can log it and evaluate it using UpTrain. We use the log_and_evaluate
method to do this. This method takes the following arguments:
project_name
: The name of your projectdata
: The data you want to log and evaluateevals
: The evaluations you want to perform on your data
You can find the list of all available evaluations here.
results = client.log_and_evaluate(
project_name="Sample-Project",
data=data,
checks=[Evals.CONTEXT_RELEVANCE, Evals.FACTUAL_ACCURACY, Evals.RESPONSE_RELEVANCE, CritiqueTone(persona="teacher")]
)
Step 3: View the results
print(json.dumps(results, indent=3))
Was this page helpful?