from uptrain import APIClient, Settings
settings = Settings(
uptrain_access_token=YOUR_API_KEY,
uptrain_server_url="https://demo.uptrain.ai"
)
client = APIClient(settings)
Go through the code below to learn how to create a dataset for your evaluation.
import polars as pl
data = pl.DataFrame(
{
"context": [
"Lolita is a 1962 psychological comedy-drama film directed by Stanley Kubrick. The film follows Humbert Humbert, a middle-aged literature lecturer who becomes infatuated with Dolores Haze, a young adolescent girl. It stars Sue Lyon as the titular character.",
"William Shakespeare was an English playwright and poet, widely regarded as the world's greatest dramatist. He is often called the Bard of Avon. His works consist of some 39 plays, 154 sonnets and a few other verses.",
"Sachin Tendulkar is a former international cricketer from India. He is widely regarded as one of the greatest batsmen in the history of cricket. He is the highest run scorer of all time in International cricket and played until 16 May 2013.",
"Python is a high-level general-purpose programming language. Its design philosophy emphasizes code readability. Its language constructs aim to help programmers write clear, logical code for both small and large-scale projects.",
"The Apollo program was a human spaceflight program carried out by NASA. It accomplished landing the first humans on the Moon from 1969 to 1972. The program was named after Apollo, the Greek god of light, music, and the sun. The first mission flown was dubbed as Apollo 1.",
],
"question": [
"What was the age of Sue Lyon when she played Lolita?",
"How many sonnets did Shakespeare write?",
"When did Sachin Tendulkar retire from cricket?",
"Who created the Python language?",
"Which was the first manned Apollo mission?",
],
"response": [
"The actress who played Lolita, Sue Lyon, was 14 at the time of filming.",
"Shakespeare wrote 154 sonnets.",
"Sachin Tendulkar retired from cricket in 2013.",
"Python language was created by Guido van Rossum.",
"The first manned Apollo mission was Apollo 1.",
],
}
)
Now that we have our data, we can log it and evaluate it using UpTrain. We use the log_and_evaluate method to do this. This method takes the following arguments:
results = client.log_and_evaluate(
project_name="Sample-Project",
data=data,
checks=[Evals.CONTEXT_RELEVANCE, Evals.FACTUAL_ACCURACY, Evals.RESPONSE_RELEVANCE, CritiqueTone(persona="teacher")]
)
print(json.dumps(results, indent=3))
