How to do it?
1
Install UpTrain
2
Create your data
You can define your data as a list of dictionaries to run evaluations on UpTrain
question
: The question you want to askcontext
: The context relevant to the questionresponse
: The response to the question
3
Enter your AZURE API details
4
Create an EvalLLM Evaluator
The model name should start with
azure/
for UpTrain to recognize you are using models hosted on Azure.For example if you are using gpt-35-turbo
via Azure, the model name should be azure/gpt-35-turbo
5
Evaluate data using UpTrain
Now that we have our data, we can evaluate it using UpTrain. We use the
evaluate
method to do this. This method takes the following arguments:data
: The data you want to log and evaluatechecks
: The evaluations you want to perform on your data
- Context Relevance: Evaluates how relevant the retrieved context is to the question specified.
- Factual Accuracy: Evaluates whether the response generated is factually correct and grounded by the provided context.
- Response Relevance: Evaluates how relevant the generated response was to the question specified.
- Tonality: Evaluates whether the generated response matches the required persona’s tone
6
Print the results