Language Quality Evals
Tonality
Evaluates whether the generated response matches the required persona’s tone
Tonality Score evaluates the response in terms of the tone used when following or deviating from standard guidelines.
It aims to ensure that the generated response not only adheres to guidelines but also communicates its adherence or deviations in an appropriate and respectful manner.
Columns required:
response
: The response given by the modelllm_persona
: The persona the LLM being assessed was exposed to follow
How to use it?
By default, we are using GPT 3.5 Turbo for evaluations. If you want to use a different model, check out this tutorial.
Sample Response:
A higher tonality score reflects that the generated response aligns with intended persona.
The tone of the generated response does not align with the expected tone that a “methodical teacher” would follow.
Resulting in low tonality scores.
How it works?
We evaluate tonality by determining which of the following three cases apply for the given task data:
- The response aligns well with the specified persona.
- The response aligns moderately with the specified persona.
- The response doesn’t align with the specified persona at all.
Was this page helpful?