Assesses how consistent the response is with the question asked as well as with the context provided.
Response Consistency is the measure of how well the generated response aligns with both the question asked and the context provided.
In evaluating response consistency, it is important to assess whether the information provided in the response directly addresses the query posed by the user and is coherent with any additional context given.
Columns required:
question
: The question asked by the usercontext
: Information retrieved to answer the questionresponse
: The response given by the modelSample Response:
The response states the use of painkillers for the treatment of pneumonia.
This contradicts the context which states that pneumonia is typically treated with antibiotics, rest, and supportive care
Resulting in a low response consistency score.
We evaluate response consistency through the following steps:
Assesses how consistent the response is with the question asked as well as with the context provided.
Response Consistency is the measure of how well the generated response aligns with both the question asked and the context provided.
In evaluating response consistency, it is important to assess whether the information provided in the response directly addresses the query posed by the user and is coherent with any additional context given.
Columns required:
question
: The question asked by the usercontext
: Information retrieved to answer the questionresponse
: The response given by the modelSample Response:
The response states the use of painkillers for the treatment of pneumonia.
This contradicts the context which states that pneumonia is typically treated with antibiotics, rest, and supportive care
Resulting in a low response consistency score.
We evaluate response consistency through the following steps: