Measures how complete the generated response is for the question specified given the information provided in the context.
Context Utilization score measures if the generated response has sufficiently used the retrieved context to answer the question being asked.
This check is important since a bad utilization of context reduces the effectiveness to answer all aspects of the question asked.
Columns required:
question
: The question asked by the usercontext
: Information retrieved to answer the questionresponse
: The response given by the modelSample Response:
The question can be divided in following 2 parts:
The context contains information about both the questions.
But, the response only answers 1 question while providing no information on “When was the Taj Mahal built?”
Ultimately, resulting in a low context utilization score.
We evaluate context utilization by determining which of the following three cases apply for the given task data:
Measures how complete the generated response is for the question specified given the information provided in the context.
Context Utilization score measures if the generated response has sufficiently used the retrieved context to answer the question being asked.
This check is important since a bad utilization of context reduces the effectiveness to answer all aspects of the question asked.
Columns required:
question
: The question asked by the usercontext
: Information retrieved to answer the questionresponse
: The response given by the modelSample Response:
The question can be divided in following 2 parts:
The context contains information about both the questions.
But, the response only answers 1 question while providing no information on “When was the Taj Mahal built?”
Ultimately, resulting in a low context utilization score.
We evaluate context utilization by determining which of the following three cases apply for the given task data: