Allows you to create your own set of evaluations
prompt
: Evaluation prompt used to generate the gradechoices
: List of choices/grades to choose fromchoices_scores
: Scores associated with each choiceeval_type
: One of [“classify”, “cot_classify”], determines if chain-of-thought prompting is to be applied or notprompt_var_to_column_mapping (optional)
: mapping between variables defined in the prompt vs column names in the data