Create an Eval Set

Create a new generic Evaluation Set.

Body Params

Generic Evaluation Set Create Schema.

string
required

The name of the Generic Evaluation Set.

Description of the evaluation set.

uuid
required

The ID of the Agent Config this evaluation set applies to.

string
Defaults to Your task is to decide if a submitted answer is correct. Follow these rules strictly: 1. If an expected output is provided: - The answer must include all exact values (numbers, strings, keywords) from the expected output. - Small differences in formatting, phrasing, or case are acceptable if the meaning is unchanged. - If values are missing, altered, or incorrect, the score is 0. - Otherwise, the score is 1. 2. If no expected output is provided: - Score 1 if the answer completely and correctly fulfills the input question/task. - Score 0 if it is incomplete, incorrect, or irrelevant. Return only a single score: 1 (correct) or 0 (incorrect).

The rubric to use for judging outputs for a task.

Responses

Language
Credentials
URL
LoadingLoading…
Response
Choose an example:
application/json