AccuracyEval

AccuracyEval parameters, run methods, and AccuracyResult fields.

AccuracyEval runs an agent or team against an input, then scores the response against an expected output with an evaluator agent on a 1-10 scale.

from agno.eval import AccuracyEval, AccuracyResult

Parameters

Parameter	Type	Default	Description
input	Union[str, Callable]	Required	Input sent to the agent or team. A callable must return the input string.
expected_output	Union[str, Callable]	Required	Expected answer the evaluator scores against. A callable must return the expected string.
agent	Optional[Agent]	None	Agent to evaluate. Provide exactly one of agent or team.
team	Optional[Team]	None	Team to evaluate. Provide exactly one of agent or team.
name	Optional[str]	None	Evaluation name.
eval_id	str	Autogenerated UUID	Evaluation ID.
num_iterations	int	1	Number of times the input is run and scored.
model	Optional[Model]	None	Model for the default evaluator agent. Falls back to OpenAI o4-mini when unset.
evaluator_agent	Optional[Agent]	None	Custom agent that scores the response. Replaces the default evaluator.
additional_guidelines	Optional[Union[str, List[str]]]	None	Guidelines appended to the default evaluator’s instructions.
additional_context	Optional[str]	None	Context appended to the default evaluator’s instructions.
print_summary	bool	False	Print the summary table with score statistics after the run.
print_results	bool	False	Print the per-iteration results table after the run.
show_spinner	bool	True	Render the progress spinner. Set to False when the eval runs inside code that must not write to the console.
file_path_to_save_results	Optional[str]	None	File path where the result is saved.
debug_mode	bool	False	Enable debug logs. Defaults to True when the AGNO_DEBUG environment variable is "true".
db	Optional[Union[BaseDb, AsyncBaseDb]]	None	Database where the eval run is stored.
telemetry	bool	True	Log minimal telemetry for the eval run.
result	Optional[AccuracyResult]	None	Result of the evaluation. Set after a run completes.

Methods

Method	Return type	Description
run(*, print_summary=True, print_results=True)	Optional[AccuracyResult]	Run the agent or team num_iterations times and score each response. Raises with an async database; use arun() instead.
arun(*, print_summary=True, print_results=True)	Optional[AccuracyResult]	Async version of run().
run_with_output(*, output, print_summary=True, print_results=True)	Optional[AccuracyResult]	Score the given output string instead of running the agent or team.
arun_with_output(*, output, print_summary=True, print_results=True)	Optional[AccuracyResult]	Async version of run_with_output().

AccuracyResult

Field	Type	Description
results	List[AccuracyEvaluation]	One entry per scored iteration.
avg_score	Optional[float]	Average score across iterations. None when there are no results.
mean_score	Optional[float]	Mean score across iterations. Same value as avg_score.
min_score	Optional[float]	Lowest score across iterations.
max_score	Optional[float]	Highest score across iterations.
std_dev_score	Optional[float]	Standard deviation of scores. 0 with a single result.

AccuracyResult also exposes print_summary(console=None) and print_results(console=None) to render the statistics and per-iteration tables.

AccuracyEvaluation

Each entry in AccuracyResult.results is an AccuracyEvaluation:

Field	Type	Description
input	str	Input sent to the agent or team.
output	str	Response produced by the agent or team.
expected_output	str	Expected answer the response was scored against.
score	int	Accuracy score from 1 to 10.
reason	str	The evaluator’s reasoning for the score.