> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Autoevals TypeScript API

> TypeScript API reference for Autoevals v0.0.131

AutoEvals is a tool to quickly and easily evaluate AI model outputs.

## Installation

<CodeGroup>
  ```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  npm install autoevals
  ```

  ```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  pnpm add autoevals
  ```
</CodeGroup>

## RAGAS Evaluators

### AnswerCorrectness

Measures answer correctness compared to ground truth using a weighted
average of factuality and semantic similarity.

<ParamField path="args" type="ScorerArgs" />

### AnswerRelevancy

Scores the relevancy of the generated answer to the given question.
Answers with incomplete, redundant or unnecessary information are penalized.

<ParamField path="args" type="ScorerArgs" />

### AnswerSimilarity

Scores the semantic similarity between the generated answer and ground truth.

<ParamField path="args" type="ScorerArgs" />

### ContextEntityRecall

Estimates context recall by estimating TP and FN using annotated answer and
retrieved context.

<ParamField path="args" type="ScorerArgs" />

### ContextPrecision

ContextPrecision evaluator function.

<ParamField path="args" type="ScorerArgs" />

### ContextRecall

ContextRecall evaluator function.

<ParamField path="args" type="ScorerArgs" />

### ContextRelevancy

ContextRelevancy evaluator function.

<ParamField path="args" type="ScorerArgs" />

### Faithfulness

Measures factual consistency of the generated answer with the given context.

<ParamField path="args" type="ScorerArgs" />

## LLM Evaluators

### Battle

Test whether an output *better* performs the `instructions` than the original
(expected) value.

<ParamField path="args" type="ScorerArgs" />

### ClosedQA

Test whether an output answers the `input` using knowledge built into the model.
You can specify `criteria` to further constrain the answer.

<ParamField path="args" type="ScorerArgs" />

### Factuality

Test whether an output is factual, compared to an original (`expected`) value.

<ParamField path="args" type="ScorerArgs" />

### Humor

Test whether an output is funny.

<ParamField path="args" type="ScorerArgs" />

### Possible

Test whether an output is a possible solution to the challenge posed in the input.

<ParamField path="args" type="ScorerArgs" />

### Security

Test whether an output is malicious.

<ParamField path="args" type="ScorerArgs" />

### Sql

Test whether a SQL query is semantically the same as a reference (output) query.

<ParamField path="args" type="ScorerArgs" />

### Summary

Test whether an output is a better summary of the `input` than the original (`expected`) value.

<ParamField path="args" type="ScorerArgs" />

### Translation

Test whether an `output` is as good of a translation of the `input` in the specified `language`
as an expert (`expected`) value.

<ParamField path="args" type="ScorerArgs" />

## String Evaluators

### EmbeddingSimilarity

A scorer that uses cosine similarity to compare two strings.

<ParamField path="args" type="ScorerArgs" />

### ExactMatch

A simple scorer that tests whether two values are equal. If the value is an object or array,
it will be JSON-serialized and the strings compared for equality.

<ParamField path="args" type="reflection" />

### Levenshtein

A simple scorer that uses the Levenshtein distance to compare two strings.

<ParamField path="args" type="reflection" />

### LevenshteinScorer

LevenshteinScorer evaluator function.

<ParamField path="args" type="reflection" />

## JSON Evaluators

### JSONDiff

A simple scorer that compares JSON objects, using a customizable comparison method for strings
(defaults to Levenshtein) and numbers (defaults to NumericDiff).

<ParamField path="args" type="ScorerArgs" />

### ValidJSON

A binary scorer that evaluates the validity of JSON output, optionally validating against a
JSON Schema definition (see [https://json-schema.org/learn/getting-started-step-by-step#create](https://json-schema.org/learn/getting-started-step-by-step#create)).

<ParamField path="args" type="ScorerArgs" />

## Custom Evaluators

### LLMClassifierFromSpec

LLMClassifierFromSpec evaluator function.

<ParamField path="name" type="string" />

<ParamField path="spec" type="reflection" />

### LLMClassifierFromSpecFile

LLMClassifierFromSpecFile evaluator function.

<ParamField path="name" type="string" />

<ParamField path="templateName" type="literal | literal | literal | literal | literal | literal | literal | literal | literal" />

### LLMClassifierFromTemplate

LLMClassifierFromTemplate evaluator function.

<ParamField path="__namedParameters" type="reflection" />

### OpenAIClassifier

OpenAIClassifier evaluator function.

<ParamField path="args" type="ScorerArgs" />

### buildClassificationTools

buildClassificationTools evaluator function.

<ParamField path="useCoT" type="boolean" />

<ParamField path="choiceStrings" type="array" />

## List Evaluators

### ListContains

A scorer that semantically evaluates the overlap between two lists of strings. It works by
computing the pairwise similarity between each element of the output and the expected value,
and then using Linear Sum Assignment to find the best matching pairs.

<ParamField path="args" type="ScorerArgs" />

## Moderation

### Moderation

A scorer that uses OpenAI's moderation API to determine if AI response contains ANY flagged content.

<ParamField path="args" type="ScorerArgs" />

## Numeric Evaluators

### NumericDiff

A simple scorer that compares numbers by normalizing their difference.

<ParamField path="args" type="reflection" />

## Configuration

### init

init evaluator function.

<ParamField path="__namedParameters" type="reflection" />

## Utilities

### makePartial

makePartial evaluator function.

<ParamField path="fn" type="Scorer" />

<ParamField path="name" type="string" />

### normalizeValue

normalizeValue evaluator function.

<ParamField path="value" type="unknown" />

<ParamField path="maybeObject" type="boolean" />

## Source Code

For the complete TypeScript source code and additional examples, visit the [autoevals GitHub repository](https://github.com/braintrustdata/autoevals).