Type Alias CriteriaEvalChainConfig

CriteriaEvalChainConfig: {
    agentTools?: undefined | StructuredToolInterface<ZodObjectAny>[];
    chainOptions?: undefined | Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface<any, BaseLanguageModelCallOptions>>, "llm">>;
    criteria?: undefined | (CriteriaLike & (Record<string, string> | Criteria));
    distanceMetric?: undefined | EmbeddingDistanceType;
    embedding?: undefined | EmbeddingsInterface;
    evaluatorType: "criteria";
    feedbackKey?: undefined | string;
    formatEvaluatorInputs: EvaluatorInputFormatter;
    llm?: undefined | BaseLanguageModelInterface<any, BaseLanguageModelCallOptions> & BaseLanguageModel<any, BaseLanguageModelCallOptions>;
}

Type declaration

  • OptionalagentTools?: undefined | StructuredToolInterface<ZodObjectAny>[]

    A list of tools available to the agent, for TrajectoryEvalChain.

  • OptionalchainOptions?: undefined | Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface<any, BaseLanguageModelCallOptions>>, "llm">>
  • Optionalcriteria?: undefined | (CriteriaLike & (Record<string, string> | Criteria))

    The criteria to use for the evaluator.

  • OptionaldistanceMetric?: undefined | EmbeddingDistanceType

    The distance metric to use for comparing the embeddings.

  • Optionalembedding?: undefined | EmbeddingsInterface

    The embedding objects to vectorize the outputs.

  • evaluatorType: "criteria"
  • OptionalfeedbackKey?: undefined | string

    The feedback (or metric) name to use for the logged evaluation results. If none provided, we default to the evaluationName.

  • formatEvaluatorInputs: EvaluatorInputFormatter

    Convert the evaluation data into formats that can be used by the evaluator. This should most commonly be a string. Parameters are the raw input from the run, the raw output, raw reference output, and the raw run.

    Example

    // Chain input: { input: "some string" }
    // Chain output: { output: "some output" }
    // Reference example output format: { output: "some reference output" }
    const formatEvaluatorInputs = ({
      rawInput,
      rawPrediction,
      rawReferenceOutput,
    }) => {
      return {
        input: rawInput.input,
        prediction: rawPrediction.output,
        reference: rawReferenceOutput.output,
      };
    };

    Returns

    The prepared data.

  • Optionalllm?: undefined | BaseLanguageModelInterface<any, BaseLanguageModelCallOptions> & BaseLanguageModel<any, BaseLanguageModelCallOptions>

Settings

Member Visibility