Interface GoogleVertexAIChatInput<AuthOptions>

Defines the input to the Google Vertex AI chat model.

interface GoogleVertexAIChatInput<AuthOptions> {
    apiVersion?: string;
    authOptions?: AuthOptions;
    cache?: boolean | BaseCache<Generation[]>;
    callbackManager?: CallbackManager;
    callbacks?: Callbacks;
    concurrency?: number;
    context?: string;
    customModelURL?: string;
    endpoint?: string;
    examples?: ChatExample[];
    location?: string;
    maxConcurrency?: number;
    maxOutputTokens?: number;
    maxRetries?: number;
    metadata?: Record<string, unknown>;
    model?: string;
    onFailedAttempt?: FailedAttemptHandler;
    tags?: string[];
    temperature?: number;
    topK?: number;
    topP?: number;
    verbose?: boolean;
}

Type Parameters

  • AuthOptions

Hierarchy

  • GoogleVertexAIBaseLLMInput<AuthOptions>
    • GoogleVertexAIChatInput

Properties

apiVersion? authOptions? cache? callbackManager? callbacks? concurrency? context? customModelURL? endpoint? examples? location? maxConcurrency? maxOutputTokens? maxRetries? metadata? model? onFailedAttempt? tags? temperature? topK? topP? verbose?

Properties

OptionalapiVersion

apiVersion?: string

The version of the API functions. Part of the path.

OptionalauthOptions

authOptions?: AuthOptions

Optionalcache

cache?: boolean | BaseCache<Generation[]>

OptionalcallbackManager

callbackManager?: CallbackManager

Deprecated

Use callbacks instead

Optionalcallbacks

callbacks?: Callbacks

Optionalconcurrency

concurrency?: number

Deprecated

Use maxConcurrency instead

Optionalcontext

context?: string

Instructions how the model should respond

OptionalcustomModelURL

customModelURL?: string

If you are planning to connect to a model that lives under a custom endpoint provide the "customModelURL" which will override the automatic URL building

This is necessary in cases when you want to point to a fine-tuned model or a model that has been hidden under VertexAI Endpoints.

In those cases, specifying the GoogleVertexAIModelParams.model param will not be necessary and will be ignored.

See

GoogleVertexAILLMConnection.buildUrl

Optionalendpoint

endpoint?: string

Hostname for the API call

Optionalexamples

examples?: ChatExample[]

Help the model understand what an appropriate response is

Optionallocation

location?: string

Region where the LLM is stored

OptionalmaxConcurrency

maxConcurrency?: number

The maximum number of concurrent calls that can be made. Defaults to Infinity, which means no limit.

OptionalmaxOutputTokens

maxOutputTokens?: number

Maximum number of tokens to generate in the completion.

OptionalmaxRetries

maxRetries?: number

The maximum number of retries that can be made for a single call, with an exponential backoff between each attempt. Defaults to 6.

Optionalmetadata

metadata?: Record<string, unknown>

Optionalmodel

model?: string

Model to use

OptionalonFailedAttempt

onFailedAttempt?: FailedAttemptHandler

Custom handler to handle failed attempts. Takes the originally thrown error object as input, and should itself throw an error if the input error is not retryable.

Optionaltags

tags?: string[]

Optionaltemperature

temperature?: number

Sampling temperature to use

OptionaltopK

topK?: number

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

OptionaltopP

topP?: number

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Optionalverbose

verbose?: boolean

Settings

Member Visibility

On This Page

Properties