Class TextLoader

A class that extends the BaseDocumentLoader class. It represents a document loader that loads documents from a text file. The load() method is implemented to read the text from the file or blob, parse it using the parse() method, and create a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

Example

const loader = new TextLoader("src/document_loaders/example_data/example.txt");
const docs = await loader.load();

Hierarchy (view full)

Constructors

constructor

Properties

filePathOrBlob

Methods

load loadAndSplit parse imports

Constructors

constructor

Properties

filePathOrBlob

filePathOrBlob: string | Blob

Methods

load

  • load(): Promise<Document<Record<string, any>>[]>

  • A method that loads the text file or blob and returns a promise that resolves to an array of Document instances. It reads the text from the file or blob using the readFile function from the node:fs/promises module or the text() method of the blob. It then parses the text using the parse() method and creates a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

    Returns Promise<Document<Record<string, any>>[]>

    A promise that resolves to an array of Document instances.

loadAndSplit

  • loadAndSplit(splitter?): Promise<Document<Record<string, any>>[]>

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Deprecated

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

Protectedparse

  • parse(raw): Promise<string[]>

  • A protected method that takes a raw string as a parameter and returns a promise that resolves to an array containing the raw text as a single element.

    Parameters

    • raw: string

      The raw text to be parsed.

    Returns Promise<string[]>

    A promise that resolves to an array containing the raw text as a single element.

Staticimports

  • imports(): Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable | BufferEncoding): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

  • A static method that imports the readFile function from the node:fs/promises module. It is used to dynamically import the function when needed. If the import fails, it throws an error indicating that the fs/promises module is not available in the current environment.

    Returns Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable | BufferEncoding): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

    A promise that resolves to an object containing the readFile function from the node:fs/promises module.

Settings

Member Visibility

On This Page

Constructors
Properties
Methods