A class that extends the BaseDocumentLoader class. It represents a document loader that loads documents from a text file. The load() method is implemented to read the text from the file or blob, parse it using the parse() method, and create a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

const loader = new TextLoader("src/document_loaders/example_data/example.txt");
const docs = await loader.load();

Hierarchy (view full)

Constructors

Properties

filePathOrBlob: string | Blob

Methods

  • A method that loads the text file or blob and returns a promise that resolves to an array of Document instances. It reads the text from the file or blob using the readFile function from the node:fs/promises module or the text() method of the blob. It then parses the text using the parse() method and creates a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

    Returns Promise<Document<Record<string, any>>[]>

    A promise that resolves to an array of Document instances.

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

  • A protected method that takes a raw string as a parameter and returns a promise that resolves to an array containing the raw text as a single element.

    Parameters

    • raw: string

      The raw text to be parsed.

    Returns Promise<string[]>

    A promise that resolves to an array containing the raw text as a single element.

  • A static method that imports the readFile function from the node:fs/promises module. It is used to dynamically import the function when needed. If the import fails, it throws an error indicating that the fs/promises module is not available in the current environment.

    Returns Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable | BufferEncoding): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

    A promise that resolves to an object containing the readFile function from the node:fs/promises module.