Class that extends the TextLoader class. It represents a document loader that loads documents from JSON Lines files. It has a constructor that takes a filePathOrBlob parameter representing the path to the JSON Lines file or a Blob object, and a pointer parameter that specifies the JSON pointer to extract.

Hierarchy (view full)

Constructors

Properties

filePathOrBlob: string | Blob
pointer: string

Methods

  • A method that loads the text file or blob and returns a promise that resolves to an array of Document instances. It reads the text from the file or blob using the readFile function from the node:fs/promises module or the text() method of the blob. It then parses the text using the parse() method and creates a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

    Returns Promise<Document<Record<string, any>>[]>

    A promise that resolves to an array of Document instances.

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

  • Method that takes a raw string as a parameter and returns a promise that resolves to an array of strings. It parses the raw JSON Lines string, splits it into lines, parses each line as JSON, and extracts the values based on the specified JSON pointer.

    Parameters

    • raw: string

      The raw JSON Lines string to parse.

    Returns Promise<string[]>

    A promise that resolves to an array of strings.

  • A static method that imports the readFile function from the node:fs/promises module. It is used to dynamically import the function when needed. If the import fails, it throws an error indicating that the fs/promises module is not available in the current environment.

    Returns Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable | BufferEncoding): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

    A promise that resolves to an object containing the readFile function from the node:fs/promises module.