A document loader that loads unstructured documents from a directory using the UnstructuredLoader. It creates a UnstructuredLoader instance for each supported file type and passes it to the DirectoryLoader constructor.

const loader = new UnstructuredDirectoryLoader("path/to/directory", {
apiKey: "MY_API_KEY",
});
const docs = await loader.load();

Hierarchy (view full)

Constructors

Properties

directoryPath: string
recursive: boolean = true
unknown: UnknownHandling = UnknownHandling.Warn

Methods

  • Loads the documents from the directory. If a file is a directory and recursive is true, it recursively loads documents from the subdirectory. If a file is a file, it checks if there is a corresponding loader function for the file extension in the loaders mapping. If there is, it loads the documents. If there is no corresponding loader function and unknown is set to Warn, it logs a warning message. If unknown is set to Error, it throws an error.

    Returns Promise<Document<Record<string, any>>[]>

    A promise that resolves to an array of loaded documents.

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

  • Imports the necessary functions from the node:path and node:fs/promises modules. It is used to dynamically import the functions when needed. If the import fails, it throws an error indicating that the modules failed to load.

    Returns Promise<{
        extname: ((path: string) => string);
        readdir: {
            (path: PathLike, options?: null | BufferEncoding | ObjectEncodingOptions & {
                withFileTypes?: false;
            }): Promise<string[]>;
            (path: PathLike, options: {
                encoding: "buffer";
                withFileTypes?: false;
            } | "buffer"): Promise<Buffer[]>;
            (path: PathLike, options?: null | BufferEncoding | ObjectEncodingOptions & {
                withFileTypes?: false;
            }): Promise<string[] | Buffer[]>;
            (path: PathLike, options: ObjectEncodingOptions & {
                withFileTypes: true;
            }): Promise<Dirent[]>;
        };
        resolve: ((...paths: string[]) => string);
    }>

    A promise that resolves to an object containing the imported functions.