A class that extends the BaseDocumentLoader and implements the DocumentLoader interface. It represents a document loader that loads documents from an Apify dataset.

const loader = new ApifyDatasetLoader("your-dataset-id", {
datasetMappingFunction: (item) =>
new Document({
pageContent: item.text || "",
metadata: { source: item.url },
}),
clientOptions: {
token: "your-apify-token",
},
});

const docs = await loader.load();

const chain = new RetrievalQAChain();
const res = await chain.invoke({ query: "What is LangChain?" });

console.log(res.text);
console.log(res.sourceDocuments.map((d) => d.metadata.source));

Type Parameters

  • Metadata extends Record<string, any>

Hierarchy (view full)

Implements

Constructors

Properties

apifyClient: ApifyClient
caller: AsyncCaller
datasetId: string
datasetMappingFunction: ApifyDatasetMappingFunction<Metadata>

Methods

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.