Class PuppeteerWebBaseLoader

Deprecated

  • Import from "@langchain/community/document_loaders/web/puppeteer" instead. This entrypoint will be removed in 0.3.0.

Class that extends the BaseDocumentLoader class and implements the DocumentLoader interface. It represents a document loader for scraping web pages using Puppeteer.

Example

const loader = new PuppeteerWebBaseLoader("https:exampleurl.com", {
  launchOptions: {
    headless: true,
  },
  gotoOptions: {
    waitUntil: "domcontentloaded",
  },
});
const screenshot = await loader.screenshot();

Hierarchy (view full)

Implements

Constructors

constructor

Properties

options webPath

Methods

load loadAndSplit scrape screenshot imports

Constructors

constructor

Properties

options

options: undefined | PuppeteerWebBaseLoaderOptions

webPath

webPath: string

Methods

load

loadAndSplit

  • loadAndSplit(splitter?): Promise<Document<Record<string, any>>[]>

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Deprecated

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

scrape

  • scrape(): Promise<string>

  • Method that calls the _scrape method to perform the scraping of the web page specified by the webPath property.

    Returns Promise<string>

    Promise that resolves to the scraped HTML content of the web page.

screenshot

  • screenshot(): Promise<Document<Record<string, any>>>

  • Screenshot a web page and return it as a Document object where the pageContent property is the screenshot encoded in base64.

    Returns Promise<Document<Record<string, any>>>

    A document object containing the screenshot of the page encoded in base64.

Staticimports

  • imports(): Promise<{
        launch: ((options?: PuppeteerLaunchOptions) => Promise<Browser>);
    }>

  • Static method that imports the necessary Puppeteer modules. It returns a Promise that resolves to an object containing the imported modules.

    Returns Promise<{
        launch: ((options?: PuppeteerLaunchOptions) => Promise<Browser>);
    }>

    Promise that resolves to an object containing the imported Puppeteer modules.

Settings

Member Visibility

On This Page

Constructors
Properties
Methods