Class HNLoader

caller

Optionalselector

OptionaltextDecoder

timeout

webPath

loadAndSplit

• loadAndSplit(splitter?): Promise<Document<Record<string, any>>[]>

• Parameters

  • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

  Returns Promise<Document<Record<string, any>>[]>

  A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

  Deprecated

  Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

scrape

• scrape(): Promise<CheerioAPI>

• Fetches the web document from the webPath and loads it using Cheerio. It returns a CheerioAPI instance.

  Returns Promise<CheerioAPI>

  A Promise that resolves to a CheerioAPI instance.

Staticimports

• imports(): Promise<{
      load: ((content:
          | string
          | Buffer
          | AnyNode
          | AnyNode[], options?: null | CheerioOptions, isDocument?: boolean) => CheerioAPI);
  }>

• A static method that dynamically imports the Cheerio library and returns the load function. If the import fails, it throws an error.

  Returns Promise<{
      load: ((content:
          | string
          | Buffer
          | AnyNode
          | AnyNode[], options?: null | CheerioOptions, isDocument?: boolean) => CheerioAPI);
  }>

  A Promise that resolves to an object containing the load function from the Cheerio library.

StaticscrapeAll

• scrapeAll(urls, caller, timeout, textDecoder?, options?): Promise<CheerioAPI[]>

• Fetches web documents from the given array of URLs and loads them using Cheerio. It returns an array of CheerioAPI instances.

  Parameters

  • urls: string[]

    An array of URLs to fetch and load.
  • caller: AsyncCaller
  • timeout: undefined | number
  • OptionaltextDecoder: TextDecoder
  • Optionaloptions: CheerioOptions

  Returns Promise<CheerioAPI[]>

  A Promise that resolves to an array of CheerioAPI instances.