Interface CrawlerStrategy<A>

Type Parameters

A extends {
[key: string]: CrawlerActor<unknown>;
}

Hierarchy

CrawlerStrategy

Index

Properties

actors context? depth entry init? limiter? post? pre? prune?

Properties

actors

actors: A

The actors that will be used to crawl.

`Optional` context

context?: BrowserContextOptions

The context options to use.

depth

depth: number

The maximum depth.

entry

entry: string

An absolute URL, the entry point.

`Optional` init

init?: ((page: Page) => void | Promise<void>)

Type declaration

- (page: Page): void | Promise<void>
- Do something to the page when it is created.
  Parameters
  - page: Page
  Returns void | Promise<void>

`Optional` limiter

limiter?: RateLimiter

The rate limiter to use, including concurrency control.

`Optional` post

post?: ((page: Page) => void | Promise<void>)

Type declaration

- (page: Page): void | Promise<void>
- Do something to the page after a task is run.
  Parameters
  - page: Page
  Returns void | Promise<void>

`Optional` pre

pre?: ((page: Page) => void | Promise<void>)

Type declaration

- (page: Page): void | Promise<void>
- Do something to the page before a task is run.
  Parameters
  - page: Page
  Returns void | Promise<void>

`Optional` prune

prune?: ((url: string) => boolean)

Type declaration

- (url: string): boolean
- Pre-pruning function. Return true to skip crawling a URL before it is loaded.
  Parameters
  - url: string
  Returns boolean

Generated using TypeDoc