Harvest::Controller::URLFilter::HostLimit - limit on number of hosts to crawl | |
| |
DESCRIPTION | |
HostLimit places a maxmimum limit upon the number of hosts to access for a Rootnode. A ``host'' is defined as a unique netloc portion of the URL - that is the combination of the scheme (http, ftp, ...), the hostname, and the port. This filter is most commonly used with a limit of '1' to restrict gathering to the same host as the rootnode.
| |
PARAMETERS | |
HostLimit takes 1 parameter - the maximum number of hosts to index. A limit of 1 will restrict the gathering to the same server as the rootnode. | |