|
|
|
|
|
This software was checked for viruses 4 days ago and was found to contain no viruses.
Click here
to see antivirus report.
|
|
| |
|
Review
Crawl a website with ActiveX Spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.
Spider ActiveX Component for Crawling the Web:
-Cache pages so future crawls can fetch from cache;
-Robots.txt compliant;
-Fetch the HTML content of each page crawled;
-Able to crawl HTTPS pages;
-Define "avoid" patterns to avoid URLs matching specific wildcard patterns;
-Define "avoid" patterns for avoiding matching outbound links. |
|
Report a wrong version
Report a wrong program title
Suggest a correction
|
Screenshots
|
|
