WEB Crawler API

The WEB crawler indexes Web sites.

• Starting the crawl
• Stopping the crawl
• Getting the status of the crawler
• Getting the status of all crawler
• Crawling one URL
• Inclusion patterns
  • Insert
  • List
  • Delete
• Exclusion patterns
  • Insert
  • List
  • Delete
• Site map
  • Insert
  • List
  • Delete
• URL Database
  • Hostname list
  • Injecting URLs
  • Delete all URLs

---

View/edit on GitHub

---