A powerful web crawler utility for webmasters to develop search directory, web site promotion.
It extracts URL, meta tag (title, desc, keyword), body text, page size, last modified date value from web site, search results or list of URLs. High speed, multi-threaded, accurate extraction - directly saves data to disk file. Program has numerous filters to restrict session, like - URL filter, text filter, data filter, domain filter, date modified, etc.
It allows user-selectable recursion levels, retrieval threads, timeout, proxy support and many other options. |
