Web Crawler
Rank #34229pulsemcp/jitsmaster-web-crawler
Configurable web crawler that extracts structured content from websites while respecting robots.txt rules and offering customizable settings for depth, delay, and concurrency.
Web Crawler is a Model Context Protocol (MCP) server published by jitsmaster. It ranks #34229 of 58,832 servers tracked on MCP Toplist, and its repository has 1 GitHub stars. Web Crawler is listed on PulseMCP, and ships as a single rolling release with no explicit version metadata. It was first listed on Jan 10, 2025.
Ranks ahead of 24,603 of 58,832 servers on MCP Toplist.
Use Web Crawler
Web Crawler doesn’t publish a machine-readable install config. Follow the setup instructions in its repository ↗, or open one of its registry listings above.
Listed on 1 registry
jitsmaster
Not versioned
This server is published through a registry that does not expose explicit version metadata, and no GitHub release tags were found on the linked repository. The listing tracks a single rolling release.
Frequently asked questions
- What is Web Crawler?
- Configurable web crawler that extracts structured content from websites while respecting robots.txt rules and offering customizable settings for depth, delay, and concurrency.
- Who maintains Web Crawler?
- Web Crawler is maintained by jitsmaster, which publishes 1 MCP server (0 total versions) tracked on MCP Toplist.
- Is Web Crawler an official MCP server?
- Web Crawler is not on the Official MCP Registry. It is listed on PulseMCP.
- How many versions does Web Crawler have?
- Web Crawler ships as a single rolling release with no explicit version metadata.
- Where can I find the source code for Web Crawler?
- The source code for Web Crawler is hosted at github.com/jitsmaster/webscrapemcpserver.