SERVER

polite-crawler

Rank #4,627

io.github.mark1russell7/polite-crawler

A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

10 versions

by mark1russell7 (1 servers)View repository ↗

First listed

Oct 13, 2026

Last publish date

Oct 25, 2026

PURPOSE

What you can do with polite-crawler

polite-crawler is a Model Context Protocol (MCP) server published by mark1russell7. It ranks #4,627 of 163,719 active servers on MCP Toplist. polite-crawler is listed on Official MCP, with 10 tracked versions. It was first listed on Oct 13, 2026 and most recently updated on Oct 25, 2026.

Not verifiedMCP Toplist has not yet written a checked profile for this server, so the paragraph above is assembled from registry data only. The repository README is the authority on what it does.

CAPABILITIES AND LIMITATIONS

What polite-crawler can and cannot do

Not verifiedMCP Toplist has not catalogued this server's tools or limitations. The repository README lists them.

REQUIREMENTS

What you need before using polite-crawler

DeclaredNo credentialsNo credentials declared

The registry listing declares no required environment variables or authentication. This is declared metadata, not a verified install — the repository README is the authority.

Declared by the Official MCP Registry, last synced Oct 30, 2026. Not independently verified by MCP Toplist.

Declared in the registry install config

POLITE_CRAWLER_DIR · optional
POLITE_CRAWLER_CONTACT · optional
POLITE_CRAWLER_ALLOW_PRIVATE · optional
POLITE_CRAWLER_ALLOW_USER_PROVIDED · optional
POLITE_CRAWLER_SIGN_KEY · optional
POLITE_CRAWLER_SIGNATURE_AGENT · optional

INSTALLATION AND FIRST TASK

Connect polite-crawler and confirm it works

npm package

claude mcp add polite-crawler -e "POLITE_CRAWLER_DIR=<your POLITE_CRAWLER_DIR>" -e "POLITE_CRAWLER_CONTACT=<your POLITE_CRAWLER_CONTACT>" -e "POLITE_CRAWLER_ALLOW_PRIVATE=<your POLITE_CRAWLER_ALLOW_PRIVATE>" -e "POLITE_CRAWLER_ALLOW_USER_PROVIDED=<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>" -e "POLITE_CRAWLER_SIGN_KEY=<your POLITE_CRAWLER_SIGN_KEY>" -e "POLITE_CRAWLER_SIGNATURE_AGENT=<your POLITE_CRAWLER_SIGNATURE_AGENT>" -- npx -y polite-crawler-mcp

Replace each placeholder with your own value before saving.

npm package

[mcp_servers.polite-crawler]
command = "npx"
args = ["-y", "polite-crawler-mcp"]

[mcp_servers.polite-crawler.env]
POLITE_CRAWLER_DIR = "<your POLITE_CRAWLER_DIR>"
POLITE_CRAWLER_CONTACT = "<your POLITE_CRAWLER_CONTACT>"
POLITE_CRAWLER_ALLOW_PRIVATE = "<your POLITE_CRAWLER_ALLOW_PRIVATE>"
POLITE_CRAWLER_ALLOW_USER_PROVIDED = "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>"
POLITE_CRAWLER_SIGN_KEY = "<your POLITE_CRAWLER_SIGN_KEY>"
POLITE_CRAWLER_SIGNATURE_AGENT = "<your POLITE_CRAWLER_SIGNATURE_AGENT>"

Add to ~/.codex/config.toml. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ],
      "env": {
        "POLITE_CRAWLER_DIR": "<your POLITE_CRAWLER_DIR>",
        "POLITE_CRAWLER_CONTACT": "<your POLITE_CRAWLER_CONTACT>",
        "POLITE_CRAWLER_ALLOW_PRIVATE": "<your POLITE_CRAWLER_ALLOW_PRIVATE>",
        "POLITE_CRAWLER_ALLOW_USER_PROVIDED": "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>",
        "POLITE_CRAWLER_SIGN_KEY": "<your POLITE_CRAWLER_SIGN_KEY>",
        "POLITE_CRAWLER_SIGNATURE_AGENT": "<your POLITE_CRAWLER_SIGNATURE_AGENT>"
      }
    }
  }
}

Add to claude_desktop_config.json (Settings → Developer → Edit Config). Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ],
      "env": {
        "POLITE_CRAWLER_DIR": "<your POLITE_CRAWLER_DIR>",
        "POLITE_CRAWLER_CONTACT": "<your POLITE_CRAWLER_CONTACT>",
        "POLITE_CRAWLER_ALLOW_PRIVATE": "<your POLITE_CRAWLER_ALLOW_PRIVATE>",
        "POLITE_CRAWLER_ALLOW_USER_PROVIDED": "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>",
        "POLITE_CRAWLER_SIGN_KEY": "<your POLITE_CRAWLER_SIGN_KEY>",
        "POLITE_CRAWLER_SIGNATURE_AGENT": "<your POLITE_CRAWLER_SIGNATURE_AGENT>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (per project). Replace each placeholder with your own value before saving.

npm package

{
  "servers": {
    "polite-crawler": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ],
      "env": {
        "POLITE_CRAWLER_DIR": "<your POLITE_CRAWLER_DIR>",
        "POLITE_CRAWLER_CONTACT": "<your POLITE_CRAWLER_CONTACT>",
        "POLITE_CRAWLER_ALLOW_PRIVATE": "<your POLITE_CRAWLER_ALLOW_PRIVATE>",
        "POLITE_CRAWLER_ALLOW_USER_PROVIDED": "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>",
        "POLITE_CRAWLER_SIGN_KEY": "<your POLITE_CRAWLER_SIGN_KEY>",
        "POLITE_CRAWLER_SIGNATURE_AGENT": "<your POLITE_CRAWLER_SIGNATURE_AGENT>"
      }
    }
  }
}

Save as .vscode/mcp.json in your workspace. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ],
      "env": {
        "POLITE_CRAWLER_DIR": "<your POLITE_CRAWLER_DIR>",
        "POLITE_CRAWLER_CONTACT": "<your POLITE_CRAWLER_CONTACT>",
        "POLITE_CRAWLER_ALLOW_PRIVATE": "<your POLITE_CRAWLER_ALLOW_PRIVATE>",
        "POLITE_CRAWLER_ALLOW_USER_PROVIDED": "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>",
        "POLITE_CRAWLER_SIGN_KEY": "<your POLITE_CRAWLER_SIGN_KEY>",
        "POLITE_CRAWLER_SIGNATURE_AGENT": "<your POLITE_CRAWLER_SIGNATURE_AGENT>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ],
      "env": {
        "POLITE_CRAWLER_DIR": "<your POLITE_CRAWLER_DIR>",
        "POLITE_CRAWLER_CONTACT": "<your POLITE_CRAWLER_CONTACT>",
        "POLITE_CRAWLER_ALLOW_PRIVATE": "<your POLITE_CRAWLER_ALLOW_PRIVATE>",
        "POLITE_CRAWLER_ALLOW_USER_PROVIDED": "<your POLITE_CRAWLER_ALLOW_USER_PROVIDED>",
        "POLITE_CRAWLER_SIGN_KEY": "<your POLITE_CRAWLER_SIGN_KEY>",
        "POLITE_CRAWLER_SIGNATURE_AGENT": "<your POLITE_CRAWLER_SIGNATURE_AGENT>"
      }
    }
  }
}

Replace each placeholder with your own value before saving.

First successful task

After adding the config, ask your client to list this server's tools. A tool list confirms the connection; an empty list or an error means the server did not start or needs credentials.

MAINTENANCE AND EVIDENCE

What changed, and when this was checked

First listed: Oct 13, 2026
Latest version published: Oct 25, 2026
Version history: 10 versions tracked (below)
Requirements evidence: Declared by the Official MCP Registry, last synced Oct 30, 2026. Not independently verified by MCP Toplist.
Profile checked: Not yet — purpose, capabilities and requirements above come from registry data only
Ranking data as of: Oct 25, 2026

Registry sightings

Official MCP ↗listed since Oct 14, 2026 · still listed

Do you maintain polite-crawler? to correct or enrich this profile.

VERSIONS

Available versions (10)

Version	Published
0.2.4	Oct 25, 2026
0.2.3	Oct 25, 2026
0.2.2	Oct 25, 2026
0.2.1	Oct 20, 2026
0.2.0	Oct 19, 2026
0.1.4	Oct 18, 2026
0.1.3	Oct 18, 2026
0.1.2	Oct 17, 2026
0.1.1	Oct 17, 2026
0.1.0	Oct 13, 2026

STANDING

#4,627of 163,719 active serversTop 5%

Ranks ahead of 159,092 of 163,719 servers on MCP Toplist.

BADGE

Show your rank

Maintain this server? Add the live rank badge to your README — it updates automatically as the leaderboard changes.

Markdown

[![MCP Toplist](https://mcptoplist.com/badge/io.github.mark1russell7%2Fpolite-crawler.svg)](https://mcptoplist.com/server/io.github.mark1russell7%2Fpolite-crawler)

HTML

<a href="https://mcptoplist.com/server/io.github.mark1russell7%2Fpolite-crawler"><img src="https://mcptoplist.com/badge/io.github.mark1russell7%2Fpolite-crawler.svg" alt="MCP Toplist: Top 5% of 163,719" /></a>

Variants: append ?metric=score or ?metric=stars to the image URL.

FAQ

Frequently asked questions

Who maintains polite-crawler?: polite-crawler is maintained by mark1russell7, which publishes 1 MCP server (10 total versions) tracked on MCP Toplist.
Is polite-crawler listed on the Official MCP Registry?: Yes — polite-crawler is listed on the Official MCP Registry.
How many versions does polite-crawler have?: MCP Toplist tracks 10 versions of polite-crawler, most recently published on Oct 25, 2026.
Where can I find the source code for polite-crawler?: The source code for polite-crawler is hosted at github.com/mark1russell7/crawler.

Used in projects

Do you run this server?

Community projects list the MCP servers behind real agents and apps. Publish yours and this page will link to it.

EXPLORE

All MCP Servers Official MCP Servers GitHub MCP Servers Best MCP Servers Most Active MCP Servers New MCP Servers Best New MCP Servers