SERVER

evalsmith-mcp

Rank #21,542

io.github.diegoperezg7/evalsmith-mcp

Evaluate RAG retrieval, agent tool calls and prompt regressions with deterministic metrics.

1 version

1 GitHub stars

by diegoperezg7 (2 servers)View repository ↗

First listed

Oct 23, 2026

Last publish date

Oct 28, 2026

PURPOSE

What you can do with evalsmith-mcp

evalsmith-mcp is a Model Context Protocol (MCP) server published by diegoperezg7. It ranks #21,542 of 171,126 active servers on MCP Toplist, and its repository has 1 GitHub stars. evalsmith-mcp is listed across 2 registries — Official MCP and Glama, with 1 tracked version. It was first listed on Oct 23, 2026 and most recently updated on Oct 28, 2026.

Not verifiedMCP Toplist has not yet written a checked profile for this server, so the paragraph above is assembled from registry data only. The repository README is the authority on what it does.

CAPABILITIES AND LIMITATIONS

What evalsmith-mcp can and cannot do

Not verifiedMCP Toplist has not catalogued this server's tools or limitations. The repository README lists them.

REQUIREMENTS

What you need before using evalsmith-mcp

DeclaredNo credentialsNo credentials declared

The registry listing declares no required environment variables or authentication. This is declared metadata, not a verified install — the repository README is the authority.

Declared by Glama, last synced Nov 7, 2026. Not independently verified by MCP Toplist.

Declared in the registry install config

EVALSMITH_JUDGE_BASE_URL · optional
EVALSMITH_JUDGE_MODEL · optional
EVALSMITH_JUDGE_API_KEY · optional
LOG_LEVEL · optional

INSTALLATION AND FIRST TASK

Connect evalsmith-mcp and confirm it works

npm package

claude mcp add evalsmith-mcp -e "EVALSMITH_JUDGE_BASE_URL=<your EVALSMITH_JUDGE_BASE_URL>" -e "EVALSMITH_JUDGE_MODEL=<your EVALSMITH_JUDGE_MODEL>" -e "EVALSMITH_JUDGE_API_KEY=<your EVALSMITH_JUDGE_API_KEY>" -e "LOG_LEVEL=<your LOG_LEVEL>" -- npx -y evalsmith-mcp

Replace each placeholder with your own value before saving.

npm package

[mcp_servers.evalsmith-mcp]
command = "npx"
args = ["-y", "evalsmith-mcp"]

[mcp_servers.evalsmith-mcp.env]
EVALSMITH_JUDGE_BASE_URL = "<your EVALSMITH_JUDGE_BASE_URL>"
EVALSMITH_JUDGE_MODEL = "<your EVALSMITH_JUDGE_MODEL>"
EVALSMITH_JUDGE_API_KEY = "<your EVALSMITH_JUDGE_API_KEY>"
LOG_LEVEL = "<your LOG_LEVEL>"

Add to ~/.codex/config.toml. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "evalsmith-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "evalsmith-mcp"
      ],
      "env": {
        "EVALSMITH_JUDGE_BASE_URL": "<your EVALSMITH_JUDGE_BASE_URL>",
        "EVALSMITH_JUDGE_MODEL": "<your EVALSMITH_JUDGE_MODEL>",
        "EVALSMITH_JUDGE_API_KEY": "<your EVALSMITH_JUDGE_API_KEY>",
        "LOG_LEVEL": "<your LOG_LEVEL>"
      }
    }
  }
}

Add to claude_desktop_config.json (Settings → Developer → Edit Config). Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "evalsmith-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "evalsmith-mcp"
      ],
      "env": {
        "EVALSMITH_JUDGE_BASE_URL": "<your EVALSMITH_JUDGE_BASE_URL>",
        "EVALSMITH_JUDGE_MODEL": "<your EVALSMITH_JUDGE_MODEL>",
        "EVALSMITH_JUDGE_API_KEY": "<your EVALSMITH_JUDGE_API_KEY>",
        "LOG_LEVEL": "<your LOG_LEVEL>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (per project). Replace each placeholder with your own value before saving.

npm package

{
  "servers": {
    "evalsmith-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "evalsmith-mcp"
      ],
      "env": {
        "EVALSMITH_JUDGE_BASE_URL": "<your EVALSMITH_JUDGE_BASE_URL>",
        "EVALSMITH_JUDGE_MODEL": "<your EVALSMITH_JUDGE_MODEL>",
        "EVALSMITH_JUDGE_API_KEY": "<your EVALSMITH_JUDGE_API_KEY>",
        "LOG_LEVEL": "<your LOG_LEVEL>"
      }
    }
  }
}

Save as .vscode/mcp.json in your workspace. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "evalsmith-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "evalsmith-mcp"
      ],
      "env": {
        "EVALSMITH_JUDGE_BASE_URL": "<your EVALSMITH_JUDGE_BASE_URL>",
        "EVALSMITH_JUDGE_MODEL": "<your EVALSMITH_JUDGE_MODEL>",
        "EVALSMITH_JUDGE_API_KEY": "<your EVALSMITH_JUDGE_API_KEY>",
        "LOG_LEVEL": "<your LOG_LEVEL>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json. Replace each placeholder with your own value before saving.

npm package

{
  "mcpServers": {
    "evalsmith-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "evalsmith-mcp"
      ],
      "env": {
        "EVALSMITH_JUDGE_BASE_URL": "<your EVALSMITH_JUDGE_BASE_URL>",
        "EVALSMITH_JUDGE_MODEL": "<your EVALSMITH_JUDGE_MODEL>",
        "EVALSMITH_JUDGE_API_KEY": "<your EVALSMITH_JUDGE_API_KEY>",
        "LOG_LEVEL": "<your LOG_LEVEL>"
      }
    }
  }
}

Replace each placeholder with your own value before saving.

First successful task

After adding the config, ask your client to list this server's tools. A tool list confirms the connection; an empty list or an error means the server did not start or needs credentials.

ALTERNATIVES

When to choose another server

Other servers by diegoperezg7

mcp-guard#16,841

View all diegoperezg7 servers →

No task-specific alternatives are catalogued for evalsmith-mcpyet; the list above is its publisher's other servers.

MAINTENANCE AND EVIDENCE

What changed, and when this was checked

First listed: Oct 23, 2026
Latest version published: Oct 28, 2026
Version history: 1 version tracked (below)
Requirements evidence: Declared by Glama, last synced Nov 7, 2026. Not independently verified by MCP Toplist.
Profile checked: Not yet — purpose, capabilities and requirements above come from registry data only
Ranking data as of: Nov 7, 2026

Registry sightings

Glama ↗listed since Oct 24, 2026 · still listed
Official MCP ↗listed since Oct 28, 2026 · still listed

Do you maintain evalsmith-mcp? to correct or enrich this profile.

VERSIONS

Available versions (1)

Version	Published
0.1.0	Oct 28, 2026

STANDING

#21,542of 171,126 active serversTop 25%

Ranks ahead of 149,584 of 171,126 servers on MCP Toplist.

BADGE

Show your rank

Maintain this server? Add the live rank badge to your README — it updates automatically as the leaderboard changes.

Markdown

[![MCP Toplist](https://mcptoplist.com/badge/io.github.diegoperezg7%2Fevalsmith-mcp.svg)](https://mcptoplist.com/server/io.github.diegoperezg7%2Fevalsmith-mcp)

HTML

<a href="https://mcptoplist.com/server/io.github.diegoperezg7%2Fevalsmith-mcp"><img src="https://mcptoplist.com/badge/io.github.diegoperezg7%2Fevalsmith-mcp.svg" alt="MCP Toplist: Top 25% of 171,126" /></a>

Variants: append ?metric=score or ?metric=stars to the image URL.

FAQ

Frequently asked questions

Who maintains evalsmith-mcp?: evalsmith-mcp is maintained by diegoperezg7, which publishes 2 MCP servers (3 total versions) tracked on MCP Toplist.
Is evalsmith-mcp listed on the Official MCP Registry?: Yes — evalsmith-mcp is listed on the Official MCP Registry, alongside Glama.
How many versions does evalsmith-mcp have?: MCP Toplist tracks 1 version of evalsmith-mcp, most recently published on Oct 28, 2026.
Where can I find the source code for evalsmith-mcp?: The source code for evalsmith-mcp is hosted at github.com/diegoperezg7/evalsmith-mcp.

Used in projects

Do you run this server?

Community projects list the MCP servers behind real agents and apps. Publish yours and this page will link to it.

EXPLORE

All MCP Servers Official MCP Servers GitHub MCP Servers Best MCP Servers Most Active MCP Servers New MCP Servers Best New MCP Servers