Who it is for
Plugin and skill authors who want a numeric read on quality and token usage before shipping.
Overview
What Plugin Eval does
Plugin Eval grades plugins and skills, then explains the grade. Asked about a named plugin, it produces an analysis, the reasoning behind the score, and an ordered list of what to address first.
Token cost is treated as something to measure rather than estimate. The app can describe a plugin's token budget, record what it actually consumes in practice, and run a benchmark against it.
The intended path starts in conversation and moves to the terminal: a plain request in chat, then either the local plugin-eval start entrypoint or whichever routed workflow command the app suggests. Five skills accompany it, covering plugin evaluation, skill evaluation, skill improvement, metric pack design, and the eval runner itself.
What stands out
Scoring is paired with token measurement, so a review covers cost as well as behaviour.
Availability
Where Plugin Eval is listed
Plugin Eval is listed in the ChatGPT apps directory only. OpenAI has no connector under the same domain in the Claude directory; 392 of the 3,150 products here are on both marketplaces, and this is not one of them.
ChatGPT apps are built on MCP, and the same server can back a Claude connector, so the absence is a distribution decision rather than a technical one.
ChatGPT listing
In the ChatGPT apps directory
- Developer
- OpenAI
- Filed under
- Developer Tools
- Listed since
- April 2026
- Authentication
- Authenticates when installed
- Website
- openai.com
- Policies published
- Privacy policy · Terms of service
- Official listing
- Open in ChatGPT
- Snapshot
- August 2026
Named skills
Plugin Eval declares 5 named skills, which puts it in the 6.7% of ChatGPT-listed apps that do.
- evaluate-plugin
- evaluate-skill
- improve-skill
- metric-pack-designer
- plugin-eval
Example prompts from the listing
- Give me an analysis of the game studio plugin.
- Evaluate this plugin.
- Why did this score that way?
Same company
More from OpenAI
Build iOS Apps
iOS development workflows for SwiftUI and Xcode
ChatGPT
Build macOS Apps
macOS app development, debugging, and packaging guidance
ChatGPT
Build Web Apps
Frontend web app building with testing and payments
ChatGPT
Build Web Data Visualization
Design and build browser-based charts and dashboards
ChatGPT
Nearby
Other Developer Tools connectors
CometChat
CometChat docs and ready-to-run integration bundles
Claude
ElevenLabs
Create and configure ElevenLabs voice agents
Claude
Sleuth Skills
Loads skills on demand from a private vault
ChatGPT
Sticklight
AI website and app builder for web professionals
ChatGPT
StubEngine
Generate live API mocks from plain descriptions
ChatGPT
Sugra API
One gateway to many primary-source data feeds
ChatGPT
This is an independent catalogue entry compiled by Node8 from Plugin Eval's public marketplace listing as it stood in August 2026. Listing facts come from the marketplaces; the comparisons against the rest of the directory and any live-endpoint measurements are Node8's. Node8 is not affiliated with OpenAI, and inclusion here is not an endorsement.
Want your company reachable inside ChatGPT and Claude?
Node8 builds MCP servers end to end: deciding which tools are worth exposing, the authentication and permission model, the review submissions, and the onboarding that gets them used. One server serves ChatGPT, Claude, and Microsoft Copilot.