CheckOps home

Tools directory

A

CheckOps

Alumnium

Alumnium is an MIT-licensed AI test harness: an MCP server for coding agents plus Java, Python and TypeScript libraries that add natural-language steps to Selenium, Playwright and Appium.

Visit website

Latest public signals

Not refreshed yet. Refresh reads the public site and, when the entry has a GitHub repository, the latest release.

Visit Website

Introduction

Alumnium is an open-source “end-to-end testing harness for agents and engineers”. The repository alumnium-hq/alumnium is MIT-licensed. The official site describes three parts: an MCP server, client libraries and a test runner for web and mobile applications.

Agents connect through MCP. Engineers add the Java, Python or TypeScript client to tests they already have on Selenium, Playwright or Appium, and write high-level steps and checks in plain language. Version 0.24 added Firefox and Xcode support.

Features

  1. MCP server: Connect Alumnium to Claude Code, Codex, Grok Build and other MCP clients with one command.
  2. Client libraries: Java, Python and TypeScript clients sit on top of Selenium, Playwright or Appium.
  3. Natural-language steps: Write actions and verifications in plain English inside existing tests.
  4. Web and mobile: Browsers plus iOS and Android through Appium; v0.24 lists Firefox and Xcode.
  5. Context efficiency: The MCP server does detailed UI work and returns concise change summaries to the coding agent.
  6. Model choice: You choose the AI provider (for example Anthropic, OpenAI or xAI) through configuration.
  7. Benchmark: The site reports Claude Code with Alumnium MCP passing 98.5% of 610 WebVoyager tasks; that is the project’s own result.

Pros

  1. MIT licence: Free to use and embed.
  2. Works with your stack: Adds AI steps to Selenium, Playwright or Appium instead of replacing them.
  3. Agents and engineers share one tool: The same harness serves MCP agents and coded suites.
  4. Active project: Commits landed the day before review.
  5. Easy install: Script, npm, pip and uv installers are documented.

Cons

  1. Bring your own model: You pay for LLM calls and manage keys.
  2. Pre-1.0: Version 0.x means APIs may change.
  3. AI steps vary: Natural-language steps are less deterministic than coded locators.
  4. Small community: Around a thousand GitHub stars at review, so fewer answers online.
  5. Benchmark is self-reported: Verify performance on your own app.

Add the MCP server to Claude Code, from the official site:

claude mcp add alumnium --env ALUMNIUM_MODEL=anthropic --env ANTHROPIC_API_KEY=... -- alumnium mcp

Or install the library (pip install alumnium or npm install alumnium) and use it alongside your Selenium, Playwright or Appium driver to run plain-language steps and checks.

Resources:

Pricing

Alumnium is free and open source under MIT. Costs come from the AI model provider you configure.

  1. Teams with existing Selenium, Playwright or Appium suites that want AI steps without migrating.
  2. Coding-agent users who want a testing MCP server that keeps the agent’s context small.
  3. Polyglot teams working in Java, Python or TypeScript.
  4. Not a fit if you need a hosted dashboard and vendor support.

Conclusion

Alumnium is an open, MIT-licensed bridge between AI agents and established automation drivers, with an MCP server and multi-language clients.

It is young and depends on your model provider, but it lets you add AI-driven checks to the suites you already maintain.

Alternatives

Other tools on CheckOps that cover similar ground. Open a page, or compare one with Alumnium side by side.