TestSprite

Visit Website

Introduction

TestSprite is an AI testing agent and automation platform from TestSprite. The official site describes it as building a regression suite automatically and verifying the running application rather than a mock: agents drive a real browser or call a live API. A pass is meant to mean a person could complete the same path on the product under test.

The same suite is available from four surfaces: an open-source CLI for coding agents and CI, an MCP server in the editor, a merge gate on pull requests, and a no-code dashboard. The official CLI is Apache-2.0 at TestSprite/testsprite-cli and is published as @testsprite/testsprite-cli. Tests created in one surface show up in the others. This is a hosted verification product with a free plan and paid plans, not a local-only test runner you fully operate yourself.

Features

1. Live-app verification: Official docs say TestSprite proves behaviour against the running app, not assertions on mocks. Frontend, backend and data can be covered together.
2. CLI for agents and CI: npm install -g @testsprite/testsprite-cli followed by testsprite setup --agent AGENT stores an API key and installs a verification skill into the coding agent’s project. The CLI exposes structured JSON (--output json) and stable exit codes (0 passed, 1 failed).
3. MCP server in the IDE: TestSprite documents an MCP server so an agent in the editor can trigger the same suite during a session. The company site lists the MCP server and CLI as the way to plug into existing coding agents.
4. Pull-request gate: A GitHub PR auto-test can queue the suite against the pull request’s deployment and comment the result. Plan limits cap how many repositories this covers.
5. No-code dashboard: The web portal is the human surface for project setup, test lists, schedules, billing and members. Official docs say it is not the scriptable interface; the CLI is.
6. Failure bundle: When a test fails, TestSprite returns one bundle for that snapshot: the failing step and its neighbours, screenshots, DOM snapshots, the test source, a root-cause hypothesis and a recommended fix.
7. Auto-heal: Paid plans describe auto-healing when the UI drifts, driven by logic rather than only the DOM. A healed test still has to be checked; a heal is not proof that the product is correct.
8. Frontend and backend generation: Official pricing and product pages list frontend and backend test generation, with advanced backend API testing starting on higher plans.
9. Scheduling and notifications: Scheduled runs and monitoring appear from the Starter plan upward. Slack notifications are listed from the Free plan. Jira and Linear integrations are listed from Standard upward.
10. Open-source CLI: The CLI is Apache-2.0. Hosted execution, credits and dashboard features are commercial. See the repository and the pricing page for what is free to run yourself versus what runs in TestSprite’s cloud.

Pros

1. Built for coding agents: The CLI and MCP server are first-class, with output an agent can parse and exit codes CI can gate on.
2. One suite, several surfaces: A test created in the terminal appears in the dashboard, and the reverse is documented, so humans and agents are not maintaining two catalogues.
3. Failures come with evidence: The failure bundle ties the step, DOM, screenshots and a suggested cause to one run, which is more useful than a bare stack trace.
4. A free plan exists: The official pricing page lists a $0 plan with a monthly credit allowance, so you can try the loop on one app before paying.
5. Live verification: Testing the running app, not a mock, catches integration failures that a mocked unit test will miss.
6. CLI is inspectable: Apache-2.0 source is on GitHub, so you can read what the client sends and how it reports results.

Cons

1. Hosted dependency: Meaningful runs use TestSprite’s cloud and an API key. The open-source CLI is the client, not a full on-prem substitute for the platform.
2. Credit-based billing: Plans are priced in monthly credits. Usage can be hard to predict until you have measured a typical run. See the pricing page for current allowances.
3. Auto-heal still needs review: A healed selector or step can hide a real product change. Treat a heal as a proposal, not a sign-off.
4. Plan limits on PR testing and environments: Free covers GitHub PR auto-test on one repository and one environment. Teams with many apps will outgrow that quickly.
5. Vendor-stated scale figures: The marketing homepage lists large user and install counts. Those figures are the company’s, not independently verified here.

Usage and Links (Example)

Official CLI docs describe this loop for a coding agent or a developer at a terminal.

Install and onboard:

npm install -g @testsprite/testsprite-cli
testsprite setup --agent AGENT

Replace AGENT with the agent name documented in TestSprite’s installation guide. setup stores your API key and installs the verification skill into the agent’s project.

Create and run a test: Official docs say you describe the behaviour in a plan file, then one command creates the test, runs it in the cloud against the live app, and blocks until a verdict. Exit 0 means passed; exit 1 means failed. Use --output json when a script or agent needs a stable contract.

In the dashboard: Create a project, point it at the app, and watch a run. The portal is the place to inspect history, schedules and members.

In an IDE: Install the TestSprite MCP server using the steps on docs.testsprite.com, then ask the coding agent to run or extend the suite.

Confirm command names and flags in the current CLI reference before you commit a pipeline. They can change between releases.

Resources:

• TestSprite website
• Pricing
• CLI overview
• MCP docs
• CLI GitHub repository

Pricing

Prices below are from TestSprite’s pricing page, read as monthly USD figures. The same page offers yearly billing at a listed discount; see that page for current yearly totals. Credit allowances and feature gates can change, so confirm before you buy.

1. Free: $0 a month. 150 credits, 1 environment, 30 days of run history, the full plan-generate-run-report loop, agent chat, CLI and IDE plugin, GitHub PR auto-test on 1 repository, and Slack notifications. Described as a way to try the product on one app.
2. Starter: $0 for the first month, then $19 a month. 400 credits, the Free features, plus auto-heal, GitHub PR auto-test on 3 repositories, advanced backend testing, and scheduled runs and monitoring.
3. Standard: $39 a month. 800 credits, 3 environments, 90 days of history, PR auto-test on 5 repositories, an advanced agent model, Jira and Linear integrations, and agent memory per project.
4. Pro: $69 a month. 1,600 credits, unlimited environments and run history, unlimited GitHub PR auto-test, the most capable agent model listed, agent memory across the workspace, and priority test execution.
5. Enterprise: From $199 a month, from 10 seats and 5,000 credits. Custom limits, single-tenant deployment, SSO, SCIM, audit logs, a named CSM and an SLA. Also listed on AWS Marketplace. Contact TestSprite for a quote.

What a “credit” deducts per run is not fully specified on the public pricing table. See the site for the current definition and overage rules.

Recommended for?

1. Teams whose coding agents already write application code and need a verifier those agents can run from the terminal or the IDE.
2. Groups that want a PR gate on a live preview or staging URL, with a comment on the pull request.
3. QA and product people who need a dashboard to watch runs, without living in the CLI.
4. Projects that can accept a hosted runner for the browser and API checks, and that can expose a reachable test environment.
5. Not a fit if you must run every check inside your own network with no vendor cloud, or if you only need a local Playwright suite you already maintain by hand.

Conclusion

TestSprite is a hosted AI testing agent that generates, runs and reports checks against a live application. The open-source CLI and the MCP server are the interfaces built for coding agents; the dashboard is the interface built for people. Official docs emphasise a single suite, a machine-readable failure bundle, and a pull-request gate.

The product is not a drop-in replacement for a fully local, fully owned test framework. Billing is credit-based, higher plans unlock auto-heal and more repositories, and a healed test still deserves a human look. If you want agents to verify what they just shipped, and you can use TestSprite’s cloud, it is a serious option. Confirm current plans and credit rules on the official pricing page before you standardise on it.