Shortest

Visit Website

Introduction

Shortest is an open-source framework from Antiwork for “QA via natural language AI tests”. The repository antiwork/shortest is MIT-licensed. You write tests in plain English and an AI executes them on Playwright.

The README lists AI-powered execution with the Anthropic Claude API, GitHub integration with 2FA support, and email validation with Mailosaur. A shortest init command scaffolds a config file and environment variables in a new or existing project.

Features

1. Plain-English tests: For example, shortest('Login to the app using email and password', {...}).
2. Built on Playwright: Uses Playwright for browser control.
3. Claude execution: The Anthropic API interprets and executes steps.
4. GitHub 2FA: Handles GitHub login with 2FA in tests.
5. Email validation: Mailosaur integration for email flows.
6. Quick setup: npx @antiwork/shortest init installs the package and creates shortest.config.ts and .env.local.

Pros

1. MIT licence: Free and permissive.
2. Very readable: Tests read like acceptance criteria.
3. Low setup: One init command.
4. Playwright underneath: Familiar browser engine.
5. Popular: About 5,700 GitHub stars at review time.

Cons

1. Tied to Anthropic: Execution uses the Anthropic API; you need a key and pay per use.
2. Slower activity: The last push was in August 2026, about three months before review.
3. Non-deterministic: AI execution can vary between runs.
4. Limited scope: Web only.
5. Small feature set: Fewer reporting and management features than platforms.

Usage and Links (Example)

From the README:

npx @antiwork/shortest init

Add your Anthropic API key and base URL in shortest.config.ts, then write tests:

import { shortest } from "@antiwork/shortest";
shortest("Login to the app using email and password", {
  username: process.env.GITHUB_USERNAME,
  password: process.env.GITHUB_PASSWORD,
});

Resources:

• Shortest website
• GitHub repository

Pricing

Shortest is free and open source under MIT. Anthropic API usage is billed by Anthropic.

Recommended for?

1. Small web teams wanting readable acceptance tests.
2. Developers already using Claude.
3. Prototyping natural-language QA before buying a platform.
4. Not a fit for teams needing model choice or enterprise reporting.

Conclusion

Shortest makes natural-language end-to-end tests simple to start on Playwright, with an MIT licence.

It is tied to Anthropic and less active lately, so pin versions and keep deterministic tests for critical paths.