Firebase Test Lab

Visit Website

Introduction

Firebase Test Lab runs mobile tests on Google’s virtual and physical devices. You upload an Android app with an instrumentation test, a Robo script, or a game loop, or an iOS app with XCTest, and Test Lab runs that binary on a matrix of device models and OS versions. Results include logs, screenshots, and videos in the Firebase console and in Cloud Storage. The gcloud CLI and the Firebase console are the two ways to start a matrix.

Google’s documentation marks Test Lab as deprecated. The published shutdown date is 30 September 2027. The same docs point teams at the Developer Device Platform on Google Cloud as the migration path. This entry exists because Test Lab is still the device cloud many Firebase apps use today, and the quotas and the shutdown date are the facts a reader needs. It is not a recommendation to start a new long-lived suite on it without a migration plan.

Features

1. Instrumentation tests: Android Espresso or UI Automator suites run on the lab’s devices. You upload the app APK and the test APK.
2. Robo test: A crawler exercises an Android app without a test suite. It can follow a Robo script for login steps. This is a crawl, not a specification of expected behaviour.
3. Game loop tests: A demo mode inside a game binary runs on Android or iOS devices. The game has to implement the loop.
4. iOS XCTest: XCTest packages run on the listed iOS devices. Robo is not the iOS path.
5. Device matrix: One invocation fans out across models, API levels, and locales. Each combination is a separate run and counts toward quota.
6. Results: Logcat or device logs, screenshots, video, and a pass or fail per matrix cell. Performance metrics appear for some Android runs.
7. Triggers: gcloud, the console, Firebase Test Lab’s Gradle plugin, and CI systems that can call gcloud. The lab does not replace the test code.
8. Spark and Blaze quotas: Daily free runs on the no-cost Spark plan, and a daily free minute allowance on the paid Blaze plan before hourly charges.

Pros

1. Real device diversity without a shelf of phones: The matrix covers device and API combinations you will not keep plugged into a laptop.
2. Uses the tests you already wrote: Espresso and XCTest packages run as-is. You are not translating them into a proprietary DSL.
3. A no-cost daily quota: Spark includes virtual and physical runs every day, which is enough to see a failure on a device you do not own.
4. CI is gcloud: A service account and gcloud firebase test android run fit a pipeline that already ships to Firebase.
5. The shutdown date is public: You can plan. The risk is staying silent about it, not the existence of the date.

Cons

1. Deprecated: Full shutdown is scheduled for 30 September 2027. New suites need an exit, documented by Google as the Developer Device Platform, or another lab.
2. Quota is daily and shared: Spark is 10 virtual and 5 physical test runs per day. A pull request that fans out across a large matrix will hit the cap and queue or fail.
3. Blaze minutes are easy to spend: After the daily included minutes, physical devices are $5 per hour and virtual devices $1 per hour, billed per minute. A 20-minute test on five physical devices is no longer a rounding error.
4. Robo is not a regression suite: It finds crashes and some obvious UI problems. It does not encode the business rules Espresso or XCTest should encode.
5. Cold runs and flakes: Lab devices, network, and sign-in dialogs add failures that do not reproduce on one emulator. Budget time to read videos, not only the exit code.

Usage and Links (Example)

Android instrumentation, after the APKs are built and gcloud is logged in to the Firebase project:

gcloud firebase test android run \
  --type instrumentation \
  --app app-debug.apk \
  --test app-debug-androidTest.apk \
  --device model=MediumPhone.arm,version=34

List current models before you hard-code one; names change:

gcloud firebase test android models list

iOS uses gcloud firebase test ios run with an XCTest zip. Watch the matrix in the Firebase console under Test Lab. Set a Blaze budget alert before a CI job can expand the matrix on every commit.

Resources:

• Firebase Test Lab docs
• Usage, quotas, and pricing
• Deprecation and migration

Pricing

From the official quotas page. Test Lab is deprecated; these rates apply until shutdown.

1. Spark (no cost): Up to 15 test runs per day: 10 on virtual devices and 5 on physical devices. The cap is per project and is shared by instrumentation, Robo, and game-loop tests.
2. Blaze included time, per day: 60 minutes on virtual devices and 30 minutes on physical devices. This mirrors the Spark allowance in minutes rather than in run counts.
3. Blaze after that: $1 per hour for each virtual device and $5 per hour for each physical device. Billing is per minute, rounded up to the next minute. Install time and result collection are not charged; the test’s own runtime is.
4. After 30 September 2027: The docs say the lab shuts down. Do not budget it as a permanent home. Google’s stated migration target is the Developer Device Platform on Google Cloud; price that product separately when you move.

Budget alerts on Blaze do not stop runs by themselves. A quota increase is a support request, not an automatic consequence of paying.

Recommended for?

1. Existing Firebase mobile apps that already run Espresso or XCTest and need a device matrix before the 2027 shutdown.
2. Projects on Spark checking a small matrix once a day, inside the 10 virtual and 5 physical runs.
3. Games with a game-loop hook that want a lab run without a full UI script.
4. Not for a new multi-year device strategy unless the migration to Google’s replacement, or to another lab, is already on the plan.

Conclusion

Firebase Test Lab still runs Android and iOS tests on hosted devices, with a small daily free quota and straightforward hourly Blaze rates after that. The fact that dominates the choice is the published shutdown on 30 September 2027. Use it as a bridge for suites you already have, keep matrices small, and put the successor on the roadmap before you add more tests that can only run here.