Beyond the Happy Path: What Happened When We Ran Maestro E2E Tests on Real Android Hardware

Push to Git.

Wait 8 minutes for CI.

The UI test fails on an arbitrary timeout.

Fix -> push -> wait 8 minutes -> fails again

Eventually, you stop trusting the tests. You mute the Slack alerts, sprinkle a few sleep(5000) calls into your test harness, and eventually rip out the entire suite the moment an Expo or React Native upgrade breaks the native test runner configuration.

If you have ever wrestled with mobile end-to-end (E2E) testing using Appium or Detox, you know that pain intimately.

A while back, I started experimenting with Maestro. Like most developers, my initial test was a basic "Hello World": tap a button, type into a login form, and assert that a welcome banner appeared. It worked within minutes. No intrusive native SDK bridging, no test code polluting the production bundle, and completely readable YAML flows.

It felt like mobile testing was finally solved.

Then we tried automating our actual production application.

Our app is offline-first. It does not just render static views - it processes local SQLite database transactions, parses binary data directly on the device filesystem, interacts with native Android document pickers (ACTION OPEN DOCUMENT), and triggers system-level Share sheets.

When you push Maestro past basic forms and into heavy on-device architectures on real physical hardware, the "Hello World" assumptions fall apart. Here are the practical roadblocks we ran into, the exact patterns we used to solve them, and how we built an E2E harness that actually stays green.

1. Managing Auth & Secrets with dotenv-cli

Before our automated flows could touch core features, we hit an immediate obstacle: the login screen.

Automating authentication introduces three common problems:

1. Zero Secret Leakage: You cannot commit test emails, passwords, or API keys inside version-controlled YAML files.

2. Environment Portability: Development, staging, and production builds frequently use different bundle identifiers (e.g., com.company.app.dev vs. com.company.app). Hardcoding appId breaks test flexibility.

3. The Silent Failure Trap: If Maestro encounters an undefined environment variable, it does not always fail immediately. Depending on the command, it can input an empty string or the raw literal text ${MAESTRO_TEST_PASS} into the form field. The auth backend rejects the string, and the entire test run stalls.

While Maestro natively supports passing variables using -e KEY=VALUE flags, typing those flags out for every run is tedious and prone to typos.

The Fix: Isolated .env + Modular Sub-Flows

We decoupled our secrets by creating a dedicated environment file inside .maestro/ and adding it to .gitignore:

# .maestro/.env (Ignored in Git)

MAESTRO_APP_ID=com.company.app
MAESTRO_TEST_EMAIL=tester@example.com
MAESTRO_TEST_PASS=SuperSecretPassword123!

Next, we isolated authentication into a reusable sub-flow .maestro/sub-flows/login.yaml) using standard variable substitution:

appId: ${MAESTRO_APP_ID}
---
- runFlow:
    when:
      notVisible:
        id: 'main-tab-bar'
    commands:
      # Clear introductory dialogs or navigation gates
      - tapOn:
          id: 'btn-goto-signin'
          optional: true

      # Enter credentials safely from environment variables
      - tapOn:
          id: 'input-email'
      - eraseText: 50
      - inputText: ${MAESTRO_TEST_EMAIL}
      - hideKeyboard:
          optional: true

      - tapOn:
          id: 'input-password'
      - eraseText: 50
      - inputText: ${MAESTRO_TEST_PASS}
      - hideKeyboard:
          optional: true

      # Submit and await main interface
      - tapOn:
          id: 'btn-submit-auth'

      - extendedWaitUntil:
          visible:
            id: 'main-tab-bar'
          timeout: 15000

To automatically load .maestro/.env into Maestro without manually exporting variables in our terminal shell, we wrapped the command using dotenv-cli:

npm install -D dotenv-cli

Then we mapped clean lifecycle commands inside package.json:

"scripts": {
  "test:e2e": "dotenv -e .maestro/.env -- maestro test .maestro/main-flows/",
  "test:e2e:auth": "dotenv -e .maestro/.env -- maestro test .maestro/main-flows/auth/"
}

Running npm run test:e2e:auth cleanly injects credentials into the process environment, keeping secrets out of Git while keeping terminal commands short.

2. The Native Document Picker Ambush

Many mobile apps allow users to select external files, such as data backups, attachments, or formatted documents. But calling ACTION_OPEN_DOCUMENT hands execution control over to the underlying Android system: Google Files or Samsung My Files.

Our initial flow triggered the document picker, searched for the test file, and tapped the result:

- tapOn:
    id: 'btn-open-picker'

# Await the system document picker
- extendedWaitUntil:
    visible:
      text: '.*(Search|Browse|Recent|Files|Downloads).*'
    timeout: 10000

- tapOn:
    id: '.*(search_button|menu_search).*'
- inputText: 'sample_dataset.csv'

# Attempt to tap the matching file
- tapOn:
    text: 'sample_dataset.csv'

The Problem

On physical Android hardware, this flow failed intermittently.

Tapping the text sample_dataset.csv often collided with the active search bar where that exact text was still displayed. Even worse, on several OEM devices, tapping the file title directly triggered an ACTION_VIEW intent (opening a system file previewer) instead of selecting the file and passing its URI back to our app.

On top of that, Android's file manager often categorizes custom or structured files under generic metadata labels like .*CSV document.*, .*Binary file.*, or .*Document.*.

The Fix: Targeting Subtitle Metadata

Instead of matching the file title or relying on the search box, we targeted the OS-generated metadata subtitle displayed underneath the file row:

# Await search results inside the system file manager
- extendedWaitUntil:
    visible:
      text: '.*Document.*'
    timeout: 10000

# Tap the file list row directly via its metadata label
- tapOn:
    text: '.*Document.*'

Targeting the subtitle metadata completely bypasses the active search query box and reliably prompts the system picker to return the selected file URI.

3. Managing Heavy On-Device I/O and Timeouts

In mock-heavy integration tests, database calls resolve almost instantaneously. On a physical mobile device running on battery power:

• Reading a multi-megabyte file requires flash storage I/O.

• Decompressing or parsing an archive consumes CPU cycles.

• Batch-inserting thousands of structured rows into indexed SQLite tables takes real wall-clock time.

Maestro’s default element wait timeout is around 5 to 10 seconds. When our automated flow confirmed a file processing action, Maestro immediately started asserting the presence of the completion view. Because the background database transactions were still running, the test threw a timeout error while the device was still actively processing data.

The Fix: Calibrated extendedWaitUntil

Rather than littering tests with arbitrary sleep() statements, calibrate your timeouts to the physical weight of each operation:

# 1. Parsing & Extraction Phase (CPU bound)
- extendedWaitUntil:
    visible:
      id: 'file-preview-card'
    timeout: 45000

- scrollUntilVisible:
    element:
      id: 'btn-confirm-process'
    direction: DOWN

- tapOn:
    id: 'btn-confirm-process'

# 2. Batch Database Insertion Phase (Disk I/O bound)
- extendedWaitUntil:
    visible:
      id: 'status-success-banner'
    timeout: 60000

By pairing extendedWaitUntil with explicit visual containers, Maestro waits only as long as the hardware needs, advancing the millisecond the operation finishes.

4. The testID Accessibility Mismatch in Native Sharing

When testing document generation and sharing, our app compiles data into a shareable bundle and launches the native Android Share sheet.

Our screen renders a list of items with action buttons. Initially, the component looked like this:

// Inside Item Row Component
<Button
  testID={`btn-share-${item.id}`}
  title="Share Summary"
  onPress={() => handleShare(item.id)}
/>

The Problem

To guarantee test isolation, an automated E2E run needs to create temporary records dynamically rather than depending on pre-seeded rows.

If Maestro creates an item titled Project Workspace 1729001234, it has no way of knowing the generated database primary key item.id).

When Maestro executed:

- tapOn:
    id: "btn-share-${output.documentTitle}" # Fails: Expected an internal database ID

The selector could not match the element, causing the test to fail.

The Fix: Human-Readable Dynamic testID

We updated the UI component to bind its testing identifier to human-readable, test-accessible properties (item.title):

<Button
  testID={`btn-share-${item.title}`}
  title="Share Summary"
  onPress={() => handleShare(item.id)}
/>

Now, the Maestro flow can dynamically generate a timestamped title, populate the record, and target the resulting button deterministically:

- evalScript: ${output.documentTitle = 'Project Workspace ' + Date.now()}

# ... item creation steps ...

# Locate the dynamic row and trigger share action
- scrollUntilVisible:
    element:
      id: "btn-share-${output.documentTitle}"
    direction: DOWN

- tapOn:
    id: "btn-share-${output.documentTitle}"

# Dismiss the native Android Share Sheet via OS navigation
- back

# Assert completion container inside the app
- extendedWaitUntil:
    visible:
      id: 'action-success-container'
    timeout: 25000

The - back command right after triggering the action dismisses the system Share dialog, returning focus immediately back to the app UI to assert the completion container.

5. Preventing Stale State with Atomic Teardown

One of the fastest ways to destroy confidence in E2E testing is database pollution.

If Test Run #1 creates an item titled Sample Item, and Test Run #2 searches for Sample Item, the second run risks targeting stale rows, encountering incorrect count assertions, or throwing unique constraint violations.

Every lifecycle test must be self-cleaning. Setup, verification, and teardown should happen within the exact same flow:

# --- ATOMIC TEARDOWN: RECLAIM LOCAL STATE ---
- tapOn:
    id: 'tab-items'

- scrollUntilVisible:
    element:
      text: ${output.documentTitle}
    direction: DOWN

- tapOn:
    text: ${output.documentTitle}

- tapOn:
    id: 'btn-edit-item'

- scrollUntilVisible:
    element:
      id: 'btn-delete-item'
    direction: DOWN

- tapOn:
    id: 'btn-delete-item'

- extendedWaitUntil:
    visible:
      id: 'dialog-confirm-delete'
    timeout: 5000

- tapOn:
    id: 'btn-dialog-confirm'

# Final Assertion: Verify complete deletion from the local view
- extendedWaitUntil:
    visible:
      id: 'items-overview-screen'
    timeout: 15000

- assertNotVisible:
    text: ${output.documentTitle}

If a test does not clean up its generated state, it leaves behind unstable conditions for subsequent runs.

6. The CI vs. Local Hardware Trade-Off

When starting with Maestro, the initial instinct is often to shove the entire test suite into a cloud CI runner with a headless Android emulator.

However, consider the operational reality of running on-device E2E in cloud virtual machines:

1. Downloading Android command-line tools and system images on clean runners.

2. Cold-booting an Android Virtual Device (AVD) without hardware acceleration.

3. Pre-seeding mock file fixtures into virtual storage via adb push.

4. Running heavy database transactions and file parsing routines on throttled virtual CPUs.

Average cloud CI execution time: 12 to 18 minutes per pull request.

Contrast that with executing against an attached physical test device locally:

• The phone's hardware environment is already running.

• Test fixtures are pre-staged in storage.

• Database queries and file operations run directly on native silicon.

Average local execution time: 2 to 3 minutes for an entire feature suite.

Our Strategy

Rather than burning CI minutes and slowing development velocity, establish a clear separation of testing boundaries:

1. In CI (GitHub Actions): Execute TypeScript compilation checks (tsc --noEmit), ESLint, and fast Jest unit/integration tests. This gives developers feedback in under 90 seconds on every push.

2. On Physical Hardware (Pre-Release): Group Maestro flows into module suites and run them against connected hardware before shipping production builds:

{
  "scripts": {
    "test:unit": "jest features/ shared/",
    "test:e2e": "dotenv -e .maestro/.env -- maestro test .maestro/main-flows/",
    "test:e2e:workflows": "dotenv -e .maestro/.env -- maestro test .maestro/main-flows/workflows/",
    "test:e2e:studio": "dotenv -e .maestro/.env -- maestro studio"
  }
}

Whenever core database schemas, file handlers, or native bridges are modified, running npm run test:e2e:workflows validates critical user journeys on real hardware in less time than it takes to get up from your desk.

Final Thoughts & Discussion

Automating complex mobile workflows comes down to designing tests around the realities of mobile operating systems rather than treating your app like a simple web view:

• Target Subtitle Metadata: Bypasses search field collisions when automating native Android file pickers.

• Expose Dynamic testID: Uses timestamped titles Date.now()) to make UI elements deterministically targetable without backend database IDs.

• Calibrate for Hardware Workloads: Employs explicit extendedWaitUntil thresholds rather than fragile sleep timers.

• Enforce Strict Teardown: Ensures each test leaves the local database completely clean.

• Protect Secrets with dotenv-cli: Decouples sensitive test accounts from version control via isolated environment files.

Point of View

This write-up comes directly from my own hands-on experience and what worked best for our local hardware test setup. There are always multiple ways to approach mobile automation. If you have discovered smoother patterns for handling native file pickers or managing test state in Maestro, I welcome other viewpoints.

If you are currently wrestling with native OS picker issues, heavy database timing, or Maestro configurations in your React Native or Expo app, drop your scenario in the comments below. Let us discuss what works, troubleshoot edge cases together, and compare hardware automation patterns.