GitHub Action

Catch flaky tests on every pull request with the FlakeMonster GitHub Action. One workflow file, automatic PR comments.

Quick Setup

Add a single workflow file to your repository and FlakeMonster will run on every pull request. Create .github/workflows/flake-check.yml with the following contents:

name: Flake Check
on: [pull_request]

permissions:
  pull-requests: write
  id-token: write

jobs:
  flake-monster:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - uses: growthboot/FlakeMonster-Action@v1
        with:
          test-command: 'npm test'

That is all you need. The action installs FlakeMonster, injects delays into your source files, runs your test command multiple times, restores your code, and posts a comment on the PR with the results.

Inputs

All inputs are optional. The defaults work well for most JavaScript projects.

Outputs

The action exposes outputs you can use in subsequent workflow steps.

PR Comments

The action automatically posts a comment on the pull request with the results of the flake check. No configuration is needed beyond the pull-requests: write permission.

When flaky tests are detected, the comment includes:

• Flaky test count — the total number of tests that passed on some runs and failed on others
• Results table — each flaky test with its name, flaky rate (percentage of runs that failed), and the specific seeds that triggered failures
• Seed list — all seeds used across every run, so you can reproduce any individual run locally

When all tests are stable across every run, the action posts a short success message confirming no flakes were found.

The action updates the same comment on subsequent pushes to the same PR rather than creating duplicate comments.

Free Tier

The FlakeMonster GitHub Action is free for small teams and open-source projects.

• 50 checks per month per repository, with no signup required
• Usage is tracked automatically via GitHub OIDC tokens — the id-token: write permission enables this without exposing any secrets
• At 30+ checks in a billing period, the action adds a usage warning to the PR comment so you can plan ahead
• At 50+ checks, the action skips the flake analysis and posts a quota message instead. Your workflow will still pass — it just will not run the flake check

Tip: If you only run the action on pull_request events (not on every push), most teams stay well within the free tier.

Pro Tier

For teams that need unlimited checks, the Pro tier costs $29/month per team.

• Unlimited flake checks across all repositories in the team
• Upgrade at the pricing page
• Cancel anytime — your access continues through the end of the billing period

Permissions

The action requires two permissions in your workflow file. Here is what each one does and why it is needed:

Note: These are the only permissions needed. The action does not request contents: write or any other elevated access. Your source code is only modified inside the runner and restored before the step completes.

Example Workflows

Below are complete workflow examples for common scenarios. Copy any of these into .github/workflows/flake-check.yml in your repository.

Basic PR Check

Fast flake check on every pull request. Uses light mode with 5 runs for quick feedback.

name: Flake Check
on: [pull_request]

permissions:
  pull-requests: write
  id-token: write

jobs:
  flake-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - uses: growthboot/FlakeMonster-Action@v1
        with:
          test-command: 'npm test'
          runs: 5
          mode: light

Thorough Nightly Build

Runs 20 iterations in hardcore mode on a schedule. Catches the most subtle flakes that quick PR checks might miss.

name: Nightly Flake Scan
on:
  schedule:
    - cron: '0 3 * * *'  # Every night at 3 AM UTC
  workflow_dispatch:  # Allow manual trigger

permissions:
  pull-requests: write
  id-token: write

jobs:
  deep-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - uses: growthboot/FlakeMonster-Action@v1
        with:
          test-command: 'npm test'
          runs: 20
          mode: hardcore
          max-delay: 100
          fail-on-flake: false  # Report only, don't fail

Playwright Project

End-to-end testing with Playwright. Targets test utilities and page objects rather than the test files themselves.

name: Flake Check (E2E)
on: [pull_request]

permissions:
  pull-requests: write
  id-token: write

jobs:
  flake-e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps
      - uses: growthboot/FlakeMonster-Action@v1
        with:
          test-command: 'npx playwright test'
          runner: playwright
          runs: 5
          mode: medium
          globs: 'src/**/*.js e2e/helpers/**/*.js'

Using Outputs to Conditionally Fail

Run the flake check without failing, then use the outputs to decide what happens next. This is useful when you want to add custom logic, like only failing when more than a threshold of flakes are found.

name: Flake Check (Custom Logic)
on: [pull_request]

permissions:
  pull-requests: write
  id-token: write

jobs:
  flake-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci

      - name: Run FlakeMonster
        id: flake
        uses: growthboot/FlakeMonster-Action@v1
        with:
          test-command: 'npm test'
          runs: 10
          fail-on-flake: false  # Don't fail yet

      - name: Fail if too many flakes
        if: fromJSON(steps.flake.outputs.flaky-count) > 3
        run: |
          echo "Found ${{ steps.flake.outputs.flaky-count }} flaky tests (threshold: 3)"
          exit 1

      - name: Parse detailed results
        if: steps.flake.outputs.flaky-count != '0'
        run: |
          echo '${{ steps.flake.outputs.result-json }}' | jq '.flakyTests[]'