Skip Navigation
Show nav
Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
    • .NET
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
View categories

Categories

  • Heroku Architecture
    • Compute (Dynos)
      • Dyno Management
      • Dyno Concepts
      • Dyno Behavior
      • Dyno Reference
      • Dyno Troubleshooting
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Developer Tools
    • Command Line
    • Heroku VS Code Extension
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery & Integration (Heroku Flow)
    • Continuous Integration
  • Language Support
    • Node.js
      • Working with Node.js
      • Node.js Behavior in Heroku
      • Troubleshooting Node.js Apps
    • Ruby
      • Rails Support
      • Working with Bundler
      • Working with Ruby
      • Ruby Behavior in Heroku
      • Troubleshooting Ruby Apps
    • Python
      • Working with Python
      • Background Jobs in Python
      • Python Behavior in Heroku
      • Working with Django
    • Java
      • Java Behavior in Heroku
      • Working with Java
      • Working with Maven
      • Working with Spring Boot
      • Troubleshooting Java Apps
    • PHP
      • PHP Behavior in Heroku
      • Working with PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
    • .NET
      • Working with .NET
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
      • Migrating to Heroku Postgres
    • Heroku Key-Value Store
    • Apache Kafka on Heroku
    • Other Data Stores
  • AI
    • Working with AI
    • Heroku Inference
      • Inference API
      • Quick Start Guides
      • AI Models
      • Inference Essentials
    • Vector Database
    • Model Context Protocol
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
      • Single Sign-on (SSO)
    • Private Spaces
      • Infrastructure Networking
    • Compliance
  • Heroku Enterprise
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • Continuous Delivery & Integration (Heroku Flow)
  • Continuous Integration
  • Heroku CI: Technical Detail on Test Run Lifecycle

Heroku CI: Technical Detail on Test Run Lifecycle

English — 日本語に切り替える

Last updated August 10, 2022

Table of Contents

  • Step 1: A CI-specific copy of your app is provisioned
  • Step 2: Required add-ons are provisioned for the CI app
  • Step 3: Applicable buildpacks are invoked
  • Step 4: The test-setup command is executed
  • Step 5: The test run is executed
  • Step 6: Test run results are reported
  • Step 7: Cleanup

This article describes the sequence of events performed by Heroku CI during a test run. The details of this sequence can be helpful if you are debugging a failing test run, or if you are adding Heroku CI support to a custom development tool or Heroku buildpack.

Step 1: A CI-specific copy of your app is provisioned

  • The app copy is provisioned on a Performance-M dyno (see Dyno Types for details).
  • Note that the CI app does not appear as a separate app in your pipeline’s details in the Heroku Dashboard.

Step 2: Required add-ons are provisioned for the CI app

  • You specify the add-ons your app requires in your app’s app.json file, as shown in Provisioning add-ons.

Step 3: Applicable buildpacks are invoked

  • If you specify custom buildpacks for your app in its app.json file, those buildpacks are invoked in the listed order.
  • If you don’t specify custom buildpacks, Heroku CI automatically determines which officially supported buildpack to invoke for your app.
  • If a particular buildpack includes a bin/test-compile script, Heroku CI runs it. Otherwise, Heroku CI runs bin/compile.
  • If any buildpack exits with a non-zero exit code, the test run is flagged as errored.

Step 4: The test-setup command is executed

  • If your app’s app.json file specifies a scripts/test-setup script (as shown in Specifying custom test commands), Heroku CI executes it from the $HOME directory.
  • If the test-setup script exits with a non-zero exit code, the test run is flagged as errored.
  • If the test-setup script completes successfully, the setup artifacts will be cached and re-used for the next run.

Step 5: The test run is executed

  • If your app’s app.json file specifies a scripts/test script (as shown in Specifying custom test commands), it is run in the $HOME directory.
  • If you don’t specify a scripts/test script, Heroku CI instead runs each associated buildpack’s bin/test script.
  • If any executed script exits with a non-zero status, the test run is flagged as a failure.
  • If all test scripts exit with a zero status, the test run is flagged as a success.
  • If the executed test scripts emit TAP-formatted output to STDOUT, individual passes and failures are counted and displayed in the test run’s UI.
  • If the executed test scripts emit TAP-formatted output to STDOUT, failures in the tests take precedence over the exit code for the test run status.

Step 6: Test run results are reported

  • Your app’s buildpacks stream output from compilation, test-setup scripts, and the test runs themselves. This output is presented in the Tests tab of your pipeline’s Heroku Dashboard page.
  • If a buildpack or test script does not support TAP format, the result of the entire test run is presented as either red (failure) or green (success).
  • Results from every test run are stored at a permanent URL in the Heroku Dashboard.

Step 7: Cleanup

  • Test runs that are still executing after one hour are terminated automatically.
  • The CI app is deprovisioned and deleted, along with any associated add-ons.

Keep reading

  • Continuous Integration

Feedback

Log in to submit feedback.

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure
  • .NET

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing
  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Github
  • LinkedIn
  • © 2025 Salesforce, Inc. All rights reserved. Various trademarks held by their respective owners. Salesforce Tower, 415 Mission Street, 3rd Floor, San Francisco, CA 94105, United States
  • heroku.com
  • Legal
  • Terms of Service
  • Privacy Information
  • Responsible Disclosure
  • Trust
  • Contact
  • Cookie Preferences
  • Your Privacy Choices