Sharing a variable between jobs

[schof]

I have a common task that I need to perform for all job “variants” in a matrixed build. My idea was to use a separate job for that but I’m unsure how to pass a build output between two jobs.

Before needing to do a matrixed build (where we build both versions in parallel), I had been doing this all as one job. My question now is, what is the best way to share information from one job (the newly incremented verison) with the matrixed job that depends on it?

See below for some abberviated code. I haven’t tried to run it yet but I’m sure it will fail since AFAIK you can’t refernce steps from another job. I’m also aware of artifacts but we’re talking about a variable here and not some file.

TIA

jobs:
  # increment version, tag and create a release
  release:
    name: Create Release
    ...
    steps:
      - name: Bump version
        uses: ...
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          major_version: 62
        id: bump_version
  build:
    name: Build APK
    needs: release
    ...
    strategy:
      matrix:
        flavor: [sandbox,live]
    steps:
      - name: Checkout
        uses: actions/checkout@v1
      - name: Set version
        run: ./client/scripts/set-version.sh
        env:
          NEW_VERSION: ${{ steps.bump_version.outputs.new_version }}
        id: set_version

[Yanjingzhu]

As jobs run in different runners, those runners are in different machines. Each job runs in a fresh instance of the virtual environment specified by runs-on. It is not possible to share environment variables between machines.  We don’t support to share variables between jobs. 
As you metioned artifacts, we would suggest you use it . You can write the version into a file , upload it as artifact in one job and then download the artifacts in the other job , read the file content. 

Please refer to: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/persisting-workflow-data-using-artifacts 

[eliezercazares]

how about environment variables?
those should be shared as they are higher in hierarchy in the yaml, than the jobs.

[schof]

I hadn’t thought of writing the variable to a file but I guess that could work. Thanks for clarifying that it’s not possible to share information any other way. I do think this could be a feature worth adding at some point. Writing to a file seems like overkill for a single variable. Thanks again. 

[distinctlab]

How to explain therefore that we can define environment variables at a global level as follows (thanks):

name: Create release
 on:
  pull_request:
    types: closed
env:
  NODE_VERSION: "12.x"
  APP_VERSION: "1.0.O"
  ....
Jobs: ....

[Yanjingzhu]

Hi distinctlab, 

Yeah, you could set a workflow level environment variable, which could be used in all jobs. But the original poster needs set the env variable inside a job. Then the env could only be used in that job.  

[aroller]

Thanks for the tip. I found Github documentation provides how to share data between jobs. 

name: Share data between jobs
on: [push]
jobs:

job_1:

name: Add 3 and 7

runs-on: ubuntu-latest

steps:

- shell: bash

run: |

expr 3 + 7 > math-homework.txt

- name: Upload math result for job 1

uses: actions/upload-artifact@v1

with:

name: homework

path: math-homework.txt
job_2:

name: Multiply by 9

needs: job_1

runs-on: windows-latest

steps:

- name: Download math result for job 1

uses: actions/download-artifact@v1

with:

name: homework

- shell: bash

run: |

value=cat homework/math-homework.txt

expr $value * 9 > homework/math-homework.txt

- name: Upload math result for job 2

uses: actions/upload-artifact@v1

with:

name: homework

path: homework/math-homework.txt
job_3:

name: Display results

needs: job_2

runs-on: macOS-latest

steps:

- name: Download math result for job 2

uses: actions/download-artifact@v1

with:

name: homework

- name: Print the final result

shell: bash

run: |

value=cat homework/math-homework.txt

echo The result is $value

[distinctlab]

 Thanks for your feedback. One of the current imits of github actions is to be able to declare from a job a global (workflow)
variable that can be shared with other jobs.
I’m talking about sharing variables not data (artifacts).
Downloasd/upload artifacts actions is not practical to meet this need.

[Yanjingzhu]

Yeah, you are right. Github Actions could not declare a global env in one job and use it in other jobs in a workflow. I would recommend you to share your idea in the Feedback form for GitHub Actions.

[asbjornu]

In the same way it’s possible to share variables between steps, it should be possible to share variables between jobs as well. Since a job can declare its dependence on another job, the dependee could easily have its ouput variables persisted and exposed to dependent jobs through an expansions of the existing steps.<step_id>.outputs.<output_name> syntax with jobs.<job_id>.steps.<step_id>.outputs.<output_name>. Example below:

jobs:
  version_job:
    name: Create version number
    runs-on: ubuntu-latest
    steps:
    - id: version_step
      run: echo "::set-output name=version_number::1.2.3"
output:

name: Output

runs-on: ubuntu-latest

needs: version_job

steps:

- run: echo "${{ jobs.version_job.steps.version_step.outputs.version_number }}"

[NikolozCh]

(Un)fortunately, this feature is deprecated as of Mar. 2024.

[bradymholt]

The new jobs.<jobs_id>.outputs syntax should support this.

jobs:job1:runs-on:ubuntu-latest# Map a step output to a job outputoutputs:output1:${{steps.step1.outputs.test}}output2:${{steps.step2.outputs.test}}steps:-id:step1run:echo"::set-output name=test::hello"-id:step2run:echo"::set-output name=test::world"job2:runs-on:ubuntu-latestneeds:job1steps:-run:echo${{needs.job1.outputs.output1}}${{needs.job1.outputs.output2}}

[cerna]

Thanks for heads up, @bradymholt, this is extremely helpful.

And exactly what I needed.

[cryptable]

Probably way too late, but I use this extension to share variables between jobs.

GitHub

Share Jobs Data - GitHub Marketplace

Share data between GitHub Actions jobs

I use it to release my artifacts into one release, so to store the release URL, which I need in another job.

...
    - name: set upload_url
      id: set_upload_url
      uses: noobly314/share-data@v1
      with:
        share-id: release_url
        mode: set
        key: url
        value: ${{ steps.create_release.outputs.upload_url }}

To get the value so I can use it again.

    - name: get upload_url
      id: get_upload_url
      uses: noobly314/share-data@v1
      with:
        share-id: release_url
        mode: get
        key: url

to access the data you do: ${{ steps.get_upload_url.outputs.data }}.

[dimisjim]

EDIT 2022/10

Due to deprecation of set-output the reccomended approach is now: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter

echo "{name}={value}" >> $GITHUB_OUTPUT

      - name: Set color
        id: random-color-generator
        run: echo "SELECTED_COLOR=green" >> $GITHUB_OUTPUT
      - name: Get color
        run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"

[IanVS]

Does this share the output to other jobs? The example does not indicate that it does. This seems to be the real solution, defining an output map for the job itself: https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs

[dimisjim]

yes, it does.

[kautlenbachs]

maybe I typoed, but I got it working by this - https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs

[SimonDarksideJ]

As a clarification on this answer, the above configuration will ONLY work on Linux distro's using Bash.

For Powershell you need to be more specific

echo "{name}={value}" >> $env:GITHUB_OUTPUT

      - name: Set color
        id: random-color-generator
        run: echo "SELECTED_COLOR=green" >> $env:GITHUB_OUTPUT
      - name: Get color
        run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"

You HAVE to specify the output as an environment variable for Powershell or other scripting pipelines with the "env" prefix

Hope this helps

[MostefaKamalLala]

@ dimisjim response should be the solution. Please change it just in case someone doesn’t scroll down enough!

[m4heshd]

@MostefaKamalLala I flagged the current solution and got the staff to change the solution. 👏t2:

[SimonDarksideJ]

Agreed, it’s a shame there is not a more flexible solution for sharing data between jobs, as even the parameters option has so many limitations (ran in to another one today)

In short:

• If it’s a single parameter, string (not JSON) or int, then you need to add boilerplate to surface outputs from steps to jobs, and then from Jobs to jobs. However, a Job needs to “Depend” on another job to share the data (at least what I’ve found)

• If it is more data than a single string / int, then you need to save that information to a file in an area sharable by different jobs, as each job can / will clear out the working folder between runs (or may be run on separate machines)

• In reality, if there is a larger requirement for sharing data, you will need to invest time in some sort of state backend API that jobs can post and read data from.

Here’s hoping for more improvement in this area.

[ukreddy-erwin]

I tried similar but not working for me.

jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: DevBuild1
outputs:
  output1: ${{ steps.build_number.outputs.buildNumber }}
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
  # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
  - uses: actions/checkout@v3
    with:
      fetch-depth: 0
       
  # Runs a single command using the runners shell
  - name: git commit count
    run: |
      pwd
      echo "checking count"
      $count = git log --oneline | wc -l
      echo $count
      echo "git_commit_count=$count" >> $env:GITHUB_ENV
  - name: Print Build Number
    run: echo ${{ env.git_commit_count }}
  - name: export Build Number
    id: build_number
    run: echo "::set-output name=buildNumber::${{ env.git_commit_count }}"

createInstaller:

# The type of runner that the job will run on

runs-on: DevBuild2

needs:

- build

env:

git_commit_count: ${{needs.build.outputs.buildNumber}}

# Steps represent a sequence of tasks that will be executed as part of the job

steps:

- name: Print build number

run: echo ${{ env.git_commit_count }}

It is failing as value not detected.

echo
MissingMandatoryParameter,Microsoft.PowerShell.Commands.WriteOutputCommand

[airtower-luna]

ukreddy-erwin:

git_commit_count: ${{needs.build.outputs.buildNumber}}

There’s no output called buildNumber in the build job, the one that exists is called output1.

[alienxp03]

Unfortunately, using output is deprecated. It's being replaced by Environment Files. However, it doesn't look like it supports sharing env between jobs. Only sharing env between steps is supported right now. So it seems like we're back to square one again.

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#environment-files

[airtower-luna]

That's incorrect. The set-output command is deprecated, but not outputs as such. You just have to set them by writing to the file given in the GITHUB_OUTPUT environment variable instead, see: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter

The format for that new file is indeed the same as for the one already used for setting environment variables.

[alienxp03]

Ah. That's true. Thanks. I missed that part it seems.

[OctavianCahyadi]

still cannot get GITHUB_OUTPUT env value at jobs level @airtower-luna

build-image:
    runs-on: ubuntu-latest
    steps:
         //other steps
       - name: Build the Docker image
          id: image
          run: |
            export VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)-dev
            echo "TAG=${VERSION}" >> $GITHUB_OUTPUT

 update-deployment:
    needs: build-image
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Update Development project using Webhook
        run: |
          export TAG=${{ needs.build-image.steps.image.outputs.TAG }}

[airtower-luna]

The step output from a previous job won't be available, you need to create a job output from the step output. See: https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs

[drsm79]

Does this work in a matrix job? If I have a matrix the steps are the same, but run with different configurations. How does that appear if the output is "flat"?

[joshbondaval]

I couldn't get this to work if the job that you want to output is ran with uses. Does anyone know if outputs are supported if the job that
performs the output is declared with uses property?

For example:

• The first job runs with uses which is referencing a shared workflow template. This job declares the outputs as described here.
• The second job runs with steps and cannot obtain the outputs from the first job.

Here is an example of the main file:

on:
  pull_request:
    branches:
      - main

jobs:
  // ...
  upload: // <-- this job has the output as declared in upload_main_revision.yaml
    uses: bondaval/.github/.github/workflows/upload_main_revision.yml@main
    secrets: inherit
    with:
      app_name: app

  version:
    runs-on: ubuntu-latest
    needs: upload
    steps:
      - name: test if we can see version
        env:
          VERSION: ${{ needs.upload.outputs.VERSION }}
        run: |
          echo "$VERSION" // <--- Does not output VERSION as set by the 'upload' job.

This is the file that delcares the steps in upload job.

// upload_main_revision.yml
on:
  workflow_call:
    inputs:
      app_name:
        required: true
        type: string

jobs:
  upload:
    runs-on: ubuntu-latest
    outputs:
      VERSION: ${{ steps.output-build-number.outputs.VERSION }}
    steps:
       // ...
      - name: Create build version // <--- creates the next build version.
        id: version
        uses: paulhatch/semantic-version@v4.0.3
        with:
          // ...

      - name: Set build version
        run: echo "VERSION=${{ steps.version.outputs.version }}" >> $GITHUB_ENV

      - name: Output build number
        id: output-build-number
        run: echo "VERSION=${{ env.VERSION }}" >> "$GITHUB_OUTPUT"

[pavelfomin]

I couldn't get this to work if the job that you want to output is ran with uses. Does anyone know if outputs are supported if the job that performs the output is declared with uses property?

For example:

• The first job runs with uses which is referencing a shared workflow template. This job declares the outputs as described here.
• The second job runs with steps and cannot obtain the outputs from the first job.

Here is an example of the main file:

on:
  pull_request:
    branches:
      - main

jobs:
  // ...
  upload: // <-- this job has the output as declared in upload_main_revision.yaml
    uses: bondaval/.github/.github/workflows/upload_main_revision.yml@main
    secrets: inherit
    with:
      app_name: app

  version:
    runs-on: ubuntu-latest
    needs: upload
    steps:
      - name: test if we can see version
        env:
          VERSION: ${{ needs.upload.outputs.VERSION }}
        run: |
          echo "$VERSION" // <--- Does not output VERSION as set by the 'upload' job.

This is the file that delcares the steps in upload job.

// upload_main_revision.yml
on:
  workflow_call:
    inputs:
      app_name:
        required: true
        type: string

jobs:
  upload:
    runs-on: ubuntu-latest
    outputs:
      VERSION: ${{ steps.output-build-number.outputs.VERSION }}
    steps:
       // ...
      - name: Create build version // <--- creates the next build version.
        id: version
        uses: paulhatch/semantic-version@v4.0.3
        with:
          // ...

      - name: Set build version
        run: echo "VERSION=${{ steps.version.outputs.version }}" >> $GITHUB_ENV

      - name: Output build number
        id: output-build-number
        run: echo "VERSION=${{ env.VERSION }}" >> "$GITHUB_OUTPUT"

you are missing:

on:
  workflow_dispatch:
  workflow_call:
    outputs:
      VERSION:
        value: ${{ jobs.upload.outputs.VERSION }}