GitHub Workflow Failures in Langchain Integrations After PR 5733

[rameshkrishna]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

Here is the current implementation that's causing integration test workflow failures:

# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

name: Unit Tests (LangChain Integrations)

on:
  push:
    branches: ["main"]
  pull_request:
    # Only run when LangChain Core, or integration package code changes.
    paths:
      - 'langchain-core/**'
      - 'libs/**/**'
  workflow_dispatch:  # Allows triggering the workflow manually in GitHub UI


# If another push to the same PR or branch happens while this workflow is still running,
# cancel the earlier run in favor of the next run.
#
# There's no point in testing an outdated version of the code. GitHub only allows
# a limited number of job runners to be active at the same time, so it's better to cancel
# pointless jobs early so that more useful jobs can run sooner.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  get-changed-files:
    runs-on: ubuntu-latest
    outputs:
      changed_files: ${{ steps.get_changes.outputs.changed_files }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 2
      - name: Get changes
        id: get_changes
        run: |
          echo "changed_files<<EOF" >> $GITHUB_OUTPUT
          git diff --name-only -r HEAD^1 HEAD | while read line; do printf "%s\n" "$line"; done >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

  unit-tests:
    name: Unit Tests
    needs: get-changed-files
    strategy:
      matrix:
        os: [ubuntu-latest]
        node-version: [18.x, 20.x]
        package: [anthropic, azure-openai, cloudflare, cohere, community, exa, google-common, google-gauth, google-genai, google-vertexai, google-vertexai-web, google-webauth, groq, mistralai, mongo, nomic, openai, pinecone, qdrant, redis, textsplitters, weaviate, yandex]
        # See Node.js release schedule at https://nodejs.org/en/about/releases/
        # include:
          # - os: windows-latest
          #   node-version: 20.x
          # - os: macos-latest
          #   node-version: 20.x
    runs-on: ${{ matrix.os }}
    if: contains(needs.get-changed-files.outputs.changed_files, 'langchain-core/') || contains(needs.get-changed-files.outputs.changed_files, 'libs/langchain-${{ matrix.package }}/')
    env:
      PUPPETEER_SKIP_DOWNLOAD: "true"
      PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
          cache: "yarn"
      - name: Install dependencies
        run: yarn install --immutable
      - name: Test '@langchain/${{ matrix.package }}' package
        run: yarn test:unit:ci --filter=@langchain/${{ matrix.package }}

Here is my implementation to fix the issue.

# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

name: Unit Tests (LangChain Integrations)

on:
  push:
    branches: ["main"]
  pull_request:
    # Only run when LangChain Core, or integration package code changes.
    paths:
      - 'langchain-core/**'
      - 'libs/**/**'
  workflow_dispatch:  # Allows triggering the workflow manually in GitHub UI


# If another push to the same PR or branch happens while this workflow is still running,
# cancel the earlier run in favor of the next run.
#
# There's no point in testing an outdated version of the code. GitHub only allows
# a limited number of job runners to be active at the same time, so it's better to cancel
# pointless jobs early so that more useful jobs can run sooner.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  get-changed-files:
    runs-on: ubuntu-latest
    outputs:
      changed_files: ${{ steps.get_changes.outputs.changed_files }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 2
      - name: Get changes
        id: get_changes
        run: |
          echo "changed_files<<EOF" >> $GITHUB_OUTPUT
          git diff --name-only -r HEAD^1 HEAD | while read line; do printf "%s\n" "$line"; done >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

  prepare-matrix:
    needs: get-changed-files
    runs-on: ubuntu-latest
    env:
      PACKAGES: "anthropic,azure-openai,cloudflare,cohere,core,community,exa,google-common,google-gauth,google-genai,google-vertexai,google-vertexai-web,google-webauth,groq,mistralai,mongo,nomic,openai,pinecone,qdrant,redis,textsplitters,weaviate,yandex"
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - id: set-matrix
        run: |
          changed_files="${{ needs.get-changed-files.outputs.changed_files }}"
          echo "Changed files: $changed_files"
          # Convert PACKAGES environment variable into an array
          IFS=',' read -r -a packages <<< "$PACKAGES"
          echo "Packages: ${packages[*]}"
          matrix="{\"include\": ["
          first_entry=true
          for package in "${packages[@]}"; do
            echo "Checking package: $package"
            if echo "$changed_files" | grep -q "$package"; then
              echo "Package $package found in changed files."
              if [ "$first_entry" = true ]; then
              matrix="$matrix{\"os\": \"ubuntu-latest\", \"node-version\": \"18\", \"package\": \"$package\"},{\"os\": \"ubuntu-latest\", \"node-version\": \"20\", \"package\": \"$package\"}"
              first_entry=false
            else
              matrix="$matrix, {\"os\": \"ubuntu-latest\", \"node-version\": \"18\", \"package\": \"$package\"},{\"os\": \"ubuntu-latest\", \"node-version\": \"20\", \"package\": \"$package\"}"
            fi
            fi
          done
          matrix="$matrix]}"
          echo "Matrix: $matrix"
          echo "matrix=$matrix" >> $GITHUB_OUTPUT

  unit-tests:
    name: Unit Tests
    needs: prepare-matrix
    # Only run this job if there are packages to test
    if: ${{ fromJson(needs.prepare-matrix.outputs.matrix).include.length != 0 }}
    runs-on: ubuntu-latest
    strategy:
      matrix: ${{ fromJson(needs.prepare-matrix.outputs.matrix) }}
    env:
      PUPPETEER_SKIP_DOWNLOAD: "true"
      PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
          cache: "yarn"
      - name: Install dependencies
        run: yarn install --immutable
      - name: Test '@langchain/${{ matrix.package }}' package
        run: yarn test:unit:ci --filter=@langchain/${{ matrix.package }}

Error Message and Stack Trace (if applicable)

Annotations
1 error
Invalid workflow file: .github/workflows/unit-tests-integrations.yml#L59The workflow is not valid. .github/workflows/unit-tests-integrations.yml (Line: 59, Col: 9): Unrecognized named-value: 'matrix'. Located at position 1 within expression: matrix.packageShow less -- Annotations 1 error [Invalid workflow file: .github/workflows/unit-tests-integrations.yml#L59](https://github.com/langchain-ai/langchainjs/actions/runs/9488089685/workflow) The workflow is not valid. .github/workflows/unit-tests-integrations.yml (Line: 59, Col: 9): Unrecognized named-value: 'matrix'. Located at position 1 within expression: matrix.package

Description

• I'm working on adding new integration to Langchain Lib,
• The GitHub workflow for unit tests in LangChain.js integrations is failing.
• I expect the workflow to correctly run the tests.
• Instead, the workflow fails with an error related to an unrecognized named-value 'matrix'.

Example of recent failure of the workflow on main branch

https://github.com/langchain-ai/langchainjs/actions/runs/9570862663

System Info

Platform : Ubuntu
Node: 18 and 20

$ yarn info

├─ @tsconfig/recommended@npm:1.0.2
│  └─ Version: 1.0.2
│
├─ @types/jest@npm:29.5.3
│  ├─ Version: 29.5.3
│  │
│  └─ Dependencies
│     ├─ expect@npm:^29.0.0 → npm:29.6.1
│     └─ pretty-format@npm:^29.0.0 → npm:29.6.1
│
├─ @types/semver@npm:7.5.6
│  └─ Version: 7.5.6
│
├─ commander@npm:11.1.0
│  └─ Version: 11.1.0
│
├─ dotenv@npm:16.0.3
│  └─ Version: 16.0.3
│
├─ langchainjs@workspace:.
│  ├─ Version: 0.0.4
│  │
│  └─ Dependencies
│     ├─ @tsconfig/recommended@npm:^1.0.2 → npm:1.0.2
│     ├─ @types/jest@npm:^29.5.3 → npm:29.5.3
│     ├─ @types/semver@npm:^7 → npm:7.5.6
│     ├─ commander@npm:^11.1.0 → npm:11.1.0
│     ├─ dotenv@npm:^16.0.3 → npm:16.0.3
│     ├─ lint-staged@npm:^13.1.1 → npm:13.1.4
│     ├─ prettier@npm:^2.8.3 → npm:2.8.4
│     ├─ semver@npm:^7.5.4 → npm:7.5.4
│     ├─ turbo@npm:^1.13.3 → npm:1.13.3
│     └─ typescript@patch:typescript@~5.1.6#~builtin<compat/typescript> → patch:typescript@npm%3A5.1.6#~builtin<compat/typescript>::version=5.1.6&hash=1f5320
│
├─ lint-staged@npm:13.1.4
│  ├─ Version: 13.1.4
│  │
│  ├─ Exported Binaries
│  │  └─ lint-staged
│  │
│  └─ Dependencies
│     ├─ chalk@npm:5.2.0 → npm:5.2.0
│     ├─ cli-truncate@npm:^3.1.0 → npm:3.1.0
│     ├─ commander@npm:^10.0.0 → npm:10.0.0
│     ├─ execa@npm:^7.0.0 → npm:7.0.0
│     ├─ lilconfig@npm:2.1.0 → npm:2.1.0
│     ├─ micromatch@npm:^4.0.5 → npm:4.0.5
│     ├─ normalize-path@npm:^3.0.0 → npm:3.0.0
│     ├─ object-inspect@npm:^1.12.3 → npm:1.12.3
│     ├─ pidtree@npm:^0.6.0 → npm:0.6.0
│     ├─ string-argv@npm:^0.3.1 → npm:0.3.1
│     ├─ supports-color@npm:9.3.1 → npm:9.3.1
│     ├─ yaml@npm:^2.2.1 → npm:2.2.1
│     ├─ debug@npm:^4.3.4 → npm:4.3.4 [d0f46]
│     └─ listr2@npm:^5.0.7 → npm:5.0.7 [d0f46]
│
├─ prettier@npm:2.8.4
│  ├─ Version: 2.8.4
│  │
│  └─ Exported Binaries
│     └─ prettier
│
├─ semver@npm:7.5.4
│  ├─ Version: 7.5.4
│  │
│  ├─ Exported Binaries
│  │  └─ semver
│  │
│  └─ Dependencies
│     └─ lru-cache@npm:^6.0.0 → npm:6.0.0
│
├─ turbo@npm:1.13.3
│  ├─ Version: 1.13.3
│  │
│  ├─ Exported Binaries
│  │  └─ turbo
│  │
│  └─ Dependencies
│     ├─ turbo-darwin-64@npm:1.13.3 → npm:1.13.3
│     ├─ turbo-darwin-arm64@npm:1.13.3 → npm:1.13.3
│     ├─ turbo-linux-64@npm:1.13.3 → npm:1.13.3
│     ├─ turbo-linux-arm64@npm:1.13.3 → npm:1.13.3
│     ├─ turbo-windows-64@npm:1.13.3 → npm:1.13.3
│     └─ turbo-windows-arm64@npm:1.13.3 → npm:1.13.3
│
└─ typescript@patch:typescript@npm%3A5.1.6#~builtin<compat/typescript>::version=5.1.6&hash=1f5320
   ├─ Version: 5.1.6
   │
   └─ Exported Binaries
      ├─ tsc
      └─ tsserver

[dosubot]

Hey there, @rameshkrishna! 👋 I'm here to help you out with any bugs, questions, or contributions while you wait for a human maintainer. Let's squash those bugs together!

To resolve the GitHub workflow failure related to the unrecognized named-value 'matrix' in the unit tests for LangChain.js integrations after PR 5733, you need to ensure that the matrix context is correctly referenced in the workflow file. The provided workflow configuration appears correct, but the error might be due to a syntax issue or a missing context.

Here is the corrected and verified workflow configuration:

name: Unit Tests (LangChain Integrations)

on:
  push:
    branches: ["main"]
  pull_request:
    # Only run when LangChain Core, or integration package code changes.
    paths:
      - 'langchain-core/**'
      - 'libs/**/**'
  workflow_dispatch:  # Allows triggering the workflow manually in GitHub UI

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  get-changed-files:
    runs-on: ubuntu-latest
    outputs:
      changed_files: ${{ steps.get_changes.outputs.changed_files }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 2
      - name: Get changes
        id: get_changes
        run: |
          echo "changed_files<<EOF" >> $GITHUB_OUTPUT
          git diff --name-only -r HEAD^1 HEAD | while read line; do printf "%s\n" "$line"; done >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

  unit-tests:
    name: Unit Tests
    needs: get-changed-files
    strategy:
      matrix:
        os: [ubuntu-latest]
        node-version: [18.x, 20.x]
        package: [anthropic, azure-openai, cloudflare, cohere, community, exa, google-common, google-gauth, google-genai, google-vertexai, google-vertexai-web, google-webauth, groq, mistralai, mongo, nomic, openai, pinecone, qdrant, redis, textsplitters, weaviate, yandex]
    runs-on: ${{ matrix.os }}
    if: contains(needs.get-changed-files.outputs.changed_files, 'langchain-core/') || contains(needs.get-changed-files.outputs.changed_files, 'libs/langchain-${{ matrix.package }}/')
    env:
      PUPPETEER_SKIP_DOWNLOAD: "true"
      PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
          cache: "yarn"
      - name: Install dependencies
        run: yarn install --immutable
      - name: Test '@langchain/${{ matrix.package }}' package
        run: yarn test:unit:ci --filter=@langchain/${{ matrix.package }}

Ensure that the matrix context is correctly referenced in the runs-on and node-version fields, and that the if condition correctly checks for changes in the relevant paths. This should resolve the issue with the unrecognized named-value 'matrix' ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[rameshkrishna]

issue fixed and PR closed #5768