Skip to content

Merge from go #347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arjunsuresh merged 14 commits into from
Oct 5, 2024
Merged

Merge from go #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
14f2901
enabled for windows + push to github added
anandhu-eng Oct 5, 2024
f19ce8a
gh action update: enabled windows run + push results to github
anandhu-eng Oct 5, 2024
1102708
Merge branch 'mlperf-inference' into ghactresultpush
arjunsuresh Oct 5, 2024
eeca7eb
Merge branch 'mlperf-inference' into bertdeepsparseghaction
arjunsuresh Oct 5, 2024
a9350ed
Merge branch 'mlperf-inference' into ghactresultpush
arjunsuresh Oct 5, 2024
2cb6de3
Merge branch 'mlperf-inference' into bertdeepsparseghaction
arjunsuresh Oct 5, 2024
4a63a2d
Update test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml
arjunsuresh Oct 5, 2024
ab1b4eb
Merge pull request #145 from anandhu-eng/bertdeepsparseghaction
arjunsuresh Oct 5, 2024
d470b0a
Update test-mlperf-inference-retinanet.yml
arjunsuresh Oct 5, 2024
836e1cd
Merge branch 'mlcommons:mlperf-inference' into mlperf-inference
arjunsuresh Oct 5, 2024
cc7f9dd
Update _cm.yaml
arjunsuresh Oct 5, 2024
9296127
Update test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml
arjunsuresh Oct 5, 2024
c7326b6
Merge branch 'mlperf-inference' into ghactresultpush
arjunsuresh Oct 5, 2024
83cd04d
Merge pull request #144 from anandhu-eng/ghactresultpush
arjunsuresh Oct 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions .github/workflows/test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
name: MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)

on:
pull_request:
pull_request_target:
branches: [ "main", "dev", "mlperf-inference" ]
paths:
- '.github/workflows/test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml'
Expand All @@ -13,20 +13,21 @@ on:

jobs:
build:

runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# 3.12 didn't work on 20240305 - need to check
python-version: [ "3.11" ]
backend: [ "deepsparse", "tf", "onnxruntime", "pytorch" ]
precision: [ "int8", "fp32" ]
os: [ubuntu-latest, windows-latest, macos-latest]
exclude:
- backend: tf
- backend: pytorch
- backend: onnxruntime
- precision: fp32
- os: windows-latest

steps:
- uses: actions/checkout@v3
Expand All @@ -38,6 +39,26 @@ jobs:
run: |
python3 -m pip install cmind
cm pull repo --url=${{ github.event.pull_request.head.repo.html_url }} --checkout=${{ github.event.pull_request.head.ref }}
- name: Test MLPerf Inference Bert (DeepSparse, TF, ONNX, PyTorch)
- name: Test MLPerf Inference Bert ${{ matrix.backend }} on ${{ matrix.os }}
if: matrix.os == 'windows-latest'
run: |
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --hw_name=gh_${{ matrix.os }} --model=bert-99 --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --adr.loadgen.tags=_from-pip --pip_loadgen=yes --precision=${{ matrix.precision }} --target_qps=1 -v --quiet
- name: Test MLPerf Inference Bert ${{ matrix.backend }} on ${{ matrix.os }}
if: matrix.os != 'windows-latest'
run: |
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="cTuning" --model=bert-99 --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --precision=${{ matrix.precision }} --target_qps=1 -v --quiet
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --hw_name=gh_${{ matrix.os }}_x86 --model=bert-99 --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --precision=${{ matrix.precision }} --target_qps=1 -v --quiet
- name: Push Results
if: github.repository_owner == 'gateoverflow'
env:
USER: "GitHub Action"
EMAIL: "admin@gateoverflow.com"
run: |
git config --global user.name "${{ env.USER }}"
git config --global user.email "${{ env.EMAIL }}"
git config --global credential.https://github.com.helper ""
git config --global credential.https://github.com.helper "!gh auth git-credential"
git config --global credential.https://gist.github.com.helper ""
git config --global credential.https://gist.github.com.helper "!gh auth git-credential"

cm run script --tags=auth,gh,cli --with_token="${{ secrets.TEST_RESULTS_GITHUB_TOKEN }}"
cm run script --tags=push,github,mlperf,inference,submission --repo_url=https://github.com/gateoverflow/mlperf_inference_test_submissions_v5.0 --repo_branch=main --commit_message="Results from Bert GH action on ${{ matrix.os }}" --quiet
33 changes: 28 additions & 5 deletions .github/workflows/test-mlperf-inference-retinanet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
name: MLPerf inference retinanet

on:
pull_request:
pull_request_target:
branches: [ "main", "dev", "mlperf-inference" ]
paths:
- '.github/workflows/test-mlperf-inference-retinanet.yml'
Expand All @@ -13,17 +13,19 @@ on:

jobs:
build:

runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: [ "3.12" ]
backend: [ "onnxruntime", "pytorch" ]
implementation: [ "python", "cpp" ]
exclude:
- backend: pytorch
implementation: cpp
- os: windows
implementation: cpp

steps:
- uses: actions/checkout@v3
Expand All @@ -35,6 +37,27 @@ jobs:
run: |
python3 -m pip install cmind
cm pull repo --url=${{ github.event.pull_request.head.repo.html_url }} --checkout=${{ github.event.pull_request.head.ref }}
- name: Test MLPerf Inference Retinanet using ${{ matrix.backend }}
- name: Test MLPerf Inference Retinanet using ${{ matrix.backend }} on ${{ matrix.os }}
if: matrix.os == 'windows-latest'
run: |
git config --system core.longpaths true
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --hw_name=gh_${{ matrix.os }} --model=retinanet --adr.loadgen.tags=_from-pip --pip_loadgen=yes --implementation=${{ matrix.implementation }} --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --quiet -v --target_qps=1
- name: Test MLPerf Inference Retinanet using ${{ matrix.backend }} on ${{ matrix.os }}
if: matrix.os != 'windows-latest'
run: |
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="cTuning" --hw_name=default --model=retinanet --implementation=${{ matrix.implementation }} --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --adr.compiler.tags=gcc --quiet -v --target_qps=1
cm run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --hw_name=gh_${{ matrix.os }}_x86 --model=retinanet --implementation=${{ matrix.implementation }} --backend=${{ matrix.backend }} --device=cpu --scenario=Offline --test_query_count=5 --adr.compiler.tags=gcc --quiet -v --target_qps=1
- name: Push Results
if: github.repository_owner == 'gateoverflow'
env:
USER: "GitHub Action"
EMAIL: "admin@gateoverflow.com"
run: |
git config --global user.name "${{ env.USER }}"
git config --global user.email "${{ env.EMAIL }}"
git config --global credential.https://github.com.helper ""
git config --global credential.https://github.com.helper "!gh auth git-credential"
git config --global credential.https://gist.github.com.helper ""
git config --global credential.https://gist.github.com.helper "!gh auth git-credential"

cm run script --tags=auth,gh,cli --with_token="${{ secrets.TEST_RESULTS_GITHUB_TOKEN }}"
cm run script --tags=push,github,mlperf,inference,submission --repo_url=https://github.com/gateoverflow/mlperf_inference_test_submissions_v5.0 --repo_branch=main --commit_message="Results from Retinanet GH action on ${{ matrix.os }}" --quiet
2 changes: 0 additions & 2 deletions script/get-sys-utils-cm/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ deps:
env:
CM_CLEAN_DIRS: bin
CM_PACKAGE_WIN_URL: https://zenodo.org/records/13868077/files/cm-artifact-os-windows-32.zip?download=1
; https://cKnowledge.org/ai/data/zlib123dllx64-bin.zip
; https://cKnowledge.org/ai/data/xz-5.2.9-win64.zip
CM_SUDO: sudo

input_mapping:
Expand Down