Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document possible Skip actions. #23943

Merged
merged 32 commits into from
Apr 11, 2022
Merged
Changes from 31 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0d799b0
patching conda version to current. removing pin of azure-mgmt-resources
scbedd Feb 23, 2022
ce5839c
Merge remote-tracking branch 'upstream/main'
scbedd Feb 25, 2022
1620a58
Merge remote-tracking branch 'upstream/main'
scbedd Feb 25, 2022
147885f
Merge remote-tracking branch 'upstream/main'
scbedd Feb 28, 2022
f21c9c3
Merge remote-tracking branch 'upstream/main'
scbedd Mar 1, 2022
1d5f071
Merge remote-tracking branch 'upstream/main'
scbedd Mar 1, 2022
718e502
Merge remote-tracking branch 'upstream/main'
scbedd Mar 2, 2022
5df1ebf
Merge remote-tracking branch 'upstream/main'
scbedd Mar 4, 2022
a6829de
Merge remote-tracking branch 'upstream/main'
scbedd Mar 4, 2022
0b2da34
Merge remote-tracking branch 'upstream/main'
scbedd Mar 4, 2022
0aaa7b7
Merge remote-tracking branch 'upstream/main'
scbedd Mar 8, 2022
7bf7373
Merge remote-tracking branch 'upstream/main'
scbedd Mar 9, 2022
9e1ba78
Merge remote-tracking branch 'upstream/main'
scbedd Mar 9, 2022
93edd4e
Merge remote-tracking branch 'upstream/main'
scbedd Mar 10, 2022
11266d8
Merge remote-tracking branch 'upstream/main'
scbedd Mar 10, 2022
f216537
Merge remote-tracking branch 'upstream/main'
scbedd Mar 14, 2022
16ee22f
Merge remote-tracking branch 'upstream/main'
scbedd Mar 15, 2022
3e7880c
Merge remote-tracking branch 'upstream/main'
scbedd Mar 16, 2022
6268a22
Merge remote-tracking branch 'upstream/main'
scbedd Mar 16, 2022
b1cd835
Merge remote-tracking branch 'upstream/main'
scbedd Mar 21, 2022
6ec282e
Merge remote-tracking branch 'upstream/main'
scbedd Mar 22, 2022
ff4732b
Merge remote-tracking branch 'upstream/main'
scbedd Mar 30, 2022
ea49514
Merge remote-tracking branch 'upstream/main'
scbedd Apr 1, 2022
a302688
Merge remote-tracking branch 'upstream/main'
scbedd Apr 5, 2022
a1ea875
Merge remote-tracking branch 'upstream/main'
scbedd Apr 5, 2022
74c4b3a
Merge remote-tracking branch 'upstream/main'
scbedd Apr 7, 2022
eb524da
Merge remote-tracking branch 'upstream/main'
scbedd Apr 8, 2022
f6237a3
Merge remote-tracking branch 'upstream/main'
scbedd Apr 11, 2022
7336006
include index of possible skips
scbedd Apr 11, 2022
d840d48
dashes not asterisks for lists
scbedd Apr 11, 2022
c6f3213
updating the conda skip to name the variable appropriately
scbedd Apr 11, 2022
7cea12a
Update doc/eng_sys_checks.md
scbedd Apr 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 59 additions & 6 deletions doc/eng_sys_checks.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,34 @@
# Azure SDK for Python - Engineering System

* [Target a specific package](#targeting-a-specific-package-at-build-time)
* [Skip a tox test environment](#skipping-a-tox-test-environment-at-queue-time)
* [Analyze Checks](#analyze-checks)
* [PR Validation Checks](#pr-validation-checks)
* [Nightly CI Checks](#nightly-ci-checks)
* [Nightly Live Checks](#nightly-live-checks)
- [Azure SDK for Python - Engineering System](#azure-sdk-for-python---engineering-system)
- [Targeting a specific package at build time](#targeting-a-specific-package-at-build-time)
- [Skipping a tox test environment at queue time](#skipping-a-tox-test-environment-at-queue-time)
- [Skipping entire sections of builds](#skipping-entire-sections-of-builds)
- [Environment variables important to CI](#environment-variables-important-to-ci)
- [Analyze Checks](#analyze-checks)
- [MyPy](#mypy)
- [Pylint](#pylint)
- [Bandit](#bandit)
- [ApiStubGen](#apistubgen)
- [black](#black)
- [Opt-in to formatting validation](#opt-in-to-formatting-validation)
- [Running locally](#running-locally)
- [Change log verification](#change-log-verification)
- [PR Validation Checks](#pr-validation-checks)
- [PR validation tox test environments](#pr-validation-tox-test-environments)
- [whl](#whl)
- [sdist](#sdist)
- [depends](#depends)
- [Nightly CI Checks](#nightly-ci-checks)
- [Latest Dependency Test](#latest-dependency-test)
- [Minimum Dependency Test](#minimum-dependency-test)
- [Regression Test](#regression-test)
- [Autorest Automation](#autorest-automation)
- [Opt-in to autorest automation](#opt-in-to-autorest-automation)
- [Running locally](#running-locally-1)
- [Nightly Live Checks](#nightly-live-checks)
- [Running Samples](#running-samples)


There are various tests currently enabled in Azure pipeline for Python SDK and some of them are enabled only for nightly CI checks. We also run some static analysis tool to verify code completeness, security and lint check.

Expand Down Expand Up @@ -46,6 +69,36 @@ This is an example setting of that narrows the default set from `whl, sdist, dep

Any combination of valid valid tox environments will work. Reference either this document or the file present at `eng/tox/tox.ini` to find what options are available.

## Skipping entire sections of builds

In certain cases,release engineers may want to disable `APIView` checks prior to releasing. Engineers who need this capability should first clear it with their lead, then set the following build time variable.

- Create variable named `Skip.CreateApiReview`
- Set variable valuable to `true`
scbedd marked this conversation as resolved.
Show resolved Hide resolved

This is the most useful skip, but the following skip variables are also supported. Setting the variable value to `true` should be used for all of the below.

- `Skip.Analyze`
- Skip the `analyze` job entirely.
- `Skip.Test`
- Skip the `test` jobs entirely.
- `Skip.TestConda`
- Skip the `conda test` jobs entirely.
- `Skip.ApiStubGen`
- Entirely omits API stub generation within `build` job.
- `Skip.VerifySdist`
- Omit `twine check` of source distributions in `build` job.
- `Skip.VerifyWhl`
- Omit `twine check` of wheels in `build` job.
- `Skip.Bandit`
- Omit `bandit` checks in `analyze` job.
- `Skip.Pylint`
- Omit linting checks in `analyze` job.
- `Skip.BreakingChanges`
- Don't verify if a changeset includes breaking changes.
- `Skip.MyPy`
- Omit `mypy` checks in `analyze` job.

## Environment variables important to CI

There are a few differences from a standard local invocation of `tox <env>`. Primarily, these differences adjust the checks to be friendly to parallel invocation. These adjustments are necessary to prevent random CI crashes.
Expand Down