Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update GItlab from upstream, update pipelines and jobs schemas to inc…
…orporate all properties (#16) * feat: add streams for group and project variables (MeltanoLabs#64) * Add support for Group Variables and Project Variables * Update README * Add config for fetching group/project variables - defaults both to False * Update README * Fixup trailing commas Co-authored-by: Warren Ersly <warren.ersly@nih.gov> Co-authored-by: Warren Ersly <erslywb@ncbi.nlm.nih.gov> * Update README.md * docs: add hyperlinks to main and legacy branches * feat: Add `ci_config_path` to projects schema (MeltanoLabs#97) * chore(deps): Bump requests from 2.20.0 to 2.31.0 (MeltanoLabs#99) Bumps [requests](https://github.com/psf/requests) from 2.20.0 to 2.31.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/releases">requests's releases</a>.</em></p> <blockquote> <h2>v2.31.0</h2> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>v2.30.0</h2> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>v2.29.0</h2> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's changelog</a>.</em></p> <blockquote> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <h2>2.28.2 (2023-01-12)</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/psf/requests/commit/147c8511ddbfa5e8f71bbf5c18ede0c4ceb3bba4"><code>147c851</code></a> v2.31.0</li> <li><a href="https://github.com/psf/requests/commit/74ea7cf7a6a27a4eeb2ae24e162bcc942a6706d5"><code>74ea7cf</code></a> Merge pull request from GHSA-j8r2-6x86-q33q</li> <li><a href="https://github.com/psf/requests/commit/302225334678490ec66b3614a9dddb8a02c5f4fe"><code>3022253</code></a> test on pypy 3.8 and pypy 3.9 on windows and macos (<a href="https://redirect.github.com/psf/requests/issues/6424">#6424</a>)</li> <li><a href="https://github.com/psf/requests/commit/b639e66c816514e40604d46f0088fbceec1a5149"><code>b639e66</code></a> test on py3.12 (<a href="https://redirect.github.com/psf/requests/issues/6448">#6448</a>)</li> <li><a href="https://github.com/psf/requests/commit/d3d504436ef0c2ac7ec8af13738b04dcc8c694be"><code>d3d5044</code></a> Fixed a small typo (<a href="https://redirect.github.com/psf/requests/issues/6452">#6452</a>)</li> <li><a href="https://github.com/psf/requests/commit/2ad18e0e10e7d7ecd5384c378f25ec8821a10a29"><code>2ad18e0</code></a> v2.30.0</li> <li><a href="https://github.com/psf/requests/commit/f2629e9e3c7ce3c3c8c025bcd8db551101cbc773"><code>f2629e9</code></a> Remove strict parameter (<a href="https://redirect.github.com/psf/requests/issues/6434">#6434</a>)</li> <li><a href="https://github.com/psf/requests/commit/87d63de8739263bbe17034fba2285c79780da7e8"><code>87d63de</code></a> v2.29.0</li> <li><a href="https://github.com/psf/requests/commit/51716c4ef390136b0d4b800ec7665dd5503e64fc"><code>51716c4</code></a> enable the warnings plugin (<a href="https://redirect.github.com/psf/requests/issues/6416">#6416</a>)</li> <li><a href="https://github.com/psf/requests/commit/a7da1ab3498b10ec3a3582244c94b2845f8a8e71"><code>a7da1ab</code></a> try on ubuntu 22.04 (<a href="https://redirect.github.com/psf/requests/issues/6418">#6418</a>)</li> <li>Additional commits viewable in <a href="https://github.com/psf/requests/compare/v2.20.0...v2.31.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/MeltanoLabs/tap-gitlab/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Managed by Pulumi * Managed by Pulumi * chore(deps): Bump requests from 2.31.0 to 2.32.0 (MeltanoLabs#101) Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/releases">requests's releases</a>.</em></p> <blockquote> <h2>v2.32.0</h2> <h2>2.32.0 (2024-05-20)</h2> <h2>🐍 PYCON US 2024 EDITION 🐍</h2> <p><strong>Security</strong></p> <ul> <li>Fixed an issue where setting <code>verify=False</code> on the first request from a Session will cause subsequent requests to the <em>same origin</em> to also ignore cert verification, regardless of the value of <code>verify</code>. (<a href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li> </ul> <p><strong>Improvements</strong></p> <ul> <li><code>verify=True</code> now reuses a global SSLContext which should improve request time variance between first and subsequent requests. It should also minimize certificate load time on Windows systems when using a Python version built with OpenSSL 3.x. (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li>Requests now supports optional use of character detection (<code>chardet</code> or <code>charset_normalizer</code>) when repackaged or vendored. This enables <code>pip</code> and other projects to minimize their vendoring surface area. The <code>Response.text()</code> and <code>apparent_encoding</code> APIs will default to <code>utf-8</code> if neither library is present. (<a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li> </ul> <p><strong>Bugfixes</strong></p> <ul> <li>Fixed bug in length detection where emoji length was incorrectly calculated in the request content-length. (<a href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li> <li>Fixed deserialization bug in JSONDecodeError. (<a href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li> <li>Fixed bug where an extra leading <code>/</code> (path separator) could lead urllib3 to unnecessarily reparse the request URI. (<a href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li> </ul> <p><strong>Deprecations</strong></p> <ul> <li>Requests has officially added support for CPython 3.12 (<a href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li> <li>Requests has officially added support for PyPy 3.9 and 3.10 (<a href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li> <li>Requests has officially dropped support for CPython 3.7 (<a href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li> <li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li> </ul> <p><strong>Documentation</strong></p> <ul> <li>Various typo fixes and doc improvements.</li> </ul> <p><strong>Packaging</strong></p> <ul> <li>Requests has started adopting some modern packaging practices. The source files for the projects (formerly <code>requests</code>) is now located in <code>src/requests</code> in the Requests sdist. (<a href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li> <li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517 build system using <code>hatchling</code>. This should not impact the average user, but extremely old versions of packaging utilities may have issues with the new packaging format.</li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/matthewarmand"><code>@matthewarmand</code></a> made their first contribution in <a href="https://redirect.github.com/psf/requests/pull/6258">psf/requests#6258</a></li> <li><a href="https://github.com/cpzt"><code>@cpzt</code></a> made their first contribution in <a href="https://redirect.github.com/psf/requests/pull/6456">psf/requests#6456</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's changelog</a>.</em></p> <blockquote> <h2>2.32.0 (2024-05-20)</h2> <p><strong>Security</strong></p> <ul> <li>Fixed an issue where setting <code>verify=False</code> on the first request from a Session will cause subsequent requests to the <em>same origin</em> to also ignore cert verification, regardless of the value of <code>verify</code>. (<a href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li> </ul> <p><strong>Improvements</strong></p> <ul> <li><code>verify=True</code> now reuses a global SSLContext which should improve request time variance between first and subsequent requests. It should also minimize certificate load time on Windows systems when using a Python version built with OpenSSL 3.x. (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li>Requests now supports optional use of character detection (<code>chardet</code> or <code>charset_normalizer</code>) when repackaged or vendored. This enables <code>pip</code> and other projects to minimize their vendoring surface area. The <code>Response.text()</code> and <code>apparent_encoding</code> APIs will default to <code>utf-8</code> if neither library is present. (<a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li> </ul> <p><strong>Bugfixes</strong></p> <ul> <li>Fixed bug in length detection where emoji length was incorrectly calculated in the request content-length. (<a href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li> <li>Fixed deserialization bug in JSONDecodeError. (<a href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li> <li>Fixed bug where an extra leading <code>/</code> (path separator) could lead urllib3 to unnecessarily reparse the request URI. (<a href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li> </ul> <p><strong>Deprecations</strong></p> <ul> <li>Requests has officially added support for CPython 3.12 (<a href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li> <li>Requests has officially added support for PyPy 3.9 and 3.10 (<a href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li> <li>Requests has officially dropped support for CPython 3.7 (<a href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li> <li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li> </ul> <p><strong>Documentation</strong></p> <ul> <li>Various typo fixes and doc improvements.</li> </ul> <p><strong>Packaging</strong></p> <ul> <li>Requests has started adopting some modern packaging practices. The source files for the projects (formerly <code>requests</code>) is now located in <code>src/requests</code> in the Requests sdist. (<a href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li> <li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517 build system using <code>hatchling</code>. This should not impact the average user, but extremely old versions of packaging utilities may have issues with the new packaging format.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/psf/requests/commit/d6ebc4a2f1f68b7e355fb7e4dd5ffc0845547f9f"><code>d6ebc4a</code></a> v2.32.0</li> <li><a href="https://github.com/psf/requests/commit/9a40d1277807f0a4f26c9a37eea8ec90faa8aadc"><code>9a40d12</code></a> Avoid reloading root certificates to improve concurrent performance (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li><a href="https://github.com/psf/requests/commit/0c030f78d24f29a459dbf39b28b4cc765e2153d7"><code>0c030f7</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a> from nateprewitt/no_char_detection</li> <li><a href="https://github.com/psf/requests/commit/555b870eb19d497ddb67042645420083ec8efb02"><code>555b870</code></a> Allow character detection dependencies to be optional in post-packaging steps</li> <li><a href="https://github.com/psf/requests/commit/d6dded3f00afcf56a7e866cb0732799045301eb0"><code>d6dded3</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6700">#6700</a> from franekmagiera/update-redirect-to-invalid-uri-test</li> <li><a href="https://github.com/psf/requests/commit/bf24b7d8d17da34be720c19e5978b2d3bf94a53b"><code>bf24b7d</code></a> Use an invalid URI that will not cause httpbin to throw 500</li> <li><a href="https://github.com/psf/requests/commit/2d5f54779ad174035c5437b3b3c1146b0eaf60fe"><code>2d5f547</code></a> Pin 3.8 and 3.9 runners back to macos-13 (<a href="https://redirect.github.com/psf/requests/issues/6688">#6688</a>)</li> <li><a href="https://github.com/psf/requests/commit/f1bb07d39b74d6444e333879f8b8a3d9dd4d2311"><code>f1bb07d</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6687">#6687</a> from psf/dependabot/github_actions/github/codeql-act...</li> <li><a href="https://github.com/psf/requests/commit/60047ade64b0b882cbc94e047198818ab580911e"><code>60047ad</code></a> Bump github/codeql-action from 3.24.0 to 3.25.0</li> <li><a href="https://github.com/psf/requests/commit/31ebb8102c00f8cf8b396a6356743cca4362e07b"><code>31ebb81</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6682">#6682</a> from frenzymadness/pytest8</li> <li>Additional commits viewable in <a href="https://github.com/psf/requests/compare/v2.31.0...v2.32.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/MeltanoLabs/tap-gitlab/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: add on_predicate decorator to handle 429 (MeltanoLabs#102) ## Description This MR aims to cover the case HTTP `429` "Too Many Requests" ([RFC685](https://www.rfc-editor.org/rfc/rfc6585#section-4))i n a more specific way rather than the current way to handle any exception between the HTTP status code 400 and 500. This is motivated by the issue opened by @kgpayne [here](MeltanoLabs#94). ## Implementation The first decorator to be executed and added on top will be only evaluated when the HTTP code is 429 and then backoff whatever time is in the `Retry-After` header that gitlab returns (see [here](https://docs.gitlab.com/ee/administration/settings/user_and_ip_rate_limits.html#response-headers)). The example is literally an implementation of the example supplied in the README.md from the backoff library [here](https://github.com/litl/backoff?tab=readme-ov-file#backoffruntime) + adding the same amount of max_times. --------- Co-authored-by: Alejandro Martinez <alejandroa@kannactnow.com> Co-authored-by: Edgar Ramírez Mondragón <16805946+edgarrmondragon@users.noreply.github.com> * fix: Bump `backoff` to 2.2.1 (MeltanoLabs#103) Follows from MeltanoLabs#102 * fix: Bump singer-python to 6.0.1 (MeltanoLabs#104) Related: * MeltanoLabs#102 * MeltanoLabs#103 * Fix things after merge * Update schemas for pipelines and jobs --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: wersly <we270@nyu.edu> Co-authored-by: Warren Ersly <warren.ersly@nih.gov> Co-authored-by: Warren Ersly <erslywb@ncbi.nlm.nih.gov> Co-authored-by: Aaron ("AJ") Steers <aj@meltano.com> Co-authored-by: Francis Potter <francispotter@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: MeltyBot <105875157+MeltyBot@users.noreply.github.com> Co-authored-by: Alejandro Martínez Otal <amartinezotal@gmail.com> Co-authored-by: Alejandro Martinez <alejandroa@kannactnow.com> Co-authored-by: Edgar Ramírez Mondragón <16805946+edgarrmondragon@users.noreply.github.com>
- Loading branch information
11 people
authored
Jun 27, 2024
1 parent
1c2bdd4
commit f2125be