Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add python library using gcp upload #1380

Merged
merged 6 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion docs/advanced/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,18 +144,59 @@ pixi global install pipx
pipx install keyring

# For Google Artifact Registry, also install and initialize its keyring backend.
# Inject this into the pipx environment
pipx inject keyring keyrings.google-artifactregistry-auth --index-url https://pypi.org/simple
gcloud auth login
```

### Using keyring
#### Using keyring with Basic Auth
Use keyring to store your credentials e.g:

```shell
keyring set https://my-index/simple your_username
# prompt will appear for your password
```

##### Configuration
Make sure to include `username@` in the URL of the registry.
An example of this would be:

```toml
[pypi-options]
index-url = "https://username@custom-registry.com/simple"
```

#### GCP
For Google Artifact Registry, you can use the Google Cloud SDK to authenticate.
Make sure to have run `gcloud auth login` before using pixi.
Another thing to note is that you need to add `oauth2accesstoken` to the URL of the registry.
An example of this would be:

##### Configuration

```toml
# rest of the pixi.toml
#
# Add's the following options to the default feature
[pypi-options]
extra-index-urls = ["https://oauth2accesstoken@<location>-python.pkg.dev/<project>/<repository>/simple"]
```

!!!Note
Include the `/simple` at the end, replace the `<location>` etc. with your project and repository and location.
To find this URL more easily, you can use the `gcloud` command:

```shell
gcloud artifacts print-settings python --project=<project> --repository=<repository> --location=<location>
```

#### Installing your environment
To actually install either configure your [Global Config](./global_configuration.md#pypi-configuration), or use the flag:
```shell
pixi install --pypi-keyring-provider subprocess
```


### `.netrc` file

`pixi` allows you to access private registries securely by authenticating with credentials stored in a `.netrc` file.
Expand Down
2 changes: 2 additions & 0 deletions examples/python-library-gcp-keyring/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML
4 changes: 4 additions & 0 deletions examples/python-library-gcp-keyring/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# pixi environments
.pixi
*.egg-info
dist/
7 changes: 7 additions & 0 deletions examples/python-library-gcp-keyring/hi/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from rich import print

def hello():
return "Hello, [bold magenta]World[/bold magenta]!", ":vampire:"

def say_hello():
print(*hello())
4,219 changes: 4,219 additions & 0 deletions examples/python-library-gcp-keyring/pixi.lock

Large diffs are not rendered by default.

79 changes: 79 additions & 0 deletions examples/python-library-gcp-keyring/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Introduction:
#
# This is a small example that builds and uploads a `.whl` to a custom pypi registry
# If you want to upload to PyPI just replace the `--repository-url`
# To use with GCP, make sure you've run a `gcloud auth`

# Using the custom registry:
#
# To use the custom GCP registry, unfortunately you have to have keyring installed globally
# Because `uv` which we use for PyPI resolution uses a subprocess to call `keyring`
# Also you need to add `oauth2accesstoken@` to the url e.g.
# `https://oauth2token@europe-west4-python.pkg.dev/test-artifact-registry-423208/test/`
#
# Example of a feature that uses this:
#
# [feature.hi.pypi-options]
# index-url = "https://oauth2accesstoken@europe-west4-python.pkg.dev/test-artifact-registry-423208/test/"
#
# [feature.hi.pypi-dependencies]
# hi = { version = "==0.1.0" }
#
# To Install:
# Use `pixi install --pypi-keyring-provider subprocess` or configure in global configuration

[project]
name = "hi"
version = "0.1.0"
description = "Add a short description here"
ruben-arts marked this conversation as resolved.
Show resolved Hide resolved
authors = [{ name = "Tim de Jager", email = "tim@prefix.dev" }]
requires-python = ">= 3.11"
dependencies = ["rich"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["osx-arm64", "linux-64", "win-64", "osx-64"]


[tool.pixi.feature.publish.dependencies]
# For building wheel using hatch
hatch = "*"
# Uploading package
twine = "*"
# Used for gcp authentication by twine
# We need this in the environment so that twine running in the
# environment is able to use it
keyring = ">=25.2.0,<25.3"
"keyrings.google-artifactregistry-auth" = ">=1.1.1,<1.2"

# Test feature
[tool.pixi.feature.test.dependencies]
pytest = "*"

# Local install for testing
[tool.pixi.pypi-dependencies]
hi = { path = ".", editable = true }

[tool.pixi.environments]
# Use this env for publishing:
publish = {features = ["publish"], solve-group = "default"}
# Use this env for testing:
test = {features = ["test"], solve-group = "default"}

# This has tasks for building and uploading
# to custom GCP instance
[tool.pixi.feature.publish.tasks]
# Build into a wheel using hatch
build = { cmd = "hatch build", inputs = ["say_hi/*"], outputs = ["dist/*"] }
# Upload to custom GCP registry using twine
publish = { cmd = "twine upload --skip-existing --repository-url https://europe-west4-python.pkg.dev/test-artifact-registry-423208/test/ dist/*", depends-on = [
"build",
] }

# Test the library
[tool.pixi.feature.test.tasks]
test = "pytest tests/"
4 changes: 4 additions & 0 deletions examples/python-library-gcp-keyring/tests/test_hi.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from hi import hello

def test_pixi_py():
assert hello() == ("Hello, [bold magenta]World[/bold magenta]!", ":vampire:")
Loading