Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Alpine package registry #23714

Merged
merged 18 commits into from
May 12, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -2439,6 +2439,8 @@ ROUTER = console
;LIMIT_TOTAL_OWNER_COUNT = -1
;; Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_TOTAL_OWNER_SIZE = -1
;; Maximum size of an Alpine upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_ALPINE = -1
;; Maximum size of a Cargo upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_CARGO = -1
;; Maximum size of a Chef upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1211,6 +1211,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
- `CHUNKED_UPLOAD_PATH`: **tmp/package-upload**: Path for chunked uploads. Defaults to `APP_DATA_PATH` + `tmp/package-upload`
- `LIMIT_TOTAL_OWNER_COUNT`: **-1**: Maximum count of package versions a single owner can have (`-1` means no limits)
- `LIMIT_TOTAL_OWNER_SIZE`: **-1**: Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_ALPINE`: **-1**: Maximum size of an Alpine upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_CARGO`: **-1**: Maximum size of a Cargo upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_CHEF`: **-1**: Maximum size of a Chef upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_COMPOSER`: **-1**: Maximum size of a Composer upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
Expand Down
133 changes: 133 additions & 0 deletions docs/content/doc/usage/packages/alpine.en-us.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
date: "2023-03-25T00:00:00+00:00"
title: "Alpine Packages Repository"
slug: "packages/alpine"
draft: false
toc: false
menu:
sidebar:
parent: "packages"
name: "Alpine"
weight: 4
identifier: "alpine"
---

# Alpine Packages Repository

Publish [Alpine](https://pkgs.alpinelinux.org/) packages for your user or organization.

**Table of Contents**

{{< toc >}}

## Requirements

To work with the Alpine registry, you need to use a HTTP client like `curl` to upload and a package manager like `apk` to consume packages.

The following examples use `apk`.

## Configuring the package registry

To register the Alpine registry add the url to the list of known apk sources (`/etc/apk/repositories`):

```
https://gitea.example.com/api/packages/{owner}/alpine/<branch>/<repository>
```

| Placeholder | Description |
| ------------ | ----------- |
| `owner` | The owner of the packages. |
| `branch` | The branch to use. |
| `repository` | The repository to use. |

If the registry is private, provide credentials in the url. You can use a password or a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}):

```
https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/alpine/<branch>/<repository>
```

The Alpine registry files are signed with a RSA key which must be known to apk. Download the public key and store it in `/etc/apk/keys/`:

```shell
curl -JO https://gitea.example.com/api/packages/{owner}/alpine/key
```

Afterwards update the local package index:

```shell
apk update
```

## Publish a package

To publish an Alpine package (`*.apk`), perform a HTTP `PUT` operation with the package content in the request body.

```
PUT https://gitea.example.com/api/packages/{owner}/alpine/{branch}/{repository}
```

| Parameter | Description |
| ------------ | ----------- |
| `owner` | The owner of the package. |
| `branch` | The branch may match the release version of the OS, ex: `v3.17`. |
| `repository` | The repository can be used [to group packages](https://wiki.alpinelinux.org/wiki/Repositories) or just `main` or similar. |

Example request using HTTP Basic authentication:

```shell
curl --user your_username:your_password_or_token \
--upload-file path/to/file.apk \
https://gitea.example.com/api/packages/testuser/alpine/v3.17/main
```

If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
You cannot publish a file with the same name twice to a package. You must delete the existing package file first.

The server reponds with the following HTTP Status codes.
silverwind marked this conversation as resolved.
Show resolved Hide resolved

| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `201 Created` | The package has been published. |
| `400 Bad Request` | The package name, version, branch, repository or architecture are invalid. |
| `409 Conflict` | A package file with the same combination of parameters exist already in the package. |

## Delete a package

To delete an Alpine package perform a HTTP `DELETE` operation. This will delete the package version too if there is no file left.

```
DELETE https://gitea.example.com/api/packages/{owner}/alpine/{branch}/{repository}/{architecture}/{filename}
```

| Parameter | Description |
| -------------- | ----------- |
| `owner` | The owner of the package. |
| `branch` | The branch to use. |
| `repository` | The repository to use. |
| `architecture` | The package architecture. |
| `filename` | The file to delete.

Example request using HTTP Basic authentication:

```shell
curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/alpine/v3.17/main/test-package-1.0.0.apk
```

The server reponds with the following HTTP Status codes.
silverwind marked this conversation as resolved.
Show resolved Hide resolved
silverwind marked this conversation as resolved.
Show resolved Hide resolved

| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `204 No Content` | Success |
| `404 Not Found` | The package or file was not found. |

## Install a package

To install a package from the Alpine registry, execute the following commands:

```shell
# use latest version
apk add {package_name}
# use specific version
apk add {package_name}={package_version}
```
1 change: 1 addition & 0 deletions docs/content/doc/usage/packages/overview.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The following package managers are currently supported:

| Name | Language | Package client |
| ---- | -------- | -------------- |
| [Alpine]({{< relref "doc/usage/packages/alpine.en-us.md" >}}) | - | `apk` |
| [Cargo]({{< relref "doc/usage/packages/cargo.en-us.md" >}}) | Rust | `cargo` |
| [Chef]({{< relref "doc/usage/packages/chef.en-us.md" >}}) | - | `knife` |
| [Composer]({{< relref "doc/usage/packages/composer.en-us.md" >}}) | PHP | `composer` |
Expand Down
2 changes: 1 addition & 1 deletion docs/content/doc/usage/packages/storage.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ menu:
sidebar:
parent: "packages"
name: "Storage"
weight: 5
weight: 2
identifier: "storage"
---

Expand Down
53 changes: 53 additions & 0 deletions models/packages/alpine/search.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT

package alpine

import (
"context"

packages_model "code.gitea.io/gitea/models/packages"
alpine_module "code.gitea.io/gitea/modules/packages/alpine"
)

// GetBranches gets all available branches
func GetBranches(ctx context.Context, ownerID int64) ([]string, error) {
return packages_model.GetDistinctPropertyValues(
ctx,
packages_model.TypeAlpine,
ownerID,
packages_model.PropertyTypeFile,
alpine_module.PropertyBranch,
nil,
)
}

// GetRepositories gets all available repositories for the given branch
func GetRepositories(ctx context.Context, ownerID int64, branch string) ([]string, error) {
return packages_model.GetDistinctPropertyValues(
ctx,
packages_model.TypeAlpine,
ownerID,
packages_model.PropertyTypeFile,
alpine_module.PropertyRepository,
&packages_model.DistinctPropertyDependency{
Name: alpine_module.PropertyBranch,
Value: branch,
},
)
}

// GetArchitectures gets all available architectures for the given repository
func GetArchitectures(ctx context.Context, ownerID int64, repository string) ([]string, error) {
return packages_model.GetDistinctPropertyValues(
ctx,
packages_model.TypeAlpine,
ownerID,
packages_model.PropertyTypeFile,
alpine_module.PropertyArchitecture,
&packages_model.DistinctPropertyDependency{
Name: alpine_module.PropertyRepository,
Value: repository,
},
)
}
62 changes: 30 additions & 32 deletions models/packages/debian/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -88,44 +88,42 @@ func SearchLatestPackages(ctx context.Context, opts *PackageSearchOptions) ([]*p

// GetDistributions gets all available distributions
func GetDistributions(ctx context.Context, ownerID int64) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, "", debian_module.PropertyDistribution)
return packages.GetDistinctPropertyValues(
ctx,
packages.TypeDebian,
ownerID,
packages.PropertyTypeFile,
debian_module.PropertyDistribution,
nil,
)
}

// GetComponents gets all available components for the given distribution
func GetComponents(ctx context.Context, ownerID int64, distribution string) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyComponent)
return packages.GetDistinctPropertyValues(
ctx,
packages.TypeDebian,
ownerID,
packages.PropertyTypeFile,
debian_module.PropertyComponent,
&packages.DistinctPropertyDependency{
Name: debian_module.PropertyDistribution,
Value: distribution,
},
)
}

// GetArchitectures gets all available architectures for the given distribution
func GetArchitectures(ctx context.Context, ownerID int64, distribution string) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyArchitecture)
}

func getDistinctPropertyValues(ctx context.Context, ownerID int64, distribution, propName string) ([]string, error) {
var cond builder.Cond = builder.Eq{
"package_property.ref_type": packages.PropertyTypeFile,
"package_property.name": propName,
"package.type": packages.TypeDebian,
"package.owner_id": ownerID,
}
if distribution != "" {
innerCond := builder.
Expr("pp.ref_id = package_property.ref_id").
And(builder.Eq{
"pp.ref_type": packages.PropertyTypeFile,
"pp.name": debian_module.PropertyDistribution,
"pp.value": distribution,
})
cond = cond.And(builder.Exists(builder.Select("pp.ref_id").From("package_property pp").Where(innerCond)))
}

values := make([]string, 0, 5)
return values, db.GetEngine(ctx).
Table("package_property").
Distinct("package_property.value").
Join("INNER", "package_file", "package_file.id = package_property.ref_id").
Join("INNER", "package_version", "package_version.id = package_file.version_id").
Join("INNER", "package", "package.id = package_version.package_id").
Where(cond).
Find(&values)
return packages.GetDistinctPropertyValues(
ctx,
packages.TypeDebian,
ownerID,
packages.PropertyTypeFile,
debian_module.PropertyArchitecture,
&packages.DistinctPropertyDependency{
Name: debian_module.PropertyDistribution,
Value: distribution,
},
)
}
3 changes: 3 additions & 0 deletions models/packages/descriptor.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
repo_model "code.gitea.io/gitea/models/repo"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/json"
"code.gitea.io/gitea/modules/packages/alpine"
"code.gitea.io/gitea/modules/packages/cargo"
"code.gitea.io/gitea/modules/packages/chef"
"code.gitea.io/gitea/modules/packages/composer"
Expand Down Expand Up @@ -136,6 +137,8 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc

var metadata interface{}
switch p.Type {
case TypeAlpine:
metadata = &alpine.VersionMetadata{}
case TypeCargo:
metadata = &cargo.Metadata{}
case TypeChef:
Expand Down
6 changes: 6 additions & 0 deletions models/packages/package.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ type Type string

// List of supported packages
const (
TypeAlpine Type = "alpine"
TypeCargo Type = "cargo"
TypeChef Type = "chef"
TypeComposer Type = "composer"
Expand All @@ -51,6 +52,7 @@ const (
)

var TypeList = []Type{
TypeAlpine,
TypeCargo,
TypeChef,
TypeComposer,
Expand All @@ -74,6 +76,8 @@ var TypeList = []Type{
// Name gets the name of the package type
func (pt Type) Name() string {
switch pt {
case TypeAlpine:
return "Alpine"
case TypeCargo:
return "Cargo"
case TypeChef:
Expand Down Expand Up @@ -117,6 +121,8 @@ func (pt Type) Name() string {
// SVGName gets the name of the package type svg image
func (pt Type) SVGName() string {
switch pt {
case TypeAlpine:
return "gitea-alpine"
case TypeCargo:
return "gitea-cargo"
case TypeChef:
Expand Down
38 changes: 38 additions & 0 deletions models/packages/package_property.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import (
"context"

"code.gitea.io/gitea/models/db"

"xorm.io/builder"
)

func init() {
Expand Down Expand Up @@ -81,3 +83,39 @@ func DeletePropertyByName(ctx context.Context, refType PropertyType, refID int64
_, err := db.GetEngine(ctx).Where("ref_type = ? AND ref_id = ? AND name = ?", refType, refID, name).Delete(&PackageProperty{})
return err
}

type DistinctPropertyDependency struct {
Name string
Value string
}

// GetDistinctPropertyValues returns all distinct property values for a given type.
// Optional: Search only in dependence of another property.
func GetDistinctPropertyValues(ctx context.Context, packageType Type, ownerID int64, refType PropertyType, propertyName string, dep *DistinctPropertyDependency) ([]string, error) {
var cond builder.Cond = builder.Eq{
"package_property.ref_type": refType,
"package_property.name": propertyName,
"package.type": packageType,
"package.owner_id": ownerID,
}
if dep != nil {
innerCond := builder.
Expr("pp.ref_id = package_property.ref_id").
And(builder.Eq{
"pp.ref_type": refType,
"pp.name": dep.Name,
"pp.value": dep.Value,
})
cond = cond.And(builder.Exists(builder.Select("pp.ref_id").From("package_property pp").Where(innerCond)))
}

values := make([]string, 0, 5)
return values, db.GetEngine(ctx).
Table("package_property").
Distinct("package_property.value").
Join("INNER", "package_file", "package_file.id = package_property.ref_id").
Join("INNER", "package_version", "package_version.id = package_file.version_id").
Join("INNER", "package", "package.id = package_version.package_id").
Where(cond).
Find(&values)
}
Loading