-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Cargo package registry #21888
Add Cargo package registry #21888
Changes from 6 commits
5f5f1bb
0db12a7
9da9d58
ef668a7
a805e10
f216281
0405d2a
16a3128
6b46a48
c53718b
7085cdf
8b287db
7805de6
2cde9ac
f9326c6
2c9a04f
b5fcf44
d391bdf
ecf5378
d026426
cc5848b
c43d84e
51cd55d
1f65da1
002d86b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
--- | ||
date: "2022-11-20T00:00:00+00:00" | ||
title: "Cargo Packages Repository" | ||
slug: "packages/cargo" | ||
draft: false | ||
toc: false | ||
menu: | ||
sidebar: | ||
parent: "packages" | ||
name: "Cargo" | ||
weight: 5 | ||
identifier: "cargo" | ||
--- | ||
|
||
# Cargo Packages Repository | ||
|
||
Publish [Cargo](https://doc.rust-lang.org/stable/cargo/) packages for your user or organization. | ||
|
||
**Table of Contents** | ||
|
||
{{< toc >}} | ||
|
||
## Requirements | ||
|
||
To work with the Cargo package registry, you need [Rust and Cargo](https://www.rust-lang.org/tools/install). | ||
|
||
Cargo stores informations about the available packages in a package index stored in a git repository. | ||
This repository is needed to work with the registry. | ||
The following section describes how to create it. | ||
|
||
## Index Repository | ||
|
||
Cargo stores informations about the available packages in a package index stored in a git repository. | ||
In Gitea this repository has the special name `_cargo-index`. | ||
After a package was uploaded, its metadata is automatically written to the index. | ||
The content of this repository should not be manually modified. | ||
|
||
The user or organization package settings page allows to create the index repository along with the configuration file. | ||
If needed this action will rewrite the configuration file. | ||
This can be useful if for example the Gitea instance domain was changed. | ||
|
||
If the case arises where the packages stored in Gitea and the information in the index repository are out of sync, the settings page allows to rebuild the index repository. | ||
This action iterates all packages in the registry and writes their information to the index. | ||
If there are lot of packages this process may take some time. | ||
|
||
## Configuring the package registry | ||
|
||
To register the package registry the Cargo configuration must be updated. | ||
Add the following text to the configuration file located in the current users home directory (for example `~/.cargo/config.toml`): | ||
|
||
``` | ||
[registry] | ||
default = "gitea" | ||
|
||
[registries.gitea] | ||
index = "https://gitea.example.com/{owner}/_cargo-index.git" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name maybe conflicted with exist repository. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have chosen the underscore to signal that this is something special, managed. I'm uncertain how to call it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm... Does this repository ever have to accept pushes? Would it make sense for a user to ever want to push to this repository? Does the repo need to have history? If no pushes we could do something like mount it on {owner}/packages/cargo.git and only allow read access externally. If there's no need for history at all we can do something even cleverer like actually just emulate git. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The cargo client does a pull on that repo to get the latest updates. There is no need for a user to push to that repo. I thought about new routes too but decided otherwise because lots of our code wants to see a repo model and there would be none. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No push, no UI view, http pull only(no ssh) and if the contents of the index file are human-readable? If it's machine-readable, it's no necessary to display them in UI. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So if one owner have many cargo packages, will all the indexes be stored in the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yes, it's an index for all packages available in the registry. Here is the official crates.io index: https://github.com/rust-lang/crates.io-index
Yep, the reason is Github does not support Cargo packages. 😄 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like every index file is a JSON text file and that repository also contains a workflow file. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, they are build in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @zeripath Do you see a realistic chance to extract the git routes to work without the need for a valid |
||
|
||
[net] | ||
git-fetch-with-cli = true | ||
``` | ||
|
||
| Parameter | Description | | ||
| --------- | ----------- | | ||
| `owner` | The owner of the package. | | ||
|
||
If the registry is private or you want to publish new packages, you have to configure your credentials. | ||
Add the credentials section to the credentials file located in the current users home directory (for example `~/.cargo/credentials.toml`): | ||
|
||
``` | ||
[registries.gitea] | ||
token = "Bearer {token}" | ||
``` | ||
|
||
| Parameter | Description | | ||
| --------- | ----------- | | ||
| `token` | Your [personal access token]({{< relref "doc/developers/api-usage.en-us.md#authentication" >}}) | | ||
|
||
## Publish a package | ||
|
||
Publish a package by running the following command in your project: | ||
|
||
```shell | ||
cargo publish | ||
``` | ||
|
||
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first. | ||
|
||
## Install a package | ||
|
||
To install a package from the package registry, execute the following command: | ||
|
||
```shell | ||
cargo add {package_name} | ||
``` | ||
|
||
| Parameter | Description | | ||
| -------------- | ----------- | | ||
| `package_name` | The package name. | | ||
|
||
## Supported commands | ||
|
||
``` | ||
cargo publish | ||
cargo add | ||
cargo install | ||
cargo yank | ||
cargo unyank | ||
cargo search | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,144 @@ | ||
// Copyright 2022 The Gitea Authors. All rights reserved. | ||
// Use of this source code is governed by a MIT-style | ||
// license that can be found in the LICENSE file. | ||
|
||
package cargo | ||
|
||
import ( | ||
"encoding/binary" | ||
"errors" | ||
"io" | ||
"regexp" | ||
|
||
"code.gitea.io/gitea/modules/json" | ||
"code.gitea.io/gitea/modules/validation" | ||
|
||
"github.com/hashicorp/go-version" | ||
) | ||
|
||
const PropertyYanked = "cargo.yanked" | ||
|
||
var ( | ||
ErrInvalidName = errors.New("package name is invalid") | ||
ErrInvalidVersion = errors.New("package version is invalid") | ||
) | ||
|
||
// Package represents a Cargo package | ||
type Package struct { | ||
Name string | ||
Version string | ||
Metadata *Metadata | ||
Content io.Reader | ||
} | ||
|
||
// Metadata represents the metadata of a Cargo package | ||
type Metadata struct { | ||
Dependencies []*Dependency `json:"dependencies,omitempty"` | ||
Features map[string][]string `json:"features,omitempty"` | ||
Authors []string `json:"authors,omitempty"` | ||
Description string `json:"description,omitempty"` | ||
DocumentationURL string `json:"documentation_url,omitempty"` | ||
ProjectURL string `json:"project_url,omitempty"` | ||
Readme string `json:"readme,omitempty"` | ||
Keywords []string `json:"keywords,omitempty"` | ||
Categories []string `json:"categories,omitempty"` | ||
License string `json:"license,omitempty"` | ||
RepositoryURL string `json:"repository_url,omitempty"` | ||
Links string `json:"links,omitempty"` | ||
} | ||
|
||
type Dependency struct { | ||
Name string `json:"name"` | ||
VersionReq string `json:"version_req"` | ||
Features []string `json:"features"` | ||
Optional bool `json:"optional"` | ||
DefaultFeatures bool `json:"default_features"` | ||
Target string `json:"target"` | ||
Kind string `json:"kind"` | ||
Registry string `json:"registry"` | ||
ExplicitNameInToml string `json:"explicit_name_in_toml"` | ||
} | ||
|
||
var nameMatch = regexp.MustCompile(`\A[a-zA-Z][a-zA-Z0-9-_]{0,63}\z`) | ||
|
||
// ParsePackage reads the metadata and content of a package | ||
func ParsePackage(r io.Reader) (*Package, error) { | ||
var size uint32 | ||
if err := binary.Read(r, binary.LittleEndian, &size); err != nil { | ||
return nil, err | ||
} | ||
|
||
p, err := parsePackage(io.LimitReader(r, int64(size))) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
if err := binary.Read(r, binary.LittleEndian, &size); err != nil { | ||
return nil, err | ||
} | ||
|
||
p.Content = io.LimitReader(r, int64(size)) | ||
KN4CK3R marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
return p, nil | ||
} | ||
|
||
func parsePackage(r io.Reader) (*Package, error) { | ||
var meta struct { | ||
Name string `json:"name"` | ||
Vers string `json:"vers"` | ||
Deps []*Dependency `json:"deps"` | ||
Features map[string][]string `json:"features"` | ||
Authors []string `json:"authors"` | ||
Description string `json:"description"` | ||
Documentation string `json:"documentation"` | ||
Homepage string `json:"homepage"` | ||
Readme string `json:"readme"` | ||
ReadmeFile string `json:"readme_file"` | ||
Keywords []string `json:"keywords"` | ||
Categories []string `json:"categories"` | ||
License string `json:"license"` | ||
LicenseFile string `json:"license_file"` | ||
Repository string `json:"repository"` | ||
Links string `json:"links"` | ||
} | ||
if err := json.NewDecoder(r).Decode(&meta); err != nil { | ||
return nil, err | ||
} | ||
|
||
if !nameMatch.MatchString(meta.Name) { | ||
return nil, ErrInvalidName | ||
} | ||
|
||
if _, err := version.NewSemver(meta.Vers); err != nil { | ||
return nil, ErrInvalidVersion | ||
} | ||
|
||
if !validation.IsValidURL(meta.Homepage) { | ||
meta.Homepage = "" | ||
} | ||
if !validation.IsValidURL(meta.Documentation) { | ||
meta.Documentation = "" | ||
} | ||
if !validation.IsValidURL(meta.Repository) { | ||
meta.Repository = "" | ||
} | ||
|
||
return &Package{ | ||
Name: meta.Name, | ||
Version: meta.Vers, | ||
Metadata: &Metadata{ | ||
Dependencies: meta.Deps, | ||
Features: meta.Features, | ||
Authors: meta.Authors, | ||
Description: meta.Description, | ||
DocumentationURL: meta.Documentation, | ||
ProjectURL: meta.Homepage, | ||
Readme: meta.Readme, | ||
Keywords: meta.Keywords, | ||
Categories: meta.Categories, | ||
License: meta.License, | ||
RepositoryURL: meta.Repository, | ||
Links: meta.Links, | ||
}, | ||
}, nil | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I used this name because other package software like Artifactory prefer Cargo too:
https://www.jfrog.com/confluence/display/JFROG/Cargo+Package+Registry
https://cloudsmith.com/blog/native-cargo-uploads-in-cloudsmith/
https://github.com/sonatype-nexus-community/nexus-repository-cargo
https://hirevo.github.io/alexandrie/introduction.html