Skip to content

Commit

Permalink
(#15004) docs: split out files and folders and the last few attributes
Browse files Browse the repository at this point in the history 
* docs: organize "adding packages" by files and methods

* typo

* modernize the main readme + second pass to improve language

* [docs] Regenerate tables of contents

Co-authored-by: prince-chrismc <prince-chrismc@users.noreply.github.com>

* update the folder and files to match other recent changes

* Apply suggestions from code review

Co-authored-by: SSE4 <tomskside@gmail.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: prince-chrismc <prince-chrismc@users.noreply.github.com>
Co-authored-by: SSE4 <tomskside@gmail.com>
  • Loading branch information
@github-actions @prince-chrismc @SSE4
4 people authored Jan 25, 2023
1 parent 2ac1597 commit 7129c01
Show file tree
Hide file tree
Showing 5 changed files with 327 additions and 150 deletions.
215 changes: 83 additions & 132 deletions docs/adding_packages/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,180 +3,131 @@
ConanCenterIndex aims to provide the best quality packages of any open source project.
Any C/C++ project can be made available by contributing a "recipe".

Getting started is easy. Try building an existing package with our [developing recipes](../developing_recipes_locally.md) tutorial.
To deepen you understanding, start with the [How to provide a good recipe](#how-to-provide-a-good-recipe) section.
Getting started is easy. Try building an existing package with our [developing recipes](../developing_recipes_locally.md)
tutorial. To deepen you understanding, start with the [How to provide a good recipe](#how-to-provide-a-good-recipe) section.
You can follow the three steps (:one: :two: :three:) described below! :tada:

<!-- toc -->
## Contents

* [Request access](#request-access)
* [:one: Request access](#one-request-access)
* [Inactivity and user removal](#inactivity-and-user-removal)
* [Submitting a Package](#submitting-a-package)
* [The Build Service](#the-build-service)
* [Recipe files structure](#recipe-files-structure)
* [`config.yml`](#configyml)
* [`conandata.yml`](#conandatayml)
* [The _recipe folder_: `conanfile.py`](#the-_recipe-folder_-conanfilepy)
* [Test Folders](#test-folders)
* [Test the recipe locally](#test-the-recipe-locally)
* [Hooks](#hooks)
* [Linters](#linters)<!-- endToc -->

## Request access

:one: The first step to add packages to ConanCenter is requesting access. To enroll in ConanCenter repository, please write a comment
requesting access in this GitHub [issue](https://github.com/conan-io/conan-center-index/issues/4). Feel free to introduce yourself and
* [:two: Creating a package](#two-creating-a-package)
* [How to provide a good recipe](#how-to-provide-a-good-recipe)
* [:three: Submitting a Package](#three-submitting-a-package)
* [The Build Service](#the-build-service)<!-- endToc -->

## :one: Request access

The first step to add packages to ConanCenter is requesting access. To enroll in ConanCenterIndex repository, please write a comment
requesting access in this GitHub [issue #4](https://github.com/conan-io/conan-center-index/issues/4). Feel free to introduce yourself and
your motivation to join ConanCenter community.

This process helps ConanCenter against spam and malicious code. The process is not not automated on purpose and the requests are generally approved
on a weekly basis.
This process helps ConanCenter against spam and malicious code. The process is not fully automated on purpose and the requests are
generally approved on a weekly basis. Feel free to continue to step :two: while waiting for approval.

> **Note** The requests are reviewed manually, checking the GitHub profile activity of the requester to avoid any misuse of the service.
> All interactions are subject to the expectations of the [code of conduct](../code_of_conduct.md). Any misuse or inappropriate behavior are subject
> to the same principals.
> All interactions are subject to the expectations of the [code of conduct](../code_of_conduct.md). Any misuse or inappropriate behavior
> are subject to the same principals.
When submitting a pull request for the first time, you will be prompted to sign the [CLA](../CONTRIBUTOR_LICENSE_AGREEMENT.md) for your code contributions. You can view your signed CLA's by going to <https://cla-assistant.io/> and signing in.
When submitting a pull request for the first time, you will be prompted to sign the [CLA](../CONTRIBUTOR_LICENSE_AGREEMENT.md) for your
code contributions. You can view your signed CLA's by going to <https://cla-assistant.io/> and signing in.

## Inactivity and user removal

For security reasons related to the CI, when a user no longer contributes for a long period, it will be considered inactive and removed from the authorized user's list.
For now, it's configured for **4 months**, and it's computed based on the latest commit, not comments or opened issues.
After that time, the CI bot will ask to remove the user name from the authorized users' list through the access request PR, which occurs a few times every week.
In case you are interested in coming back, please, ask again to be included in the issue [#4](https://github.com/conan-io/conan-center-index/issues/4), the process will be precise like for newcomers.

## Submitting a Package

:two: To contribute a package, you can submit a [Pull Request](https://github.com/conan-io/conan-center-index/pulls) to this GitHub repository https://github.com/conan-io/conan-center-index.

The specific steps to add new packages are:

* Fork the [conan-center-index](https://github.com/conan-io/conan-center-index) git repository, and then clone it locally.
* Copy a template from [package_templates](../package_templates) folder in the recipes/ folder and rename it to the project name (it should be lower-case). Read templates [documentation](../package_templates/README.md) to find more information.
* Make sure you are using a recent [Conan client](https://conan.io/downloads) version, as recipes might evolve introducing features of the newer Conan releases.
* Commit and Push to GitHub then submit a pull request.
* Our automated build service will build 100+ different configurations, and provide messages that indicate if there were any issues found during the pull request on GitHub.

:three: When the pull request is [reviewed and merged](../review_process.md), those packages are published to [JFrog ConanCenter](https://conan.io/center/) and available for everyone.

### The Build Service

The **build service** associated to this repo will generate binary packages automatically for the most common platforms and compilers. See [the Supported Platforms and Configurations page](../supported_platforms_and_configurations.md) for a list of generated configurations. For a C++ library, the system is currently generating more than 100 binary packages.

> ⚠️ **Note**: This not a testing service, it is a binary building service for package **released**. Unit tests shouldn't be built nor run in recipes by default, see the [FAQs](../faqs.md#why-conancenter-does-not-build-and-execute-tests-in-recipes) for more. Before submitting a pull request, please ensure that it works locally for some configurations.
- The CI bot will start a new build only after the author is approved. Your PR may be reviewed in the mean time, but is not guaranteed.
- The CI system will also report with messages in the PR any error in the process, even linking to the logs to see more details and debug.

The pipeline will report errors and build logs by creating a comment in the pull-request after every commit. The message will include links to the logs for inspecting.

Packages generated and uploaded by this build service don't include any _user_ or _channel_ (existing references with any `@user/channel` should be considered as deprecated in favor of packages without it). Once the packages are uploaded, you will be able to install them using the reference as `name/version` (requires Conan >= 1.21): `conan install cmake/3.18.2@`.

## Recipe files structure

Every entry in the `recipes` folder contains all the files required by Conan to create the binaries for all the versions of one library. Those
files don't depend on any other file in the repository (we are not using `python_requires`) and every pull-request can modify only one of those
folders at a time.

This is the canonical structure of one of these folders, where the same `conanfile.py` recipe is suitable to build all the versions of the library:
When you are interested in contributing again, simply ask again to be included in the [issue #4](https://github.com/conan-io/conan-center-index/issues/4).
The process will be precisely like for newcomers.

```
.
+-- recipes
| +-- library_name/
| +-- config.yml
| +-- all/
| +-- conanfile.py
| +-- conandata.yml
| +-- patches/
| +-- add-missing-string-header-2.0.0.patch
| +-- test_package/
| +-- conanfile.py
| +-- CMakeLists.txt
| +-- test_package.cpp
| +-- test_v1_package/
| +-- conanfile.py
| +-- CMakeLists.txt
```
## :two: Creating a package

If it becomes too complex to maintain the logic for all the versions in a single `conanfile.py`, it is possible to split the folder `all` into
two or more folders, dedicated to different versions, each one with its own `conanfile.py` recipe. In any case, those folders should replicate the
same structure.
Once you've successfully built an existing recipe following [developing recipes](../developing_recipes_locally.md) tutorial.
You are set to being adding a new package.

### `config.yml`
Make sure you have:

This file lists the versions and the folders where they are located:
* Forked and then cloned the [conan-center-index](https://github.com/conan-io/conan-center-index) git repository.
* Make sure you are using a recent [Conan client](https://conan.io/downloads) version, as recipes improve by introducing features of the newer Conan releases.

```yml
versions:
"1.1.0":
folder: 1.x.x
"1.1.1":
folder: 1.x.x
"2.0.0":
folder: all
"2.1.0":
folder: all
```
The easiest way to start is copying a template from our [`package_templates/`](../package_templates) folder to the [`recipes/`](../../recipes/) folder.
Rename the new folder following the [project name](conanfile_attributes.md#name) guidelines. Read templates [documentation](../package_templates/README.md)
to find more information.

Quickly, there's a few items to look at:

### `conandata.yml`
* Add _only_ the latest version in the [`config.yml`](folders_and_files.md#configyml) and [`conandata.yml`](folders_and_files.md#conandatayml)
* Make sure to update the [`ConanFile` attributes](conanfile_attributes.md) like `license`, `description`, etc...

This file lists **all the sources that are needed to build the package**: source code, patch files, license files,... any file that will be used by the recipe
should be listed here. The file is organized into two sections, `sources` and `patches`, each one of them contains the files that are required
for each version of the library. All the files that are downloaded from the internet should include a checksum, so we can validate that
they are not changed.
In ConanCenter, our belief is recipes should always match upstream, in other words, what the original author(s) intended.

A detailed breakdown of all the fields can be found in [conandata_yml_format.md](conandata_yml_format.md). We **strongly** encourage adding the [patch fields](conandata_yml_format.md#patches-fields) to help track where patches come from and what issue they solve.
* Options should [follow these recommendations](conanfile_attributes.md#options) as well as match the default value used by the upstream project.
* [Package information](build_and_package.md), libraries, components should match as well. This includes exposing supported build system names.

Inside the `conanfile.py` recipe, this data is available in a `self.conan_data` attribute that can be used as follows:
Where dependencies are involved, there's no shortcuts, inspect the upstream's build scripts for how they are usually consumed. Pick the Conan
generator that matches. The most common example is CMake's `find_package` that can be satisfied by Conan's
[`CMakeDeps`](https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmakedeps.html) generator. There are a few
things to be cautious about, many projects like to "vendor" other projects within them. This can be files checked into the repository or
downloaded during the build process. These should be removed, see the [Dependencies section](dependencies.md#handling-internal-dependencies)
for more information.

```py
def export_sources(self):
export_conandata_patches(self)
### How to provide a good recipe

def source(self):
files.get(self, **self.conan_data["sources"][self.version], destination=self.source_folder, strip_root=True)
Take a look at existing [recipes](https://github.com/conan-io/conan-center-index/tree/master/recipes) available in ConanCenterIndex which can be
used as good examples, you can use them as the base for your recipe. The GitHub search is very good for matching code snippets, you can see if,
how or when a function is used in other recipes.

def build(self):
files.apply_conandata_patches(self)
[...]
```
> **Note**: Conan features change over time and our best practices evolve so some minor details may be out of date due to the vast number of recipes.
### The _recipe folder_: `conanfile.py`
More often than not, ConanCenter recipes are built in more configuration than the upstream project. This means some edge cases need minor tweaks.
We **strongly encourage** everyone to contribute back to the upstream project. This reduces the burden of re-applying patches and overall makes the
the code more accessible.

The main files in this repository are the `conanfile.py` ones that contain the logic to build the libraries from sources for all the configurations,
as we said before there can be one single recipe suitable for all the versions inside the `all` folder, or there can be several recipes targeting
different versions in different folders. For maintenance reasons, we prefer to have only one recipe, but sometimes the extra effort doesn't worth
it and it makes sense to split and duplicate it, there is no common rule for it.
Read the docs! The [FAQs](../faqs.md) are a great place to find short answers.
The documents in this folder are written to explain each folder, file, method, and attribute.

Together with the recipe, there can be other files that are needed to build the library: patches, other files related to build systems,
... all these files will usually be listed in `exports_sources` and used during the build process.
1. [Folders and Files](folders_and_files.md)
2. [Sources and Patches](sources_and_patches.md)
1. [`conandata.yml` format](conandata_yml_format.md)
3. [`ConanFile` Attributes](conanfile_attributes.md)
4. [Dependencies](dependencies.md)
5. [Build and Package](build_and_package.md)
1. [Revisit Patches](sources_and_patches.md#policy-about-patching)
6. [Test Package](test_packages.md)

Also, **every `conanfile.py` should be accompanied by one or several folder to test the generated packages** as we will see below.
The one place you are certain to find a lot of information is <https://docs.conan.io>, this has the documentation for everything in Conan.
There are helpful tutorials for cross-build, detailed explication for profiles and settings and much much more!

### Test Folders
## :three: Submitting a Package

All the packages in this repository need to be tested before they join ConanCenter. A `test_package/` folder with its corresponding `conanfile.py` and
a minimal project to test the package is strictly required. You can read about it in the
[Conan documentation](https://docs.conan.io/en/latest/creating_packages/getting_started.html).
To contribute a package, you can submit a [Pull Request](https://github.com/conan-io/conan-center-index/pulls) to this GitHub
repository <https://github.com/conan-io/conan-center-index>.

Learn more about the ConanCenterIndex requirements in the [test packages](test_packages.md) document.
The specific steps to submitting changes are:

## Test the recipe locally
* Build and test the new recipe in several combinations. Check [developing recipes](../developing_recipes_locally.md) for tips.
* [Commit and push to your fork repository](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository) then
[submit a pull request](https://github.com/conan-io/conan-center-index/compare).
* Our automated [build service](#the-build-service) will build 100+ different configurations, and provide messages that indicate if there
were any issues found during the pull request on GitHub.

### Hooks
When the pull request is [reviewed and merged](../review_process.md), those packages are published to [JFrog's ConanCenter](https://conan.io/center/)
and are made available for everyone.

The system will use the [conan-center hook](https://github.com/conan-io/hooks) to perform some quality checks. These are required for the
the CI to merge any pull request.
## The Build Service

Follow the [Developing Recipes Locally](../developing_recipes_locally.md#installing-the-conancenter-hooks) guide for instructions.
The **build service** associated to this repository will generate binary packages automatically for the most common platforms and compilers.
See [the Supported Platforms and Configurations page](../supported_platforms_and_configurations.md) for a list of generated configurations.
For a C++ library, the system is currently generating more than 100 binary packages.

Go to the [Error Knowledge Base](../error_knowledge_base.md) page to know more about Conan Center hook errors.
Some common errors related to Conan can be found on the [troubleshooting](https://docs.conan.io/en/latest/faq/troubleshooting.html) section.
> **Note**: This not a testing service, it is a binary building service for **released** packages. Unit tests shouldn't be built nor run in recipes by default, see the [FAQs](../faqs.md#why-conancenter-does-not-build-and-execute-tests-in-recipes) for more. Before submitting a pull request, please ensure that it works locally for some configurations.
### Linters
* The CI bot will start a new build only [after the author is approved](#one-request-access). Your PR may be reviewed in the mean time, but is not guaranteed.
* The CI system will also report errors and build logs by creating a comment in the pull-request, the message will include links to the logs for inspecting.
* The Actions are used to lint and ensure the latest conventions are being used. You'll see comments from bots letting you know.

Linters are always executed by Github actions to validate parts of your recipe, for instance, if it uses migrated Conan tools imports.
All executed linters are documented in [linters.md](../linters.md).
Check the [Developing Recipes](../developing_recipes_locally.md#running-the-python-linters) page for running them locally.
Packages generated and uploaded by this build service do not include any _user_ or _channel_ (we generally recommend using `@user/channel` for private package
repositories as an easy way to distinguish them from public ones). Once the packages are uploaded, you will be able to install them using the reference as
`name/version` so example `conan install fmt/9.1.0@` for 1.x client or `conan install --requires=fmt/9.1.0` for 2.x clients.
Loading

0 comments on commit 7129c01

Please sign in to comment.