Skip to content

[Nexthop] Update Distro Image documentation #878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
travisb-nexthop wants to merge 1 commit into
base: main
Choose a base branch
from
+124 −130
Open

[Nexthop] Update Distro Image documentation #878

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 118 additions & 94 deletions fboss-image/distro_cli/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,46 +26,13 @@ Requires Python 3.10+. No external dependencies.

The manifest is a JSON file that defines what components to build, where to get them, and how to compose the final FBOSS distribution image.

### What Goes Into the Manifest

The manifest describes your FBOSS image composition through these sections:

**1. Output Formats** (`distribution_formats` - required)

Specifies what image formats to produce. Supported formats:
- `onie`: ONIE installer binary for network switches
- `usb`: Bootable ISO image for USB installation

**2. Kernel** (`kernel` - required)

Where to get the Linux kernel. Specify a `download` URL pointing to the kernel tarball.

**3. Dependencies** (`other_dependencies` - optional)

Additional packages needed for your image (RPMs, tools, etc.). Each dependency specifies:
- `download`: URL (http/https) or local file path (`file:path/to/package.rpm`)

**4. FBOSS Components** (all optional)

The FBOSS software stack components to build:
- `fboss-platform-stack`: Platform services and hardware abstraction
- `bsps`: Board Support Packages for specific hardware platforms (array)
- `sai`: Switch Abstraction Interface implementation
- `fboss-forwarding-stack`: Core switching/routing logic

Each component specifies:
- `execute`: Command to build it (string or array of command + args)

**5. Build Hooks** (`image_build_hooks` - optional)

Customization points in the build process:
- `after_pkgs`: Additional package configuration for extra OS packages

### Example
For example:

```json
{
"distribution_formats": {
"pxe": "FBOSS-k6.4.3.tar",
"onie": "FBOSS-k6.4.3.bin",
"usb": "FBOSS-k6.4.3.iso"
},
Expand All @@ -77,7 +44,7 @@ Customization points in the build process:
{"download": "file:vendor_debug_tools/tools.rpm"}
],
"fboss-platform-stack": {
"execute": ["fboss/fboss/oss/scripts/build.py", "platformstack"]
"execute": ["../fboss/oss/scripts/build_fboss_stack.sh", "platform"]
},
"bsps": [
{"execute": "vendor_bsp/build.make"},
Expand All @@ -87,87 +54,144 @@ Customization points in the build process:
"execute": "fboss_brcm_sai/build.sh"
},
"fboss-forwarding-stack": {
"execute": ["fboss/fboss/oss/scripts/build.py", "forwardingstack"]
"execute": ["../fboss/oss/scripts/build_fboss_stack.sh", "forwarding"]
},
"image_build_hooks": {
"after_pkgs": "additional_os_pkgs.xml"
"after_pkgs_install": "additional_os_pkgs.json",
"after_pkgs_execute": "additional_image_mods.json"
}
}
```

## Build Order
### Distribution Formats

Components are built in the following order:
1. `kernel`
2. `other_dependencies`
3. `fboss-platform-stack`
4. `bsps`
5. `sai`
6. `fboss-forwarding-stack`
7. Image composition with `distribution_formats`
`distribution_formats` specifies which output image formats to produce. Supported formats:
- `onie`: ONIE installer binary for network switches. Normal extension `.bin`
- `usb`: Bootable ISO image for USB installation. Normal extension `.iso`
- `pxe`: Tarball for network installation (PXE). Normal extension `.tar`

## Development
### Image Build Hooks

### Running Tests
`image_build_hooks` specifies customization points in the build process. Supported hooks:
- `after_pkgs_install`: JSON file with list of additional packages to install
- `after_pkgs_execute`: JSON file with list of commands to execute after packages are installed

```bash
# Run all tests
cd fboss-image/distro_cli
python3 -m unittest discover -s tests -p '*_test.py'
To install additional files or RPM packages not in the CentOS RPM repos, use the "other_dependencies" component.

# Run specific test
python3 -m unittest tests.cli_test
`after_pkgs_install` hook file example:
```json
{
"packages": [
"tcpdump"
]
}
```

### Linting

```bash
cd fboss-image/distro_cli
python3 -m ruff check .
`after_pkgs_execute` hook file example:
```json
{
"execute": [
[ "date"],
[ "uname", "-a"]
]
}
```

### Project Structure
### Components

Each remaining top-level key in the manifest represents a component or component list to build. Components can be one of
two types:
- `download`: URL or local file path to download the component from. The component will be extracted from the downloaded
archive. If the component artifact type is a tarball, then it can be compressed or uncompressed. `file:` URLs are
relative to the manifest file. `file:/` URLs are absolute.
- `execute`: Script to execute to build the component. The command is executed in the directory of the script file. Two
forms are supported: a string script path to execute, or an array of the script path followed by arguments to pass to
the script.

Component build scripts are responsible for caching and incremental build support. All build scripts are executed inside
a fresh build container with the component dependencies either installed or available in `/deps` as appropriate. The
output artifact of the build script must be placed into /output to be copied outside the build container.

For components that are lists (e.g. bsps), each list element is built independently and the artifacts are collected
together.

### Kernel output artifact

The kernel build script must output an uncompressed tarball containing the kernel RPMs along the standard Redhat
structure. See the FBOSS-OSS kernel under `fboss-image/kernel` for an example.

### Other dependencies output artifact

Each other_dependency must produce a single RPM which is installed in the final image. These RPMs may have runtime
dependencies which will be satisfied by dnf. See `examples/thirdparty_build.sh` for an example of how to build a
third-party RPM.

During each of these builds, the kernel devel package will be installed in the build container.

### FBOSS Platform and Forwarding Stack

See `fboss/oss/scripts/build_fboss_stack.sh` for details on how to build the FBOSS stacks. The artifact format is a
tarball of files which will be extracted under `/opt/fboss` in the final image.

Each of these two stacks are built separately. The Platform Stack has only the kernel installed, while the Forwarding
Stack also has the SAI implementation installed.

### BSPs

Each BSP artifact is a tarball of RPMs. The RPM files will be copied (not installed) into the image in the local RPM
repository at `/usr/local/share/local_rpm_repo`. platform_manager will install the BSP RPMs from this repository at
runtime.

During each of these builds, the kernel devel package will be installed in the build container.

### SAI

The SAI build script must output a tarball containing the following file structure:
```
sai_build.env -- FBOSS SAI build environment variables
include/ -- SAI headers
lib/ -- SAI library (libsai_impl.a)
sai-runtime.rpm -- SAI runtime package
```

The `sai_build.env` file defines the environment variables needed to build the FBOSS platform and forwarding stacks
against the SAI implementation. See [Building FBOSS on docker
containers](https://facebook.github.io/fboss/docs/build/building_fboss_on_docker_containers/#set-important-environment-variables).
The file might look like:
```
fboss-image/distro_cli/
├── fboss-image # Main CLI entry point
├── lib/ # Core libraries
│ ├── cli.py # CLI framework (argparse abstraction)
│ ├── manifest.py # Manifest parsing and validation
│ ├── builder.py # Image builder
│ └── logger.py # Logging setup
├── cmds/ # Command implementations
│ ├── build.py # Build command
│ └── device.py # Device commands
└── tests/ # Unit tests
├── cli_test.py # CLI framework tests
├── manifest_test.py
├── image_builder_test.py
├── build_test.py
└── device_test.py
SAI_BRCM_IMPL=1
SAI_SDK_VERSION=SAI_VERSION_13_3_0_0_ODP
SAI_VERSION=1.16.1
```

### CLI Framework
The `sai-runtime.rpm` package contains the SAI runtime dependencies, normally kernel modules and init scripts. It is
installed into the image at build-time.

The SAI is built with the kernel devel packages installed.

A special `fboss/oss/scripts/fake-sai-devel.tar.zstd` is provided which will activate the Fake-SAI build of the
Forwarding Stack.

The CLI uses a custom OOP wrapper around argparse (stdlib only, no external dependencies):
# Environment variables

```python
from lib.cli import CLI
The environment `FBOSS_BUILDER_CACHE_EXPIRATION_HOURS` controls how long to cache the fboss_builder Docker image,
irrespective of whether the hashed contents have changed. The default is 24 hours.

# Create CLI
cli = CLI(description='My CLI')
## Development

### Running Tests

# Add simple command
cli.add_command('build', build_func,
help_text='Build something',
arguments=[('file', {'help': 'Input file'})])
```bash
# Build and run all tests
cmake --build . --target distro_cli_tests

# Run via CTest
ctest -R distro_cli -V

# Add command group with subcommands
device = cli.add_command_group('device',
help_text='Device commands',
arguments=[('mac', {'help': 'MAC address'})])
device.add_command('ssh', ssh_func, help_text='SSH to device')
### Running manual tests
A few tests are deliberately skipped while unit testing due to longer execution times. They can however be run manually as needed. The following are some of the examples.

# Run
cli.run(setup_logging_func=setup_logging)
# Testing kernel build with compressed artifacts
```bash
python -m pytest distro_cli/tests/kernel_build_test.py::TestKernelBuildE2E::test_real_kernel_build_compressed -v -s -m e2e
```
42 changes: 6 additions & 36 deletions fboss-image/image_builder/README
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,41 +1,11 @@
This directory contains files necessary to generate bootable images for FBOSS using the kiwi ng tool.

KIWI NG Documentation:
https://osinside.github.io/kiwi/overview.html

KIWI NG XML Schema:
https://github.com/OSInside/kiwi.git

Steps to validate schema changes:
1) Clone the above repository
2) xmllint --relaxng kiwi/kiwi/schema/kiwi.rng <templates/centos-09.0/config.xml>

The directories here are:

bin - executable scripts that do the real work
lib - helper code that is common across scripts
logs - all logs go here
templates - kiwi-ng image templates
output - all build output goes here
`image_builder/templates` contains the template(s) that contain the information needed to build the image.

image_builder/bin:
build_image.sh - Build docker (if needed) and launch build_image_in_container.sh
build_image_in_container.sh - Script that builds the image inside docker container
`centos-09.0` contains the Kiwi-NG template files for building a CentOS-9.0 based FBOSS image. Of particular note
is the `root_files` directory which contains the files that will be copied into the root filesystem of the image.

image_builder/templates:
Contains the template(s) that contain the information needed by kiwi-ng-3
to generate
centos-09.0 - CentOS-9.0 based FBOSS image template files
`onie` contains the template files for building the ONIE installer with the FBOSS image.

image_builder/output:
All generated output goes into this directory

To build a bootable image the steps are as follows (all scripts support -h or --help option)

Build and launch the docker:
git clone https://github.com/facebook/fboss <wsdir>
<wsdir>/fboss-image/image_builder/bin/build_image.sh -b # Launch image build inside docker

This will generate the many files in /image_builder/output, the main being:
FBOSS-Distro-Image.x86_64-1.0.install.tar
FBOSS-Distro-Image.x86_64-1.0.install.iso
KIWI NG Documentation:
https://osinside.github.io/kiwi/overview.html
Loading