Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues consolidations UX #28

Merged
merged 73 commits into from
Feb 21, 2025
Merged
Show file tree
Hide file tree
Changes from 65 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
5cbb676
Update default image name
mikusaq Oct 23, 2024
e68f4b0
Update default docker image name
mikusaq Oct 23, 2024
d7be974
Refactor
mikusaq Oct 30, 2024
1710d4f
Change commiting to Git Lfs
mikusaq Oct 31, 2024
5f907de
Refactor ContextManager
mikusaq Nov 1, 2024
f0588f9
Change warnings color to orange
mikusaq Nov 1, 2024
0c263a6
Add check for discrepancies between Json definitions and Git Lfs
mikusaq Nov 1, 2024
85a01e1
Change order of checks
mikusaq Nov 1, 2024
cd3aae5
Checks moved to function
mikusaq Nov 1, 2024
1ff9fb8
Add info logging for prebuild checks
mikusaq Nov 7, 2024
14ecb6f
Remove indent when logging some warnings
mikusaq Nov 7, 2024
ac7e395
Remove calling Fatal
mikusaq Nov 7, 2024
fb50fe3
Add also cleaning all changes
mikusaq Nov 7, 2024
58930c0
Add SIGINT handling for Git Lfs, refactor
mikusaq Nov 7, 2024
2729470
Add create-sysroot command
mikusaq Nov 12, 2024
c3293dc
Refactor
mikusaq Nov 13, 2024
096518e
Refactor, add check if platformString is nil
mikusaq Nov 13, 2024
aed0b7c
Better error prints
mikusaq Nov 20, 2024
c2b6b8b
Add --build-deps-on flag
mikusaq Nov 25, 2024
fa50883
Remove redundant nil check
mikusaq Nov 25, 2024
42beb6e
Add built packages json writing to sysroot
mikusaq Nov 27, 2024
683cd3e
Add updating built packages to sysroot initialization
mikusaq Nov 30, 2024
a9941c5
Refactor, add doc
mikusaq Nov 30, 2024
541602e
Make --image-name flag required
mikusaq Dec 4, 2024
31a7242
Merge branch 'master' into BAF-976/issues-consolidation-ux
mikusaq Dec 16, 2024
dcd8978
Git lfs consistency check look only in related directories
mikusaq Dec 20, 2024
5dba046
Add check for non-existent expected path
mikusaq Dec 20, 2024
4384611
Fix create-sysroot command behaviour when not empty git status in lfs…
mikusaq Jan 2, 2025
6a7c540
Add check for wrong image name
mikusaq Jan 2, 2025
787d22f
Add check for empty image-name
mikusaq Jan 2, 2025
55be4f4
Fix using default Docker struct
mikusaq Jan 3, 2025
eadf66e
Update debian12 docker file to be more consistent
koudis Jan 4, 2025
4bc367a
Make context cli flag required, help doc improvements
mikusaq Jan 4, 2025
b0b26af
Fix missing error of specific flag combination
mikusaq Jan 4, 2025
94304a4
Add --build-deps-on-recursive flag
mikusaq Jan 4, 2025
fb6ec46
Fix wrong help doc of build-deps-on flags
mikusaq Jan 4, 2025
e3e0c4d
Fix doc
mikusaq Jan 6, 2025
07b5c35
Split git lfs check warning
mikusaq Jan 6, 2025
697633c
Remove invalid wait from error log
mikusaq Jan 6, 2025
9b79868
Fix issues in create-sysroot command
mikusaq Jan 8, 2025
0204d9a
Fix logger call warnings
mikusaq Jan 8, 2025
a19e49e
Improve error handling
mikusaq Jan 8, 2025
158f640
Remove creating Git Lfs directory when create-sysroot
mikusaq Jan 8, 2025
71e0a83
Log improvements
mikusaq Jan 8, 2025
88fc624
Improve error handling
mikusaq Jan 8, 2025
ea5d4e0
Improve error handling
mikusaq Jan 8, 2025
b0f8686
Add check for empty Git repo
mikusaq Jan 8, 2025
0f6f3e9
Fedora 41 added
koudis Jan 13, 2025
d588cb3
Add unit tests
mikusaq Jan 13, 2025
9845e1d
Update README, add Use Case Scenarios
mikusaq Jan 13, 2025
32df37d
Fix lsb_release instability
koudis Jan 13, 2025
58d45f4
No need to retry, lsb_release fixed
koudis Jan 13, 2025
ce969e8
Remove unused constants
koudis Jan 13, 2025
a872449
Merge pull request #31 from bringauto/BAF-1032/lsb_release_bugfix
koudis Jan 13, 2025
23513fc
Fix test data
mikusaq Jan 15, 2025
321cec5
Refactor, add check for missing same build type package
mikusaq Jan 15, 2025
07edef5
Fix some tests, add error handling
mikusaq Jan 15, 2025
6544952
Fix some issues and log improvements in tests
mikusaq Jan 22, 2025
7eabfa4
Add and adjust tests, test refactor
mikusaq Jan 29, 2025
ec3d2d8
Fix infinite loop for circular dependencies for build-deps-on option
mikusaq Jan 29, 2025
5604a74
Add error handling when no package is built when using build-deps-on …
mikusaq Jan 29, 2025
4f12072
Update documentation
mikusaq Jan 31, 2025
f65cacb
Doc fixes
mikusaq Feb 2, 2025
bcb27fb
Fix IsLibrary and IsDevLib flags to not affect the Package name
mikusaq Feb 3, 2025
ac2330a
Remove support for any_machine mode
mikusaq Feb 3, 2025
f0eafad
Improve, add and fix docs
mikusaq Feb 4, 2025
ca11297
Improve doc
mikusaq Feb 5, 2025
1722890
Fix and improve docs
mikusaq Feb 12, 2025
baae508
Refactor tests
mikusaq Feb 12, 2025
09aa3ff
Change default image name
mikusaq Feb 19, 2025
cb380a2
Small changes, doc improvements
mikusaq Feb 19, 2025
6e08d94
Refactor, doc improvements
mikusaq Feb 19, 2025
0ed3759
Add error handling
mikusaq Feb 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,17 @@ Additional requirements for `build.sh`:

## Usage

The `bap-builder` builds and stores all dependencies in the git repository (LFS enabled is recommended).
The `packager` (`bap-builder`) has these commands:
- `build-image` for building Docker images
- `build-package` for building Packages
- `create-sysroot` for creating sysroot from already built Packages

The `build-package` and `create-sysroot` commands are using Git Repository as storage for built
Packages. Given Git Repository must be created before usage.

**NOTE:** Detailed use case scenarios is decribed in [UseCaseScenarios](./doc/UseCaseScenarios.md) document.

### Example

1. Create a git repository (optionally with LFS):

Expand All @@ -49,19 +59,25 @@ The `bap-builder` builds and stores all dependencies in the git repository (LFS
cd ../
```

2. Build all Docker images needed for the build:
2. Build Docker image needed for the build:

```bash
bap-builder build-image --context ./example --name debian12
```

3. Build all Packages for the given distro:

```bash
bap-builder build-image --context ./example --name debian11
bap-builder build-package --context ./example --image-name debian12 --output-dir ./lfsrepo --all
```

3. Build all packages for the given distro:
4. Create sysroot for built Packages:

```bash
bap-builder build-package --context ./example --image-name debian11 --output-dir ./lfsrepo --all
bap-builder create-sysroot --context ./example --image-name debian12 --git-lfs ./lfsrepo --sysroot-dir ./new_sysroot
```

*Note: If you do not have `bap-builder` in your system path, you need to use `./bap-builder/bap-builder` instead of `bap-builder`.*
**Note:** If you do not have `bap-builder` in your system path, you need to use `./bap-builder/bap-builder` instead of `bap-builder`.

## Documentation

Expand Down
108 changes: 93 additions & 15 deletions bap-builder/CmdArgs.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ type BuildPackageCmdLineArgs struct {
Name *string
// BuildDeps Build all dependencies of package when building single package
BuildDeps *bool
// BuildDepsOn Build package with all packages which depends on it
BuildDepsOn *bool
// BuildDepsOn Build package with all packages which depends on it recursively
BuildDepsOnRecursive *bool
// DockerImageName is a name of docker image to which packages will be build.
// If empty all docker images from DockerMatrix in config file are used for a given package.
// If not empty, only packages which contains DockerImageName in DockerMatrix will be built.
Expand All @@ -40,6 +44,17 @@ type BuildPackageCmdLineArgs struct {
OutputDirMode *OutputDirMode
}

// CreateSysrootCmdLineArgs
// Options/setting for Sysroot mode
type CreateSysrootCmdLineArgs struct {
// Path to the Git Lfs repository with packages
Repo *string
// Name of the new sysroot directory to be created
Sysroot *string
// Name of the docker image which are the packages build for
ImageName *string
}

// CmdLineArgs
// Represents Cmd line arguments passed to cmd line of the target program.
// Program operates in two modes
Expand All @@ -54,11 +69,15 @@ type CmdLineArgs struct {
// Standard Cmd line arguments for Docker mode
BuildImagesArgs BuildImageCmdLineArgs
// If true the program is in the "Package" mode
BuildPackage bool
BuildPackageArgs BuildPackageCmdLineArgs
buildImageParser *argparse.Command
buildPackageParser *argparse.Command
parser *argparse.Parser
BuildPackage bool
// If true the program is in the "Sysroot" mode
CreateSysroot bool
BuildPackageArgs BuildPackageCmdLineArgs
CreateSysrootArgs CreateSysrootCmdLineArgs
buildImageParser *argparse.Command
buildPackageParser *argparse.Command
createSysrootParser *argparse.Command
parser *argparse.Parser
}

// InitFlags
Expand All @@ -68,9 +87,8 @@ func (cmd *CmdLineArgs) InitFlags() {
cmd.parser = argparse.NewParser("BringAuto Packager", "Build and track C++ dependencies")
cmd.Context = cmd.parser.String("", "context",
&argparse.Options{
Required: false,
Default: ".",
Help: "Command context",
Required: true,
Help: "Context directory where are the json definition of packages",
},
)

Expand All @@ -96,6 +114,22 @@ func (cmd *CmdLineArgs) InitFlags() {
Help: "Build all dependencies of package when building single package",
},
)
cmd.BuildPackageArgs.BuildDepsOn = cmd.buildPackageParser.Flag("", "build-deps-on",
&argparse.Options{
Required: false,
Default: false,
Help: "Build packages which depends on given package without itself, " +
"the packages are built with its dependencies",
},
)
cmd.BuildPackageArgs.BuildDepsOnRecursive = cmd.buildPackageParser.Flag("", "build-deps-on-recursive",
&argparse.Options{
Required: false,
Default: false,
Help: "Build packages which depends on given package without itself recursively, " +
"the packages are built with its dependencies",
},
)
cmd.BuildPackageArgs.OutputDir = cmd.buildPackageParser.String("", "output-dir",
&argparse.Options{
Required: true,
Expand All @@ -104,11 +138,11 @@ func (cmd *CmdLineArgs) InitFlags() {
)
cmd.BuildPackageArgs.DockerImageName = cmd.buildPackageParser.String("", "image-name",
&argparse.Options{
Required: false,
Default: "",
Help: "Docker image name for which packages will be build.\n" +
"Only packages that contains image-name in the DockerMatrix will be built.\n" +
"Given packages will be build by toolchain represented by image-name",
Required: true,
Validate: checkForEmpty,
Help: "Docker image name for which packages will be build. " +
"Only packages that contains image-name in the DockerMatrix will be built. " +
"Given packages will be build by toolchain represented by image-name",
},
)

Expand All @@ -126,6 +160,36 @@ func (cmd *CmdLineArgs) InitFlags() {
Help: "Name of the docker image to build",
},
)

cmd.createSysrootParser = cmd.parser.NewCommand("create-sysroot", "Create Sysroot")
cmd.CreateSysrootArgs.Sysroot = cmd.createSysrootParser.String("", "sysroot-dir",
&argparse.Options{
Required: true,
Help: "Name of the sysroot directory which will be created",
},
)
cmd.CreateSysrootArgs.Repo = cmd.createSysrootParser.String("", "git-lfs",
&argparse.Options{
Required: true,
Help: "Git Lfs directory where packages are stored",
},
)
cmd.CreateSysrootArgs.ImageName = cmd.createSysrootParser.String("", "image-name",
&argparse.Options{
Required: true,
Validate: checkForEmpty,
Help: "Name of docker image which are the packages built for",
},
)
}

func checkForEmpty(args []string) error {
if len(args) == 1 {
if len(args[0]) == 0 {
return fmt.Errorf("cannot be empty")
}
}
return nil
}

// ParseArgs
Expand All @@ -143,8 +207,22 @@ func (cmd *CmdLineArgs) ParseArgs(args []string) error {

cmd.BuildImage = cmd.buildImageParser.Happened()
cmd.BuildPackage = cmd.buildPackageParser.Happened()
if *cmd.BuildPackageArgs.All && *cmd.BuildPackageArgs.BuildDeps {
return fmt.Errorf("all and build-deps flags at the same time")
cmd.CreateSysroot = cmd.createSysrootParser.Happened()

if *cmd.BuildPackageArgs.All {
if *cmd.BuildPackageArgs.BuildDeps {
return fmt.Errorf("all and build-deps flags at the same time")
}
if *cmd.BuildPackageArgs.BuildDepsOn {
return fmt.Errorf("all and build-deps-on flags at the same time")
}
if *cmd.BuildPackageArgs.BuildDepsOnRecursive {
return fmt.Errorf("all and build-deps-on-recursive flags at the same time")
}
} else if *cmd.BuildPackageArgs.BuildDepsOn {
if *cmd.BuildPackageArgs.BuildDepsOnRecursive {
return fmt.Errorf("build-deps-on and build-deps-on-recursive flags at the same time")
}
}

return nil
Expand Down
6 changes: 0 additions & 6 deletions bap-builder/Constant.go

This file was deleted.

5 changes: 3 additions & 2 deletions bap-builder/DockerMode.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,15 @@ package main
import (
"bringauto/modules/bringauto_log"
"bringauto/modules/bringauto_docker"
"bringauto/modules/bringauto_context"
"path"
)

// BuildDockerImage
// process Docker mode of cmd line
//
func BuildDockerImage(cmdLine *BuildImageCmdLineArgs, contextPath string) error {
contextManager := ContextManager{
contextManager := bringauto_context.ContextManager{
ContextPath: contextPath,
}
buildAll := cmdLine.All
Expand All @@ -29,7 +30,7 @@ func BuildDockerImage(cmdLine *BuildImageCmdLineArgs, contextPath string) error
// builds all docker images in the given contextPath.
// It returns nil if everything is ok, or not nil in case of error
//
func buildAllDockerImages(contextManager ContextManager) error {
func buildAllDockerImages(contextManager bringauto_context.ContextManager) error {
dockerfilePathList, err := contextManager.GetAllImagesDockerfilePaths()
if err != nil {
return err
Expand Down
Loading