Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 47 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, ubuntu-latest]
os: [ubuntu-latest]
env:
DO_DOCKER: 0
steps:
Expand All @@ -28,9 +28,25 @@ jobs:
with:
python-version: '3.12'
architecture: 'x64'
- run: cargo test --features rp
- run: cargo test --features jem
- run: bash ./ci/script.sh
- name: Install ARM cross-compiler and C libraries
run: |
sudo apt-get update
sudo apt-get install -y gcc-arm-linux-gnueabihf libc6-dev-armhf-cross
# If icasadi_rosenbrock or other deps need C++:
# sudo apt-get install -y g++-arm-linux-gnueabihf
- name: Cargo tests (RP and JEM)
run: |
cargo test --features rp
cargo test --features jem
- name: Run tests (script.sh)
# Set environment variables for the cc crate
env:
CC_arm_unknown_linux_gnueabihf: arm-linux-gnueabihf-gcc
AR_arm_unknown_linux_gnueabihf: arm-linux-gnueabihf-ar
# If C++ is involved and you installed g++-arm-linux-gnueabihf:
# CXX_arm_unknown_linux_gnueabihf: arm-linux-gnueabihf-g++
run: |
bash ./ci/script.sh

ci_macos:
runs-on: ${{ matrix.os }}
Expand All @@ -55,4 +71,30 @@ jobs:
python-version: '3.12'
- run: cargo test --features rp
- run: cargo test --features jem
- run: bash ./ci/script.sh
- name: Install ARM cross-compiler toolchain (via Homebrew)
run: |
# Tap the repository that provides the cross-compiler
brew tap messense/macos-cross-toolchains
# Update brew to ensure the tap is recognized (can sometimes be needed)
brew update
# Install the full toolchain (includes gcc, binutils, sysroot)
# This specific formula provides the entire toolchain.
brew install arm-unknown-linux-gnueabihf

# The above `brew install` might have linking conflicts if other partial
# toolchains were somehow pre-installed or installed by other steps.
# If it fails with link errors, you might need:
# brew link --overwrite arm-unknown-linux-gnueabihf

# Verify the compiler is found
which arm-linux-gnueabihf-gcc || (echo "arm-linux-gnueabihf-gcc not found in PATH" && exit 1)
- name: Run tests (script.sh)
# Set environment variables for the cc crate and PyO3 (if needed)
env:
CC_arm_unknown_linux_gnueabihf: arm-linux-gnueabihf-gcc
AR_arm_unknown_linux_gnueabihf: arm-linux-gnueabihf-ar
# If you are building PyO3 bindings and need to specify Python libs from the sysroot:
# PYO3_CROSS_LIB_DIR: "/opt/homebrew/opt/arm-unknown-linux-gnueabihf/arm-unknown-linux-gnueabihf/sysroot/usr/lib" # Adjust path and Python version
# PYO3_CROSS_INCLUDE_DIR: "/opt/homebrew/opt/arm-unknown-linux-gnueabihf/arm-unknown-linux-gnueabihf/sysroot/usr/include/python3.x" # Adjust path and Python version
run: |
bash ./ci/script.sh
2 changes: 1 addition & 1 deletion .github/workflows/dox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
deploy:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
Expand Down
5 changes: 5 additions & 0 deletions ci/script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,15 @@ regular_test() {
# --- install opengen
pip install .

# --- rust dependencies
rustup update
rustup target add arm-unknown-linux-gnueabihf

# --- run the tests
export PYTHONPATH=.
python -W ignore test/test_constraints.py -v
python -W ignore test/test.py -v
python -W ignore test/test_raspberry_pi.py -v


# Run Clippy for generated optimizers
Expand Down
83 changes: 81 additions & 2 deletions docs/python-advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ A complete list of solver options is given in the following table

## Build options

### Build mode

During the design phase, one needs to experiment with the problem
formulation and solver parameters. This is way the default build
mode is the "debug" mode, which compiles fast, but it suboptimal.
Expand All @@ -99,8 +101,10 @@ build_config.with_build_mode(
og.config.BuildConfiguration.RELEASE_MODE)
```

### Cross-compilation

You can either compile for your own system, or cross-compile for a
different target system. For example, to cross-compile for a Raspberry Pi,
different target system. For example, to cross-compile for a **Raspberry Pi**,
set the following option

```python
Expand All @@ -113,8 +117,83 @@ or
build_config.with_target_system("rpi") # Raspberry Pi
```

Note that you need to install the necessary target first.
Note that you need to install the necessary target first.

<details>
<summary><b>See setup details</b></summary>
To cross-compile for a Raspberry Pi you need to run the following in your terminal

```bash
rustup target add arm-unknown-linux-gnueabihf
```

You also need to install the following dependencies

<!--DOCUSAURUS_CODE_TABS-->

<!--Linux-->
```bash
sudo apt-get update
sudo apt-get install -y gcc-arm-linux-gnueabihf libc6-dev-armhf-cross
```

<!--MacOS-->
```bash
# Tap the repository that provides the cross-compiler
brew tap messense/macos-cross-toolchains
# Update brew to ensure the tap is recognized (can sometimes be needed)
brew update
# Install the full toolchain (includes gcc, binutils, sysroot)
# This specific formula provides the entire toolchain.
brew install arm-unknown-linux-gnueabihf

# Verify the compiler is found
which arm-linux-gnueabihf-gcc || (echo "arm-linux-gnueabihf-gcc not found in PATH" && exit 1)
```
<!--END_DOCUSAURUS_CODE_TABS-->
</details>

<br>
If you need to compile for a target other than `arm-linux-gnueabihf-gcc` (`rpi`)
some manual configuration may be needed (you may need to install the target
and/or a compiler/linker) and you may need to edit the auto-generated
`.cargo/config.toml` files you will find in your auto-generated solvers.

<details>
<summary><b>Non-supported targets</b></summary>
The auto-generated `.cargo/config.toml` files contain entries like

```toml
[target.arm-unknown-linux-gnueabihf]
linker="arm-linux-gnueabihf-gcc"
```

Here you may have to insert manually your own target.
Feel free to open an [issue](https://github.com/alphaville/optimization-engine/issues)
on GitHub if you would like us to add support for a particular target (create a feature
request); see the [contributing guidelines](https://alphaville.github.io/optimization-engine/docs/contributing).
</details>


When cross-compiling for a Raspberry Pi you may want to configure a TCP server
so you can call the optimizer remotely. You can find more information about this
[below](#tcpip-interface).
Once you have cross-compiled, locate the file
```text
{your_optimizer}/tcp_iface_{your_optimizer}/target/arm-unknown-linux-gnueabihf/release/tcp_iface_{your_optimizer}
```
—where `{your_optimizer}` is the name of your optimizer—and copy it to your Raspberry Pi.
On your Raspberry, change the permissions so you can execute this file
```bash
chmod u+x ./tcp_iface_{your_optimizer}
```
and [run it](https://alphaville.github.io/optimization-engine/docs/python-tcp-ip). Your OpEn server is live.
Read also the [documentation](https://alphaville.github.io/optimization-engine/docs/python-tcp-ip)
on the TCP sockets protocol of OpEn servers.

### Other build options

All build options are shown below

| Method | Explanation |
|-------------------------------|---------------------------------------------|
Expand Down
14 changes: 14 additions & 0 deletions open-codegen/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

Note: This is the Changelog file of `opengen` - the Python interface of OpEn

## [0.9.4] - 2025-05-08


### Fixed

- Fixed issues with cross compilation (each sub-project has its own `.cargo/config.toml`) and updated [documentation](https://alphaville.github.io/optimization-engine/docs/python-advanced#cross-compilation)

### Changed

- Rename auto-generated bindings file from `.cargo/config` to `.cargo/config.toml` (backwards compatible change)
- Updated min cmake version from 2.8 to 3.5
- Updated auto-generated example C/C++ bindings


## [0.9.3] - 2024-12-06

Expand Down Expand Up @@ -226,6 +239,7 @@ Note: This is the Changelog file of `opengen` - the Python interface of OpEn
* Fixed `lbfgs` typo


[0.9.4]: https://github.com/alphaville/optimization-engine/compare/opengen-0.9.3...opengen-0.9.4
[0.9.3]: https://github.com/alphaville/optimization-engine/compare/opengen-0.9.2...opengen-0.9.3
[0.9.2]: https://github.com/alphaville/optimization-engine/compare/opengen-0.9.1...opengen-0.9.2
[0.9.1]: https://github.com/alphaville/optimization-engine/compare/opengen-0.9.0...opengen-0.9.1
Expand Down
2 changes: 1 addition & 1 deletion open-codegen/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.9.3
0.9.4
41 changes: 31 additions & 10 deletions open-codegen/opengen/builder/optimizer_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,6 @@ def __prepare_target_project(self):
"""Creates folder structure

Creates necessary folders
Runs `cargo init` in that folder

"""
self.__logger.info("Creating necessary folders")

Expand All @@ -173,6 +171,13 @@ def __prepare_target_project(self):
os.makedirs(target_dir)
else:
make_dir_if_not_exists(target_dir)

# make folder {root}/.cargo
dot_cargo_dir = os.path.join(target_dir, ".cargo")
make_dir_if_not_exists(dot_cargo_dir)
# copy cargo_config.toml into .cargo/config.toml
cargo_config_file = os.path.join(og_dfn.templates_dir(), 'cargo_config.toml')
shutil.copy(cargo_config_file, os.path.join(dot_cargo_dir, 'config.toml'))

def __copy_icasadi_to_target(self):
"""
Expand All @@ -181,13 +186,21 @@ def __copy_icasadi_to_target(self):
self.__logger.info("Copying icasadi interface to target directory")
origin_icasadi_dir = og_dfn.original_icasadi_dir()
target_icasadi_dir = self.__icasadi_target_dir()
if not os.path.exists(target_icasadi_dir):
os.makedirs(target_icasadi_dir)
shutil.rmtree(target_icasadi_dir)
target_icasadi_cargo_config_dir = os.path.join(target_icasadi_dir, ".cargo")

if os.path.exists(target_icasadi_dir):
shutil.rmtree(target_icasadi_dir)

shutil.copytree(origin_icasadi_dir,
target_icasadi_dir,
ignore=shutil.ignore_patterns(
'*.lock', 'ci*', 'target', 'auto*'))

# Copy cargo_config.toml into .cargo/config.toml
make_dir_if_not_exists(target_icasadi_cargo_config_dir)
cargo_config_file = os.path.join(og_dfn.templates_dir(), 'cargo_config.toml')
shutil.copy(cargo_config_file, os.path.join(
target_icasadi_cargo_config_dir, 'config.toml'))

def __generate_icasadi_cargo_toml(self):
"""
Expand Down Expand Up @@ -571,7 +584,7 @@ def __generate_build_rs(self):
def __build_optimizer(self):
target_dir = os.path.abspath(self.__target_dir())
command = self.__make_build_command()
p = subprocess.Popen(command, cwd=target_dir)
p = subprocess.Popen(command, cwd=target_dir, shell=False)
process_completion = p.wait()
if process_completion != 0:
raise Exception('Rust build failed')
Expand Down Expand Up @@ -676,13 +689,13 @@ def __generate_code_python_bindings(self):
with open(target_python_rs_path, "w") as fh:
fh.write(python_rs_output_template)

# move cargo_config into .cargo/config
# copy cargo_config into .cargo/config
target_cargo_config_dir = os.path.join(python_bindings_dir, '.cargo')
make_dir_if_not_exists(target_cargo_config_dir)
cargo_config_file = os.path.join(
og_dfn.templates_dir(), 'python', 'cargo_config')
og_dfn.templates_dir(), 'cargo_config.toml')
shutil.copy(cargo_config_file, os.path.join(
target_cargo_config_dir, 'config'))
target_cargo_config_dir, 'config.toml'))

def __generate_code_tcp_interface(self):
self.__logger.info(
Expand All @@ -694,10 +707,12 @@ def __generate_code_tcp_interface(self):
tcp_iface_dir_name = _TCP_IFACE_PREFIX + self.__meta.optimizer_name
tcp_iface_dir = os.path.join(target_dir, tcp_iface_dir_name)
tcp_iface_source_dir = os.path.join(tcp_iface_dir, "src")
tcp_iface_cargo_config_dir = os.path.join(tcp_iface_dir, ".cargo")

# make tcp_iface/ and tcp_iface/src
make_dir_if_not_exists(tcp_iface_dir)
make_dir_if_not_exists(tcp_iface_source_dir)
make_dir_if_not_exists(tcp_iface_cargo_config_dir)

# generate tcp_server.rs for tcp_iface
tcp_rs_template = OpEnOptimizerBuilder.__get_template(
Expand All @@ -718,6 +733,12 @@ def __generate_code_tcp_interface(self):
target_tcp_rs_path = os.path.join(tcp_iface_dir, "Cargo.toml")
with open(target_tcp_rs_path, "w") as fh:
fh.write(tcp_rs_output_template)

# Copy cargo_config.toml into .cargo/config.toml
cargo_config_file = os.path.join(
og_dfn.templates_dir(), 'cargo_config.toml')
shutil.copy(cargo_config_file, os.path.join(
tcp_iface_cargo_config_dir, 'config.toml'))

def __generate_yaml_data_file(self):
self.__logger.info("Generating YAML configuration file")
Expand Down Expand Up @@ -850,7 +871,7 @@ def build(self):
"""
self.__initialize() # initialize default value (if not provided)
self.__check_user_provided_parameters() # check the provided parameters
self.__prepare_target_project() # create folders; init cargo project
self.__prepare_target_project() # create folders
self.__copy_icasadi_to_target() # copy icasadi/ files to target dir
self.__generate_icasadi_cargo_toml() # generate icasadi's Cargo.toml file
self.__generate_cargo_toml() # generate Cargo.toml using template
Expand Down
2 changes: 1 addition & 1 deletion open-codegen/opengen/templates/c/example_cmakelists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 2.8)
cmake_minimum_required(VERSION 3.5)

# Project name
project({{meta.optimizer_name}})
Expand Down
13 changes: 11 additions & 2 deletions open-codegen/opengen/templates/c/example_optimizer_c_bindings.c
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,20 @@
*
* Compile with:
*
* gcc -Wall -std=c99 -pedantic \
* $ gcc -Wall -std=c99 -pedantic \
example_optimizer.c -l:lib{{meta.optimizer_name}}.a \
-L./target/{{build_config.build_mode}} -pthread -lm -ldl \
-o optimizer
*
* OR ...
*
* $ gcc -Wall -std=c99 -pedantic \
example_optimizer.c -l{{meta.optimizer_name}} \
-L./target/{{build_config.build_mode}} -pthread -lm -ldl \
-o optimizer
*
* Or simply do:
* cmake .; make run
*/

#include <stdio.h>
Expand All @@ -17,7 +26,7 @@
* Feel free to customize the following code...
*/

int main() {
int main(void) {
int i;

/* parameters */
Expand Down
Loading
Loading