Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

odgi: add linux-aarch64 build #48682

Closed
wants to merge 4 commits into from
Closed

Conversation

martin-g
Copy link
Contributor

Describe your pull request here


Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

  • If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
  • New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
  • PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
  • Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes.
This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe.
If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation).
Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case run_exports statement
semantic versioning {{ pin_subpackage("myrecipe", max_pin="x") }}
semantic versioning (0.x.x) {{ pin_subpackage("myrecipe", max_pin="x.x") }}
known breakage in minor versions {{ pin_subpackage("myrecipe", max_pin="x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions {{ pin_subpackage("myrecipe", max_pin="x.x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning {{ pin_subpackage("myrecipe", max_pin=None) }}

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

@BiocondaBot please update Merge the master branch into a PR.
@BiocondaBot please add label Add the please review & merge label.
@BiocondaBot please fetch artifacts Post links to CI-built packages/containers.
You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

@martin-g martin-g added the aarch64 Related to adding linux-aarch64 support label Jun 24, 2024
Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
@martin-g
Copy link
Contributor Author

The Linux ARM64 build passes with warnings:

/tmp/odgi/odgi/src/dset64.hpp:62:2: warning: #warning "odgi should only be built on an x86_64 machine (64-bit Intel/AMD)" [-Wcpp]
   62 | #warning "odgi should only be built on an x86_64 machine (64-bit Intel/AMD)"
      |  ^~~~~~~

but the tests fail:

�[0m
09:02:01 �[32mBIOCONDA INFO�[0m (OUT) /opt/conda/conda-bld/odgi_1719219134420/work/src/unittest/handle.cpp:2040: FAILED:�[0m
09:02:01 �[32mBIOCONDA INFO�[0m (OUT)   {Unknown expression after the reported line}�[0m
09:02:01 �[32mBIOCONDA INFO�[0m (OUT) due to a fatal error condition:�[0m
09:02:01 �[32mBIOCONDA INFO�[0m (OUT)   SIGSEGV - Segmentation violation signal�[0m
09:02:01 �[32mBIOCONDA INFO�[0m (OUT) �[0m

@martin-g martin-g closed this Jun 24, 2024
@martin-g martin-g reopened this Oct 16, 2024
Copy link
Contributor

coderabbitai bot commented Oct 16, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request involve modifications to the build.sh, meta.yaml, and the addition of a patch file for the odgi package. In build.sh, architecture-specific compilation options are introduced, utilizing uname -m to determine the machine's architecture and setting the ARCH_OPTS variable accordingly. This variable is incorporated into the cmake command, replacing hardcoded values. The script also enables debugging and error handling by adding set -xe.

The meta.yaml file is updated to include a new patch entry for the aarch64 architecture, enhancing compatibility with this platform. The extra section now specifies linux-aarch64 as a supported platform. Additionally, a new patch file, odgi-linux-aarch64.patch, modifies the CMakeLists.txt and src/main.cpp files. The patch simplifies optimization flags by removing the -msse4.2 flag and comments out a specific architecture constraint in the main function declaration. Overall, these changes aim to improve the build configuration and compatibility of the odgi package with various architectures.

Possibly related PRs

Suggested labels

please review & merge, osx-arm64

Suggested reviewers

  • AndreaGuarracino

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between e022601 and daef4c4.

📒 Files selected for processing (1)
  • recipes/odgi/meta.yaml (2 hunks)
🧰 Additional context used
🔇 Additional comments (4)
recipes/odgi/meta.yaml (4)

16-16: LGTM. Build number increment is appropriate.

Incrementing the build number from 0 to 1 is correct. This change is necessary when modifying a package without changing its version, which is the case here with the addition of aarch64 support.


Line range hint 1-67: Address reported warnings and test failures

The changes made in this file are necessary steps towards adding linux-aarch64 build support for odgi. However, there are some concerns that need to be addressed:

  1. A warning was reported indicating that "odgi should only be built on an x86_64 machine (64-bit Intel/AMD)". This suggests potential compatibility issues with ARM64 architecture.
  2. Test failures were reported, specifically a segmentation fault in the unit test located in handle.cpp at line 2040.

These issues are not addressed by the current changes in the recipe.

Please investigate and address the following:

  1. The source of the warning about x86_64 build requirement. Is there a way to safely build odgi on ARM64?
  2. The cause of the test failure and segmentation fault. Are there any architecture-specific assumptions in the code that need to be adjusted?

Consider running the following commands to gather more information:

Based on the results, you may need to make additional changes to the source code or build process to ensure compatibility with ARM64 architecture.


53-54: LGTM. Verify complete aarch64 support.

The addition of linux-aarch64 as an additional platform is correct and aligns with the PR objective. This change in the extra section is the proper way to specify additional build platforms in conda-forge.

To ensure complete aarch64 support, please verify if any other changes are required in the recipe. For instance, check if any dependencies need to be specified differently for aarch64.

✅ Verification successful

aarch64 support verified. The addition of linux-aarch64 in recipes/odgi/meta.yaml is correctly implemented, and no further aarch64-specific configurations are necessary.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any aarch64-specific configurations in the recipe

# Test: Search for aarch64-specific configurations
echo "Searching for aarch64-specific configurations in meta.yaml:"
rg --type yaml "aarch64" recipes/odgi/meta.yaml

echo "Checking for any skipped builds or architecture-specific requirements:"
rg --type yaml "(skip:|# \[.*\])" recipes/odgi/meta.yaml

Length of output: 411


11-12: LGTM. Please provide more information about the patch file.

The addition of the odgi-linux-aarch64.patch for the aarch64 architecture is in line with the PR objective. The conditional application # [aarch64] ensures it only affects the intended architecture.

Could you please provide more information about the contents of the odgi-linux-aarch64.patch file? This would help in understanding the specific changes made for aarch64 compatibility.

✅ Verification successful

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (4)
recipes/odgi/build.sh (1)

5-12: Architecture-specific compilation options added

The addition of architecture-specific compilation options is a good improvement:

  • It detects the machine architecture using uname -m.
  • For x86_64, it sets ARCH_OPTS="-march=sandybridge".
  • For other architectures, it sets ARCH_OPTS to an empty string.

This change allows for architecture-specific optimizations, which is beneficial. However, consider the following suggestions:

  1. You might want to add specific optimizations for other common architectures, such as ARM64 (aarch64).
  2. Consider using more specific x86_64 architecture flags if targeting newer processors.
  3. It might be helpful to add a comment explaining why "sandybridge" was chosen as the target for x86_64.

Here's a suggested improvement to handle more architectures:

case $(uname -m) in
    x86_64)
        ARCH_OPTS="-march=sandybridge"
        ;;
    aarch64)
        ARCH_OPTS="-march=armv8-a"
        ;;
    *)
        ARCH_OPTS=""
        ;;
esac

This change would add specific optimizations for ARM64 architecture while maintaining the current behavior for other architectures.

recipes/odgi/odgi-linux-aarch64.patch (3)

8-11: Approval: Removed x86-specific optimization flag

The removal of the -msse4.2 flag from EXTRA_FLAGS is a good change. This flag is specific to x86 architectures and not compatible with ARM64. Its removal will allow the package to build on ARM64 systems while retaining other important optimization flags.

This change is crucial for cross-architecture compatibility, especially for the targeted Linux ARM64 (aarch64) build support.


20-23: Approval with suggestion: Update comment for clarity

Commenting out the x86-64 specific attribute for the main function is appropriate for enabling ARM64 support. This change allows the function to be compiled without x86-64 specific optimizations.

Consider updating the comment above this line to reflect the change and explain the rationale. For example:

// We previously ensured main was compiled for x86-64 architecture.
// This has been disabled to support multiple architectures, including ARM64.
//int main(int argc, char *argv[]) __attribute__((__target__("arch=x86-64")));

1-26: Overall approval with considerations for cross-architecture performance

The changes in this patch file successfully remove x86-specific optimizations and constraints, which is crucial for enabling Linux ARM64 (aarch64) build support. This aligns well with the PR objectives.

Consider the following points for future improvements:

  1. Performance testing: Ensure that removing x86-specific optimizations doesn't significantly impact performance on x86 systems. Consider implementing architecture-specific optimizations that can be conditionally applied at build time.

  2. Documentation: Update any relevant documentation to reflect that the package now supports multiple architectures, including ARM64.

  3. CI/CD: If not already in place, set up continuous integration tests on both x86 and ARM64 systems to catch any architecture-specific issues early.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between c5610fa and e022601.

📒 Files selected for processing (3)
  • recipes/odgi/build.sh (2 hunks)
  • recipes/odgi/meta.yaml (2 hunks)
  • recipes/odgi/odgi-linux-aarch64.patch (1 hunks)
🧰 Additional context used
🔇 Additional comments (6)
recipes/odgi/build.sh (3)

3-3: Improved error handling and debugging

The addition of set -xe enhances the script's robustness and debuggability:

  • -x enables debug mode, printing each command before execution.
  • -e causes the script to exit immediately if any command fails.

This change will make it easier to identify and diagnose issues during the build process.


Line range hint 1-36: Overall assessment of changes

The modifications to build.sh are generally positive and improve the build process:

  1. Enhanced error handling and debugging with set -xe.
  2. Introduction of architecture-specific optimizations.
  3. Integration of these optimizations into the cmake command.

These changes should make the build process more robust and potentially more efficient on different architectures. However, consider the suggestions made in the previous comments to further improve the script:

  1. Handle more specific architectures (e.g., ARM64).
  2. Verify the necessity and safety of the -Ofast flag.
  3. Ensure consistency of optimization flags across different packages in the repository.

29-29: Architecture-specific options integrated into cmake command

The modification to include ${ARCH_OPTS} in the cmake command is appropriate and consistent with the earlier changes:

  • It allows the architecture-specific options to be passed to the build system.
  • This change enables the use of optimizations tailored to the target architecture.

However, I'd like to draw attention to the continued use of the -Ofast flag:

  1. -Ofast enables aggressive optimizations that may not be standards-compliant.
  2. It can potentially lead to unexpected behavior in some cases.

Consider verifying if -Ofast is necessary and safe for this package. If you're unsure, you might want to consult with the package maintainers or run extensive tests to ensure it doesn't introduce any issues.

To check if -Ofast is used consistently across different build scripts in the repository, you can run:

This will help ensure consistency in optimization levels across different packages.

recipes/odgi/meta.yaml (3)

53-54: LGTM. Additional platform correctly specified.

The addition of 'linux-aarch64' as an additional platform in the 'extra' section is correct and consistent with the PR objective. This explicitly declares support for the Linux ARM64 architecture in the package metadata.


11-12: LGTM. Verify the patch file contents.

The addition of the odgi-linux-aarch64.patch for the aarch64 architecture is appropriate and aligns with the PR objective. The conditional application # [aarch64] ensures it's only used when necessary.

To ensure the patch file is correct and present, please run the following command:

✅ Verification successful

Verified the patch file contents. The odgi-linux-aarch64.patch appropriately modifies CMakeLists.txt and src/main.cpp to support the aarch64 architecture as intended.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the existence and content of the odgi-linux-aarch64.patch file

# Test: Check if the patch file exists and display its content
if [ -f recipes/odgi/odgi-linux-aarch64.patch ]; then
    echo "Patch file exists. Content:"
    cat recipes/odgi/odgi-linux-aarch64.patch
else
    echo "Error: Patch file not found"
    exit 1
fi

Length of output: 1291


Line range hint 1-54: Overall changes look good, but address reported issues.

The changes to meta.yaml are minimal and correctly focused on adding ARM64 support, which aligns with the PR objective. However, there are some concerns that need to be addressed:

  1. Build Warnings: The PR comments mention warnings about odgi being built on an x86_64 machine. This suggests potential compatibility issues.

  2. Test Failures: The unit test in handle.cpp failed with a segmentation violation (SIGSEGV), indicating a serious issue.

These problems suggest that while the build configuration is correct, there might be underlying code issues when running on ARM64.

To investigate further, please run the following commands:

Please review the output of these commands and consider:

  1. Are there any ARM64-specific code paths that need adjustment?
  2. Is there a need for additional architecture-specific optimizations or checks in the code?
  3. Should we add conditional compilation or runtime checks for ARM64 support?

@martin-g
Copy link
Contributor Author

Even with 0.9.0 the test still fails with segfault:

06:49:47 �[32mBIOCONDA INFO�[0m (OUT) + odgi test�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) �[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) odgi test is a Catch v2.13.7 host application.�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) Run with -? for options�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) �[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) -------------------------------------------------------------------------------�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) Deletable handle graphs with mutable paths work�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) -------------------------------------------------------------------------------�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) /opt/conda/conda-bld/odgi_1729060814300/work/src/unittest/handle.cpp:2040�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) ...............................................................................�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) �[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) /opt/conda/conda-bld/odgi_1729060814300/work/src/unittest/handle.cpp:2040: FAILED:�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT)   {Unknown expression after the reported line}�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) due to a fatal error condition:�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT)   SIGSEGV - Segmentation violation signal�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) �[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) ===============================================================================�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) test cases:     6 |     5 passed | 1 failed�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) assertions: 60588 | 60587 passed | 1 failed�[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) �[0m
06:49:47 �[32mBIOCONDA INFO�[0m (OUT) /opt/conda/conda-bld/odgi_1729060814300/test_tmp/run_test.sh: line 7:  3883 Segmentation fault      (core dumped) odgi test�[0m
06:49:49 �[32mBIOCONDA INFO�[0m (OUT) WARNING: Tests failed for odgi-0.9.0-py39h6d2695b_1.tar.bz2 - moving package to /opt/conda/conda-bld/broken�[0m
06:49:49 �[32mBIOCONDA INFO�[0m (OUT) TESTS FAILED: odgi-0.9.0-py39h6d2695b_1.tar.bz2�[0m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aarch64 Related to adding linux-aarch64 support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant