Skip to content

Conversation

@tdavidcl
Copy link
Member

No description provided.

@github-actions
Copy link
Contributor

Thanks @tdavidcl for opening this PR!

You can do multiple things directly here:
1 - Comment pre-commit.ci run to run pre-commit checks.
2 - Comment pre-commit.ci autofix to apply fixes.
3 - Add label autofix.ci to fix authorship & pre-commit for every commit made.
4 - Add label light-ci to only trigger a reduced & faster version of the CI (need the full one before merge).
5 - Add label trigger-ci to create an empty commit to trigger the CI.

Once the workflow completes a message will appear displaying informations related to the run.

Also the PR gets automatically reviewed by gemini, you can:
1 - Comment /gemini review to trigger a review
2 - Comment /gemini summary for a summary
3 - Tag it using @gemini-code-assist either in the PR or in review comments on files

@gemini-code-assist
Copy link

Summary of Changes

Hello @tdavidcl, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands Shamrock's capabilities for integrating with and emulating Phantom SPH simulations, providing a more streamlined and robust workflow for users working with Phantom data. A core focus was placed on optimizing distributed data communication through a new caching and multi-buffer approach, which is seamlessly integrated across various simulation components. These changes collectively aim to improve performance, flexibility, and user experience when running complex astrophysical simulations.

Highlights

  • Phantom Emulation Mode: Introduced a new Python utility module (shamrock.utils.phantom) and an example script (run_phantom_run_emulation.py) to facilitate setting up and running Phantom SPH simulations directly within the Shamrock framework. This includes parsing Phantom input files, loading dumps, and evolving simulations with optional callbacks.
  • Optimized Sparse Distributed Data Communication: Implemented a new caching mechanism (DDSCommCache) for sparse distributed data communication in the shamalgs library. This enhancement allows for more efficient memory management by dynamically allocating multiple communication buffers based on message sizes, improving performance for large-scale distributed simulations. An environment variable SPARSE_COMM_MODE is added to switch between the new and old communication modes.
  • Ghost Zone Exchange Refactoring: Refactored the management of ghost zone exchange nodes (ExchangeGhostLayer, ExchangeGhostField) in SPH and GSPH solvers. These nodes are now stored in SolverStorage and passed explicitly to ghost handling methods, promoting better modularity and control over communication patterns.
  • Unit System Handling for Phantom Dumps: Enhanced the Phantom dump loading and saving functionality to include robust unit system detection and conversion. The system now warns if Phantom dump units are not explicitly set and performs CGS to SI conversions (and vice-versa) for relevant physical quantities.
  • Python Utilities and Bindings: Added a new Python utility module (shamrock.utils.url) for downloading files with progress reporting. A Python binding for shamrock.sys.mpi_barrier() was also introduced for better control over MPI synchronization in Python scripts.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant new feature: a Phantom emulation mode. This includes Python utilities for loading and running Phantom simulations, along with an example script. On the C++ side, there's a major refactoring of the sparse communication layer to support multiple buffers and caching, which should improve performance for large messages. The changes are extensive and well-structured. My review focuses on improving the robustness and clarity of the new Python utilities.


plt.title("t = {:0.3f} [code unit]".format(model.get_time()))
plt.xlabel("x")
plt.ylabel("z")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The y-axis of the plot is labeled "z", but the view seems to be in the x-y plane, as indicated by delta_y = (0.0, radius * 2, 0.0). For a face-on view of a disc integrated along the z-axis, the y-axis label should be "y".

Suggested change
plt.ylabel("z")
plt.ylabel("y")


# phantom dumps are 00000.tmp if before start and then sim_name_{:05d}
# parse the dump number
dump_number = int(dump_file_name.split("_")[1].split(".")[0])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current parsing of dump_number using split('_') is fragile and will fail if the simulation name (sim_name) contains underscores. Using rsplit('_', 1) would be more robust as it splits from the right, separating the dump number from the simulation name reliably.

Suggested change
dump_number = int(dump_file_name.split("_")[1].split(".")[0])
dump_number = int(dump_file_name.rsplit("_", 1)[1].split(".")[0])

with open(os.path.join(simulation_folder, f"{sim_name}.in"), "w") as f:
for line in lines:
if "dumpfile" in line:
line = f" dumpfile = {get_ph_dump_file_name(dump_number)} ! dump file to start from\n"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The line to update the dumpfile in the input file has hardcoded indentation. This can alter the file's formatting if the original indentation is different. It's better to preserve the original indentation of the line and only replace the value.

Suggested change
line = f" dumpfile = {get_ph_dump_file_name(dump_number)} ! dump file to start from\n"
line = f"{line[:len(line) - len(line.lstrip())]}dumpfile = {get_ph_dump_file_name(dump_number)} ! dump file to start from\n"

@github-actions
Copy link
Contributor

Workflow report

workflow report corresponding to commit 3a81a4d
Commiter email is timothee.davidcleris@proton.me

Light CI is enabled. This will only run the basic tests and not the full tests.
Merging a PR require the job "on PR / all" to pass which is disabled in this case.

Pre-commit check report

Pre-commit check: ✅

trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check for merge conflicts................................................Passed
check that executables have shebangs.....................................Passed
check that scripts with shebangs are executable..........................Passed
check for added large files..............................................Passed
check for case conflicts.................................................Passed
check for broken symlinks................................................Passed
check yaml...............................................................Passed
detect private key.......................................................Passed
No-tabs checker..........................................................Passed
Tabs remover.............................................................Passed
Validate GitHub Workflows................................................Passed
clang-format.............................................................Passed
ruff check...............................................................Passed
ruff format..............................................................Passed
Check doxygen headers....................................................Passed
Check license headers....................................................Passed
Check #pragma once.......................................................Passed
Check SYCL #include......................................................Passed
No ssh in git submodules remote..........................................Passed
No UTF-8 in files (except for authors)...................................Passed

Test pipeline can run.

Doxygen diff with main

Removed warnings : 34
New warnings : 38
Warnings count : 8112 → 8116 (0.0%)

Detailed changes :
+ src/shambackends/include/shambackends/DeviceBufferRef.hpp:24: warning: Member DeviceBufferRef (typedef) of namespace sham is not documented.
+ src/shambackends/include/shambackends/DeviceBufferRef.hpp:24: warning: Member DeviceBufferRef (typedef) of namespace sham is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:100: warning: Member rtree_rint_field (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:103: warning: Member rtree_rint_field (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:107: warning: Member ghost_layout (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:109: warning: Member merged_patchdata_ghost (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:110: warning: Member ghost_layout (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:112: warning: Member merged_patchdata_ghost (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:116: warning: Member soundspeed (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:119: warning: Member soundspeed (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\rho'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\rho'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\rho'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:120: warning: Found unknown command '\rho'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:121: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:121: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:121: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:121: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:122: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:122: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:122: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:122: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:123: warning: Found unknown command '\rho'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:123: warning: Found unknown command '\rho'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:123: warning: Found unknown command '\rho'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:123: warning: Found unknown command '\rho'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:125: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:125: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:125: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:125: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:126: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:126: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:126: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:126: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:127: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:127: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:127: warning: Found unknown command '\nabla'
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:127: warning: Found unknown command '\nabla'
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:132: warning: Member old_duint (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:135: warning: Member old_duint (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:136: warning: Member interface (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:137: warning: Member neighbors (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:138: warning: Member io (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:139: warning: Member interface (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:140: warning: Member neighbors (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:141: warning: Member io (variable) of struct shammodels::gsph::SolverStorage::Timings is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:141: warning: Member reset() (function) of struct shammodels::gsph::SolverStorage::Timings is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:142: warning: Member timings_details (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:144: warning: Member reset() (function) of struct shammodels::gsph::SolverStorage::Timings is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:145: warning: Member timings_details (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:73: warning: Member exchange_gz_node (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:74: warning: Member exchange_gz_positions (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:75: warning: Member part_counts_with_ghost (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:78: warning: Member part_counts_with_ghost (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:79: warning: Member hpart_with_ghosts (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:82: warning: Member hpart_with_ghosts (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:92: warning: Member ghost_patch_cache (variable) of struct shammodels::gsph::SolverStorage is not documented.
+ src/shammodels/gsph/include/shammodels/gsph/modules/SolverStorage.hpp:95: warning: Member ghost_patch_cache (variable) of struct shammodels::gsph::SolverStorage is not documented.
- src/shammodels/sph/src/io/Phantom2Shamrock.cpp:198: warning: Member write_shamrock_units_in_phantom_dump< f64 >(std::optional< shamunits::UnitSystem< f64 > > &units, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
- src/shammodels/sph/src/io/Phantom2Shamrock.cpp:198: warning: Member write_shamrock_units_in_phantom_dump< f64 >(std::optional< shamunits::UnitSystem< f64 > > &units, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
+ src/shammodels/sph/src/io/Phantom2Shamrock.cpp:213: warning: Member write_shamrock_units_in_phantom_dump< f64 >(std::optional< shamunits::UnitSystem< f64 > > &units, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
+ src/shammodels/sph/src/io/Phantom2Shamrock.cpp:213: warning: Member write_shamrock_units_in_phantom_dump< f64 >(std::optional< shamunits::UnitSystem< f64 > > &units, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
- src/shammodels/sph/src/io/Phantom2Shamrock.cpp:218: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
- src/shammodels/sph/src/io/Phantom2Shamrock.cpp:218: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
- src/shammodels/sph/src/io/Phantom2Shamrock.cpp:218: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
+ src/shammodels/sph/src/io/Phantom2Shamrock.cpp:233: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
+ src/shammodels/sph/src/io/Phantom2Shamrock.cpp:233: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.
+ src/shammodels/sph/src/io/Phantom2Shamrock.cpp:233: warning: Member write_shamrock_boundaries_in_phantom_dump(BCConfig< Tvec > &cfg, std::tuple< Tvec, Tvec > box_size, PhantomDump &dump, bool bypass_error) (function) of namespace shammodels::sph is not documented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant