Skip to content

Convert some Perl build scripts to Python #13231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jsquyres
Copy link
Member

@jsquyres jsquyres commented May 5, 2025

Inspired by #13230, continue the slow trend of removing Perl from our build process, and move to a more sustainable (at least by number of developers who know the language) language: Python.

Use generative AI to convert 3 Perl scripts to Python: mpif-values.pl, gen-mpi-mangling.pl, and gen-mpi-sizeof.pl. Validate the results by code review+inspection and diff'ing the results of the Python-generated files with the original Perl-generated files.


Hey @ggouaillardet I was inspired by your PR about fixing the alignment of Fortran constants to see if generative AI could convert some of our legacy Perl scripts to Python. It worked surprisingly well (it did about 80-90% of the conversion work; I had to tweak a bit after the fact). So I made this PR to do 2 things:

  1. Convert the 3 Perl scripts named above to Python.
    • NOTE: This is not an attempt to remove all Perl from the build system -- just these 3 (I'm looking mainly at you, autogen.pl, but there's also a handful of other Perl scripts in the build, and a truckload of minor helper Perl scripts in contrib).
  2. Take your commit from fortran: fix common symbol sizes and alignments #13230 and adapt it to the newly-Pythonized script.

What do you think? We could use this PR for main / v6.0, and re-target yours for v5.0.x (because let's not take the Perl-to-Python conversion to the stable v5.0.x branch).

@rhc54
Copy link
Contributor

rhc54 commented May 5, 2025

@jsquyres can you please point me to something that explains how to use the AI tools for this purpose? Would like to learn how

@jsquyres
Copy link
Member Author

jsquyres commented May 5, 2025

@jsquyres can you please point me to something that explains how to use the AI tools for this purpose? Would like to learn how

All I did was:

  1. Drag-n-drop the .pl file to the chat box (it uploaded + attached the perl script).
  2. In the prompt, I typed "Convert the attached perl script to python"

That was it.

ChatGPT -- the free version -- wasn't great at this. It would sorta convert some of it, and then say "Would you like me to convert the rest?" I would type "Yes, finish converting the entire rest of the perl script to python." It would say, "Ok, I have converted the next part of the perl script...". And it was doing it in piecemeal interior chunks -- i.e., it would spit out a whole new python script with a bunch of new stuff scattered throughout the code. I repeated this several times but couldn't get ChatGPT to convert any of the 3 perl scripts in their entirety.

I then tried the Gemini Pro version that I have access to from my University / faculty access, and did the same thing: attach the .pl script and say "Convert the attached perl script to python". Gemini did the entire conversion in the single go. I don't know if that's because I have access to non-free Gemini, or if it would have done in that in the free version.

That being said, I did still have to read the Gemini-produced code and make some tweaks and changes. But it probably did 80-90% of the conversion work for me.

It's also worth mentioning that the 3 scripts that I converted are a little lengthy -- shortest one was ~200 lines of Perl; longest was ~600 lines. Perhaps free ChatGPT would do better with shorter input Perl scripts...? 🤷‍♂️

@ggouaillardet
Copy link
Contributor

@jsquyres I'm not a python guy but that looks good to me.

I did update the commit message on my PR to explain what has been done (no other code change) so you might want to copy/paste it here and we'll be good to go.

jsquyres and others added 2 commits May 6, 2025 11:46
Continue the slow trend of removing Perl from our build process, and
move to a more sustainable (at least by number of developers who know
the language) language: Python.

Use generative AI to convert 3 Perl scripts to Python: mpif-values.pl,
gen-mpi-mangling.pl, and gen-mpi-sizeof.pl.  Validate the results by
code review+inspection and diff'ing the results of the
Python-generated files with the original Perl-generated files.

Signed-off-by: Jeff Squyres <jeff@squyres.com>
with mpif-h and usempi, MPI constants (e.g. MPI_COMM_WORLD)
are all parts of a unique common block. Fortran compilers
generally have alignment requirements for these (16 with gfortran,
32 with ifort or 64 with nvfortran to name a few), so pass
these requirements to the actual symbols that are defined in
the C code to make pick linkers (e.g. ubuntu) happy pandas.
Such linkers also complain about the size of MPI_STATUS, so define
these are arrays intead of pointer.

Refs open-mpi#13043

Thanks MJ Rutter for the report

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
@jsquyres jsquyres force-pushed the pr/convert-some-perl-scripts-to-python branch from 63dddbc to ec2b164 Compare May 6, 2025 15:47
@jsquyres
Copy link
Member Author

jsquyres commented May 6, 2025

I did update the commit message on my PR to explain what has been done (no other code change) so you might want to copy/paste it here and we'll be good to go.

Done!

@hppritcha
Copy link
Member

this looks good and it works for me.

@hppritcha hppritcha self-requested a review May 6, 2025 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants