Feature request: a modular way of implementing external packages

[kskyten]

It would be very useful to be able to transpile code that imports other packages. I think these package specific translations should go to their own Julia packages (e.g P2JNumpy.jl for #2). To do this, we need to come up with a way of registering the implemented translations in Py2Jl somehow. Any ideas how to do this @thautwarm?

[kskyten]

Here is a rough idea: We have a Context object that is passed along in the ASTGen function calls. This object keeps track of imports and symbols that are in scope with https://github.com/JuliaStaging/NameResolution.jl. We also a have a FileIO style registry, which is used to look up how to expand external functions.

[thautwarm]

@kskyten
The feature you want can be easily achieved by evaluating the generated julia code in a custom module that provides the symbols of external packages.

However, I'm sorry that so far Py2Jl.jl is now not in active development due to

• the limited human resources, as I'm the only developer.
• the lack of prospects, as python to julia conversion is not considered as a really useful tool.

As a result, I put this package in a lower priority and do not see a chance to actively develop it again in recent.

Fortunately, the principal of the Py2Jl is quite easy to understand once you understand the pattern matching used in this repo. Refactoring a new translation package is not a big task, so you could make a new one. I can give your support about the related techniques and tools.

[kskyten]

I'm happy to work on it on my own. Fortunately there is not a lot of code. It is a fairly advanced application of MLStyle with heavy use of macros, so I'm having some trouble understanding the code. Any guidance will be much appreciated.

Could you clarify what you mean by evaluating the generated code in a custom module? I was thinking of making separate packages that contain functions that map to expressions, such as fundef in my PR.

[thautwarm]

Any guidance will be much appreciated.

We could talk by e-mails or just ping me on Slack!

[thautwarm]

If you understand the Record pattern matching, other helper functions are not neccessary to understand, because I believe you can do the conversion(Python ast to julia Expr) in a better approach.

This repo is implemented in a hurry to do the proof of concepts, but after I knew python to julia conversion is feasible I quicly lost my interests.

Besides, feel free to ask me(my id is Taine Zhao on Slack) anything concerned.

You can see Python AST structures at here:
https://raw.githubusercontent.com/python/cpython/3.7/Parser/Python.asdl
Though it changes over versions.