generate op frontend code for IDE auto-complete #8244
Conversation
szha
force-pushed
the
code_gen
branch
2 times, most recently
from
6078bb0 to
420f222
Compare
|
As a PyCharm user, I am looking forward to this very much. :) |
python/mxnet/ndarray/op.py
Outdated
| try: | ||
| if int(_os.environ.get("MXNET_ENABLE_CYTHON", True)) == 0: | ||
| from .._ctypes.ndarray import NDArrayBase | ||
| from .._ctypes.ndarray import CachedOp, _imperative_invoke | ||
| from .._ctypes.ndarray import NDArrayBase, CachedOp |
There was a problem hiding this comment.
Import these from register?
python/mxnet/symbol/register.py
Outdated
| symbol_function.__module__ = 'mxnet.symbol' | ||
| return symbol_function | ||
|
|
||
| if not _internal.__dict__.get('skip_register'): |
There was a problem hiding this comment.
where does skip_register come from?
There was a problem hiding this comment.
The idea is that if the code is already generated, a flag will be written to the internal module indicating that. I will include its generation in the next commit.
I'm currently testing locally the generation on different python versions.
There was a problem hiding this comment.
You can detect it from len(op.__all__)
szha
force-pushed
the
code_gen
branch
2 times, most recently
from
09133e7 to
0fa1528
Compare
|
awesome feature! |
szha
changed the title
| """Post-installation for installation mode.""" | ||
| def run(self): | ||
| install.run(self) | ||
| from mxnet.base import _generate_op_module_signature |
There was a problem hiding this comment.
Is this guaranteed to import the mxnet that's just installed? Can you read the path from install or self?
There was a problem hiding this comment.
There's no guarantee, and neither is there guarantee that self or install resides in the target site-packages. Currently what this code does in practice is:
- generate the code in the source code folder first (hence the .gitignore change)
- copy source codes into $PWD/build/mxnet and add packaging info
- copy $PWD/build to the site-packages and byte-compile the .py files.
There was a problem hiding this comment.
What happens for pip package?
There was a problem hiding this comment.
It copies the whole tree from mxnet-distro/mxnet-build/python/mxnet (without generated source) to mxnet-distro/mxnet, and does the above step 1 and 2. Finally, the whole tree in mxnet-distro/build is packaged into a wheel which contains the generated source.
* decouple codegen and register * address comments on refactor * auto generate code * trigger test
Description
This PR aims at providing auto-complete for text editors by actually writing the generated op functions to module source codes. The solution should work for pip/setuptools install command.
python setup.py developshould remain runtime-generated.Checklist
Essentials
make lint)Changes
Comments
python setup.py develop.setup.py, another change will happen in the distro package to take effect in pip.