|
1 |
| -# how to build/test/install pycorese java gateway and python wrapper from sources |
| 1 | +# Development Installation |
2 | 2 |
|
3 |
| -## build/install the python package |
| 3 | +This document describes how to install the development environment for the **pycorese** package. |
4 | 4 |
|
5 |
| -### prerequesite |
| 5 | +**pycorese** is a Python wrapper for the [Corese-core](https://github.com/corese-stack/corese-core) Java library. To build and use this package, you need to have Java installed on your system. |
6 | 6 |
|
| 7 | +**pycorese** provides two options for running the Corese Java code using the following Python packages to access Java objects: |
| 8 | + |
| 9 | +* [Py4J](https://www.py4j.org/) |
| 10 | +* [JPype](https://jpype.readthedocs.io/en/latest/) |
| 11 | + |
| 12 | +That's the reason the installation process is a bit more complex than for a standard Python package. |
| 13 | + |
| 14 | +## Clone the GitHub repository |
| 15 | + |
| 16 | +```bash |
| 17 | +git clone https://github.com/corese-stack/corese-python.git |
| 18 | +cd corese-python |
7 | 19 | ```
|
8 |
| -pip install --upgrade pip setuptools wheel build |
| 20 | + |
| 21 | + |
| 22 | +## Python build environment |
| 23 | + |
| 24 | +You can use the provided [conda](https://docs.conda.io/en/latest/) environment file to create a virtual environment with the necessary dependencies. |
| 25 | + |
| 26 | +```bash |
| 27 | +conda env update -f pkg/env/corese-python.yaml |
| 28 | +conda activate corese-python |
9 | 29 | ```
|
| 30 | +Or install the dependencies manually: |
| 31 | + |
| 32 | +* project dependencies: |
| 33 | + ```bash |
| 34 | + pip install py4j jpype1 pandas |
| 35 | + ``` |
| 36 | +* build dependencies: |
| 37 | + ```bash |
| 38 | + pip install --upgrade pip setuptools wheel build |
| 39 | + ``` |
| 40 | +* test dependencies: |
| 41 | + ```bash |
| 42 | + pip install pytest pytest-cov |
| 43 | + ``` |
| 44 | +* documentation dependencies: |
| 45 | + ```bash |
| 46 | + pip install sphinx pydata_sphinx_theme |
| 47 | + ``` |
| 48 | + |
| 49 | +<!-- TODO: Add other documentation dependencies install support--> |
| 50 | +<!-- TODO: Add conda install support--> |
| 51 | + |
| 52 | +## Java build environment |
| 53 | + |
| 54 | +To build the package Java Development Kit (JDK) version 11 or higher and the [Gradle](https://docs.gradle.org/current/userguide/userguide.html) build tool are required. |
| 55 | + |
| 56 | +If Java is not installed, visit the [official website](https://www.java.com/en/download/help/download_options.html) for installation instructions. |
| 57 | + |
| 58 | +Gradle can be installed as an extension to your IDE or as a standalone tool. |
| 59 | + |
| 60 | +* Gradle extension for VSCode is available from the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-gradle) |
10 | 61 |
|
11 |
| -### clean all (not necessary if your just downloaded the sources) |
| 62 | +* To install Gradle as a standalone tool, follow the instructions on the [official website](https://gradle.org/install/). |
| 63 | + |
| 64 | + |
| 65 | +## Building the package |
| 66 | + |
| 67 | +Clean all the build directories (not necessary if you just downloaded the source code): |
12 | 68 |
|
13 | 69 | ```
|
14 |
| -rm -fr dist build resources |
| 70 | +rm -fr dist build resources src/pycorese.egg-info |
15 | 71 | ```
|
16 | 72 |
|
17 |
| -### build the package |
| 73 | +Build the package: |
18 | 74 |
|
19 | 75 | ```
|
20 | 76 | python -m build
|
21 | 77 | ```
|
22 | 78 |
|
23 |
| -which build the packages into `./dist` |
| 79 | +This command builds the packages into `./dist` directory. Note that the custom `sdist` command is implemented in [setup.py](./setup.py). |
24 | 80 |
|
25 |
| -Remark: |
26 |
| -- do not run `python setup.py` which will not build the full package |
27 |
| -- the described install process will: |
| 81 | +The custom `sdist` command adds the following steps: |
28 | 82 |
|
29 |
| - 1/ compile the `corese-python-4.x.y-jar-with-dependencies.jar` file |
30 |
| - 2/ download the `corese-core-4.x.y-jar-with-dependencies.jar` file from maven |
| 83 | +* compiling the `corese-python-x.y.z-jar-with-dependencies.jar` file using the Gradle build tool. This jar file is required to run Corese using the `Py4J` bridge. |
| 84 | +* downloading the `corese-core-x.y.z-jar-with-dependencies.jar` file from the Maven repository. This jar file is required to run Corese using the `JPype` bridge. |
| 85 | +* copying the jar files to the `./resources` directory. |
31 | 86 |
|
32 |
| -- these two files are necessary to run the wrappers and are part of the distribution |
| 87 | +> [!NOTE] |
| 88 | +> - do not run `python setup.py` that will not build the full package. |
| 89 | +> - the versions of `pycorese` and Java libraries are maintained separately. |
| 90 | +> - `corese-python` version should be the same as `corese-core` it depends on, for simplicity reasons. |
| 91 | +> - the commands for the first two steps are provided in the [Obtaining Java libraries manually](#obtain-java-libraries-manually) section. |
33 | 92 |
|
34 |
| -### test |
| 93 | +## Testing the package |
35 | 94 |
|
36 |
| -From the top directory, or in the `tests` sub-directory |
37 |
| - |
38 |
| -``` |
39 |
| -pip install pytest |
40 |
| -``` |
| 95 | +From the top directory, or in the `./tests` sub-directory run the command: |
41 | 96 |
|
42 | 97 | ```
|
43 | 98 | pytest -v
|
44 | 99 | ```
|
45 | 100 |
|
46 |
| -If a specific test fails, you can have more information, using: |
47 |
| -(you need to know the filename, test class name, test name) |
| 101 | +If a specific test fails, you can have more information, using the following command: |
48 | 102 |
|
49 |
| -eg: |
50 | 103 | ```
|
51 | 104 | pytest tests/test_api.py::Test_api::test_bad_bridge
|
52 | 105 | ```
|
53 | 106 |
|
54 |
| -### code coverage |
55 |
| - |
56 |
| -Install the coverage package: |
57 |
| - |
58 |
| -``` |
59 |
| -pip install pytest-cov |
60 |
| -``` |
| 107 | +> [!NOTE] |
| 108 | +> - substitute the filename, test class name, and test name with your specific test. |
61 | 109 |
|
62 |
| -And run the test coverage: |
| 110 | +Run the test coverage: |
63 | 111 |
|
64 | 112 | ```
|
65 | 113 | pytest --cov
|
66 | 114 | ```
|
67 | 115 |
|
68 |
| -If you prefer a browsable coverage report: |
| 116 | +For the HTML coverage report, run the following commands: |
69 | 117 |
|
70 | 118 | ```
|
71 | 119 | pytest --cov --cov-report=html
|
72 | 120 | open htmlcov/index.html
|
73 | 121 | ```
|
74 | 122 |
|
75 | 123 |
|
76 |
| -### install the locally built package |
| 124 | +## Installing the locally built package |
77 | 125 |
|
78 | 126 | ```
|
79 |
| -pip install dist/pycorese-1.0.1-py3-none-any.whl |
| 127 | +pip install dist/pycorese-0.1.1-py3-none-any.whl |
80 | 128 | ```
|
81 | 129 |
|
82 | 130 | or
|
83 | 131 | ```
|
84 |
| -pip install dist/pycorese-1.0.1.tar.gz |
| 132 | +pip install dist/pycorese-0.1.1.tar.gz |
85 | 133 | ```
|
86 | 134 |
|
87 |
| -- verify your installation |
| 135 | +## Verifying the installation |
88 | 136 |
|
89 | 137 | ```
|
90 | 138 | $ pip list | grep corese
|
91 |
| -pycorese 1.0.1 |
| 139 | +pycorese 0.1.1 |
92 | 140 |
|
93 | 141 | $ python -c 'import pycorese'
|
94 | 142 | ```
|
| 143 | +> [!NOTE] |
| 144 | +> - change the version number accordingly. |
95 | 145 |
|
96 |
| -## Appendix 1: run local python example |
97 | 146 |
|
98 |
| -### Conda environment |
| 147 | +## Run a simple example |
99 | 148 |
|
100 |
| -If necessary, we provide a conda environment: |
| 149 | +Without installing the package you can run the following command (the default Java bridge is `py4j`): |
101 | 150 |
|
102 |
| -```bash |
103 |
| -conda env update -f pkg/env/corese-python.yaml |
104 |
| -conda activate corese-python |
105 | 151 | ```
|
106 |
| - |
107 |
| -This makes available the python libraries: `pandas`, `py4j`, `jpype1` |
108 |
| - |
109 |
| -### run a simple example using py4j bridge (without installing) |
110 |
| - |
111 |
| -``` |
112 |
| -./python_examples/simple_query.py -j $PWD/build/libs/corese-python-4.6.0-jar-with-dependencies.jar |
| 152 | +./examples/simple_query.py -j $PWD/build/libs/corese-python-4.6.0-jar-with-dependencies.jar |
113 | 153 | ```
|
114 | 154 |
|
115 |
| -Remark: to build this jar file, you must follow the Appendix 2 instructions |
116 |
| - |
117 |
| -### experimental: run a simple example using jpype bridge (without installing) |
118 |
| - |
119 |
| -We focus the development on the py4j wrapping. The (still provided) jpype interface |
120 |
| -may still work (without garanty): |
| 155 | +or change the bridge to `jpype`: |
121 | 156 |
|
122 | 157 | ```
|
123 |
| -./python_examples/simple_query.py -b jpype -j /somewhere/corese-core-4.6.0-jar-with-dependencies.jar |
| 158 | +./examples/simple_query.py -b jpype -j $PWD/build/libs/corese-core-4.6.0-jar-with-dependencies.jar |
124 | 159 | ```
|
125 | 160 |
|
| 161 | +<!-- **_NOTE:_** --> |
| 162 | +> [!NOTE] |
| 163 | +> - the jar files are obtained either by [building the package](#building-the-package) or [manually](#obtain-java-libraries-manually). |
| 164 | +> - the primary development focus is on using the `py4j` bridge. |
126 | 165 |
|
127 |
| -## Appendix 2: java compilation description |
128 | 166 |
|
129 |
| -Remark: all these commands are launched then building/installing using the previous described process |
| 167 | +## Obtain Java libraries manually |
130 | 168 |
|
131 |
| -### build jar file locally |
| 169 | +In case you want to build `corese-python-x.y.z-jar-with-dependencies.jar` Java library separately, use the following commands: |
132 | 170 |
|
133 | 171 | ```
|
134 |
| -./gradlew shadowJar |
| 172 | +gradlew shadowJar |
135 | 173 | ```
|
136 | 174 |
|
137 |
| -### download |
| 175 | +In case you want to download the `corese-core-x.y.z-jar-with-dependencies.jar` Java library separately, use the following commands: |
138 | 176 |
|
139 | 177 | ```
|
140 |
| -./gradlew downloadCoreseCore |
| 178 | +gradlew downloadCoreseCore |
141 | 179 | ```
|
| 180 | +These tasks are defined in the [build.gradle.kts](./build.gradle.kts) file. |
0 commit comments