Skip to content

Commit d23940d

Browse files
pitmonticonepitmonticone
authored
Merge branch 'cjdoris:main' into main
2 parents 7f76195 + df664df commit d23940d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+1652
-327
lines changed

‎.github/workflows/CompatHelper.yml

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,35 @@ on:
33
schedule:
44
- cron: 0 0 * * *
55
workflow_dispatch:
6+
permissions:
7+
contents: write
8+
pull-requests: write
69
jobs:
710
CompatHelper:
811
runs-on: ubuntu-latest
912
steps:
13+
- name: Check if Julia is already available in the PATH
14+
id: julia_in_path
15+
run: which julia
16+
continue-on-error: true
17+
- name: Install Julia, but only if it is not already available in the PATH
18+
uses: julia-actions/setup-julia@v1
19+
with:
20+
version: '1'
21+
arch: ${{ runner.arch }}
22+
if: steps.julia_in_path.outcome != 'success'
23+
- name: "Add the General registry via Git"
24+
run: |
25+
import Pkg
26+
ENV["JULIA_PKG_SERVER"] = ""
27+
Pkg.Registry.add("General")
28+
shell: julia --color=yes {0}
1029
- name: "Install CompatHelper"
1130
run: |
1231
import Pkg
1332
name = "CompatHelper"
1433
uuid = "aa819f21-2bde-4658-8897-bab36330d9b7"
15-
version = "2"
34+
version = "3"
1635
Pkg.add(; name, uuid, version)
1736
shell: julia --color=yes {0}
1837
- name: "Run CompatHelper"
@@ -23,3 +42,4 @@ jobs:
2342
env:
2443
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
2544
COMPATHELPER_PRIV: ${{ secrets.DOCUMENTER_KEY }}
45+
# COMPATHELPER_PRIV: ${{ secrets.COMPATHELPER_PRIV }}

‎.github/workflows/docs.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@ name: Docs
22

33
on:
44
push:
5-
branches: [main]
6-
tags: '*'
5+
branches:
6+
- main
7+
tags:
8+
- '*'
79

810
jobs:
911
docs:

‎.github/workflows/register-pypi.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
name: Register to PyPI
2+
3+
on:
4+
workflow_dispatch:
5+
6+
jobs:
7+
register:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: actions/checkout@master
11+
- name: Set up Python
12+
uses: actions/setup-python@v3
13+
with:
14+
python-version: "3.10"
15+
- name: Install build
16+
run: >-
17+
python -m
18+
pip install
19+
build
20+
--user
21+
- name: Build distribution
22+
run: >-
23+
python -m
24+
build
25+
--sdist
26+
--wheel
27+
--outdir dist/
28+
.
29+
- name: Publish to PyPI
30+
uses: pypa/gh-action-pypi-publish@release/v1
31+
with:
32+
password: ${{ secrets.PYPI_API_TOKEN }}

‎.github/workflows/tests-nightly.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,13 @@ name: Tests (nightly)
22

33
on:
44
pull_request:
5-
branches: [main]
5+
branches:
6+
- main
67
push:
7-
branches: [main]
8-
tags: '*'
8+
branches:
9+
- main
10+
tags:
11+
- '*'
912

1013
jobs:
1114
tests:

‎.github/workflows/tests.yml

Lines changed: 47 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,17 @@ name: Tests
22

33
on:
44
pull_request:
5-
branches: [main]
5+
branches:
6+
- main
67
push:
7-
branches: [main]
8-
tags: '*'
8+
branches:
9+
- main
10+
tags:
11+
- '*'
912

1013
jobs:
11-
tests:
12-
name: Test (${{ matrix.os }}-${{ matrix.arch }}, julia ${{ matrix.jlversion }})
14+
julia:
15+
name: Test (${{ matrix.os }}, julia ${{ matrix.jlversion }})
1316
runs-on: ${{ matrix.os }}
1417
strategy:
1518
fail-fast: false
@@ -19,7 +22,8 @@ jobs:
1922
jlversion: ['1','1.6']
2023
steps:
2124
- uses: actions/checkout@v2
22-
- uses: julia-actions/setup-julia@v1
25+
- name: Set up Julia ${{ matrix.jlversion }}
26+
uses: julia-actions/setup-julia@v1
2327
with:
2428
version: ${{ matrix.jlversion }}
2529
arch: ${{ matrix.arch }}
@@ -33,11 +37,43 @@ jobs:
3337
${{ runner.os }}-test-${{ env.cache-name }}-
3438
${{ runner.os }}-test-
3539
${{ runner.os }}-
36-
- uses: julia-actions/julia-buildpkg@v1
37-
- uses: julia-actions/julia-runtest@v1
40+
- name: Build package
41+
uses: julia-actions/julia-buildpkg@v1
42+
- name: Run tests
43+
uses: julia-actions/julia-runtest@v1
3844
env:
3945
JULIA_DEBUG: PythonCall
40-
- uses: julia-actions/julia-processcoverage@v1
41-
- uses: codecov/codecov-action@v1
46+
- name: Process coverage
47+
uses: julia-actions/julia-processcoverage@v1
48+
- name: Upload coverage to Codecov
49+
uses: codecov/codecov-action@v2
50+
python:
51+
name: Test (${{ matrix.os }}, python ${{ matrix.pyversion }})
52+
runs-on: ${{ matrix.os }}
53+
strategy:
54+
matrix:
55+
os: [ubuntu-latest, windows-latest, macos-latest]
56+
pyversion: ["3.7", "3.10"]
57+
steps:
58+
- uses: actions/checkout@v3
59+
- name: Set up Python ${{ matrix.pyversion }}
60+
uses: actions/setup-python@v4
4261
with:
43-
file: lcov.info
62+
python-version: ${{ matrix.pyversion }}
63+
- name: Install dependencies
64+
run: |
65+
python -m pip install --upgrade pip
66+
pip install flake8 pytest pytest-cov
67+
cp pysrc/juliacall/juliapkg-dev.json pysrc/juliacall/juliapkg.json
68+
pip install -e .
69+
- name: Lint with flake8
70+
run: |
71+
# stop the build if there are Python syntax errors or undefined names
72+
flake8 . --count --select=E9,F63,F7,F82 --ignore=F821 --show-source --statistics
73+
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
74+
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
75+
- name: Run tests
76+
run: |
77+
pytest -s --cov=pysrc
78+
- name: Upload coverage to Codecov
79+
uses: codecov/codecov-action@v2

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ __pycache__
55
build/
66
dist/
77
.CondaPkg/
8+
/jltest.*

‎CondaPkg.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11

2+
[deps.libstdcxx-ng]
3+
channel = "conda-forge"
4+
version = "<=julia"
5+
26
[deps.python]
37
channel = "conda-forge"
48
build = "*cpython*"

‎Project.toml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name = "PythonCall"
22
uuid = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"
33
authors = ["Christopher Doris <github.com/cjdoris>"]
4-
version = "0.9.3"
4+
version = "0.9.10"
55

66
[deps]
77
CondaPkg = "992eb4ea-22a4-4c89-a5bb-47a3300528ab"
@@ -10,13 +10,14 @@ Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
1010
MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
1111
Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
1212
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
13+
REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
1314
Requires = "ae029012-a4dd-5104-9daa-d747884805df"
1415
Serialization = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
1516
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
1617
UnsafePointers = "e17b2a0c-0bdf-430a-bd0c-3a23cae4ff39"
1718

1819
[compat]
19-
CondaPkg = "0.2.11"
20+
CondaPkg = "0.2.15"
2021
MacroTools = "0.5"
2122
Requires = "1"
2223
Tables = "1"
@@ -26,6 +27,7 @@ julia = "1.6.1"
2627
[extras]
2728
Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
2829
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
30+
TestItemRunner = "f8b46487-2199-4994-9208-9a1283c18c0a"
2931

3032
[targets]
31-
test = ["Aqua", "Test"]
33+
test = ["Aqua", "Test", "TestItemRunner"]

‎bump.jl

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
using Dates
2+
3+
oldver = ARGS[1]
4+
newver = ARGS[2]
5+
6+
function bump(file, oldpat, newpat)
7+
println("Bumping $file...")
8+
@assert oldpat != newpat
9+
oldtext = read(file, String)
10+
newtext = replace(oldtext, oldpat => newpat)
11+
@assert newtext != oldtext
12+
write(file, newtext)
13+
end
14+
15+
function bumpver(file, pattern, oldver, newver)
16+
@assert oldver != newver
17+
oldpat = replace(pattern, "{}" => oldver)
18+
@assert oldpat != pattern
19+
newpat = replace(pattern, "{}" => newver)
20+
@assert newpat != pattern
21+
bump(file, oldver, newver)
22+
end
23+
24+
bumpver("Project.toml", "version = \"{}\"\n", oldver, newver)
25+
bumpver("setup.cfg", "version = {}\n", oldver, newver)
26+
bumpver("pysrc/juliacall/__init__.py", "__version__ = '{}'\n", oldver, newver)
27+
bumpver("pysrc/juliacall/juliapkg.json", "\"version\": \"={}\"", oldver, newver)
28+
bumpver("pysrc/juliacall/juliapkg-dev.json", "\"version\": \"={}\"", oldver, newver)
29+
bumpver("src/PythonCall.jl", "VERSION = v\"{}\"", oldver, newver)
30+
bump("docs/src/releasenotes.md", "## Unreleased", "## $newver ($(today()))")

‎docs/make.jl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ makedocs(
4848
"Python to Julia" => "conversion-to-julia.md",
4949
],
5050
"compat.md",
51+
"faq.md",
5152
"pycall.md",
5253
"releasenotes.md",
5354
]

‎docs/src/compat.md

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,14 @@
22

33
Some packages require a little extra help to work nicely with PythonCall.
44

5-
Some of these are "fixes" that are silently applied for you, and some are just extra functions to bridge a gap. We aim to keep these as minimal as possible.
5+
Some of these are "fixes" that are silently applied for you, and some are just extra
6+
functions to bridge a gap. We aim to keep these as minimal as possible.
67

7-
## Stdlib
8+
## Python standard library
89

910
Whenever a Python exception is displayed by Julia, `sys.last_traceback` and friends are set. This allows the post-mortem debugger `pdb.pm()` to work. Disable by setting `PythonCall.CONFIG.auto_sys_last_traceback = false`.
1011

11-
## Tabular data & Pandas
12+
## Tabular data / Pandas
1213

1314
The abstract type [`PyTable`](@ref) is for wrapper types around Python tables, providing the
1415
[Tables.jl](https://github.com/JuliaData/Tables.jl) interface. `PyTable(x)` is shorthand
@@ -38,7 +39,7 @@ We also provide a simple MatPlotLib backend: `mpl.use("module://juliacall.matplo
3839
Now you can call `plt.show()` to display the figure with Julia's display mechanism.
3940
You can specify the format like `plt.show(format="png")`.
4041

41-
## GUIs (including MatPlotLib)
42+
## Python GUIs (including MatPlotLib)
4243

4344
### Event loops
4445

@@ -58,6 +59,23 @@ PythonCall.fix_qt_plugin_path
5859

5960
## IPython
6061

61-
If Python is running an IPython kernel, then:
62-
- Currently disabled: Julia's `Base.stdout` is set to Python's `sys.stdout`.
63-
- A `PythonDisplay` and `IPythonDisplay` are pushed onto Julia's display stack, so that `display(x)` goes to IPython if possible.
62+
The `juliacall.ipython` IPython extension adds these features to your IPython session:
63+
- The line magic `%jl code` executes the given Julia code in-line.
64+
- The cell magic `%%jl` executes a cell of Julia code.
65+
- Julia's `stdout` and `stderr` are redirected to IPython.
66+
- Calling `display(x)` from Julia will display `x` in IPython.
67+
68+
Enable the extension with `%load_ext juliacall.ipython`.
69+
See [the IPython docs](https://ipython.readthedocs.io/en/stable/config/extensions/).
70+
71+
## Asynchronous Julia code (including Makie)
72+
73+
Asynchronous Julia code will not normally run while Python is executing, unless it is in a
74+
separate thread.
75+
76+
This can be fixed by calling `jl.yield()` periodically from Python code, allowing the
77+
Julia scheduler to run.
78+
79+
When working at the Python REPL, you may call `juliacall.interactive()` which will allow
80+
Julia async code to run while the prompt is showing. This will allow interactive plots such
81+
as Makie to work.

‎docs/src/faq.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# FAQ & Troubleshooting
2+
3+
## Heap corruption when using PyTorch ([issue 215](https://github.com/cjdoris/PythonCall.jl/issues/215))
4+
5+
On some systems, you may see an error like the following when using `torch` and `juliacall`:
6+
```text
7+
Python(65251,0x104cf8580) malloc: Heap corruption detected, free list is damaged at 0x600001c17280
8+
*** Incorrect guard value: 1903002876
9+
Python(65251,0x104cf8580) malloc: *** set a breakpoint in malloc_error_break to debug
10+
[1] 65251 abort ipython
11+
```
12+
13+
A solution is to ensure that `juliacall` is imported before `torch`.

‎docs/src/juliacall.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,21 @@ In this example:
4646
If you are writing a package which uses Julia, then to avoid polluting the global `Main`
4747
namespace you instead should start with:
4848
```python
49-
import juliacall; jl = juliacall.newmodule("SomeName");
49+
import juliacall
50+
jl = juliacall.newmodule("SomeName")
51+
```
52+
53+
Julia modules have a special method `seval` which will evaluate a given piece of code given
54+
as a string in the module. This is most frequently used to import modules:
55+
```python
56+
from array import array
57+
jl.seval("using Statistics")
58+
x = array('i', [1, 2, 3])
59+
jl.mean(x)
60+
# 2.0
61+
y = array('i', [2,4,8])
62+
jl.cor(x, y)
63+
# 0.9819805060619657
5064
```
5165

5266
What to read next:

0 commit comments

Comments
 (0)