Commit 7dbbd24
authored
Stop HistorySavingThread before fork (ipython#15115)
Python 3.12+ issues a DeprecationWarning if `os.fork()` is called while
there are multiple threads, and in fact this is not strictly safe on any
version of Python, although on Linux with glibc it mostly works well
enough that we don't notice. The man page for `fork()` says:
> After a fork() in a multithreaded program, the child can safely call
only async-signal-safe functions (see signal-safety(7)) until such time
as it calls execve(2).
This is incompatible with executing any Python code, hence the warning.
IPython is always multi-threaded because of HistorySavingThread, so it's
never safe to call `os.fork()` or use multiprocessing's fork context.
However, I think we can easily resolve this by stopping the history
saving thread before fork, and starting it again after, so at the moment
of fork it's a single-threaded process. In principle, I think it's safe
to start it again in the `after_in_parent` handler, but Python's check
for threads to issue the warning runs after that callback, so to avoid
triggering the warning, it's better to start it again when there's
something for it to do.
This unfortunately only solves the issue for IPython in the terminal,
since IPykernel uses several more threads. But fixing it in the terminal
is already useful, and this would also be one small step towards fixing
it in IPykernel.
-----
Testing:
```python
import os
import time
def func_with_fork():
if (pid := os.fork()) > 0:
print("Parent")
t = os.waitpid(pid, 0)
print("Parent: child finished", t)
else:
print("Child")
time.sleep(0.5)
print("Child finishing")
os._exit(0)
if __name__ == "__main__":
func_with_fork()
```
```shell
PYTHONWARNINGS=default ipython
```1 file changed
+30
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| |||
689 | 690 | | |
690 | 691 | | |
691 | 692 | | |
| 693 | + | |
692 | 694 | | |
693 | 695 | | |
694 | 696 | | |
| |||
699 | 701 | | |
700 | 702 | | |
701 | 703 | | |
| 704 | + | |
| 705 | + | |
702 | 706 | | |
703 | 707 | | |
704 | 708 | | |
| |||
709 | 713 | | |
710 | 714 | | |
711 | 715 | | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
712 | 730 | | |
713 | 731 | | |
714 | 732 | | |
| |||
970 | 988 | | |
971 | 989 | | |
972 | 990 | | |
973 | | - | |
974 | | - | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
975 | 995 | | |
976 | 996 | | |
977 | 997 | | |
| |||
1003 | 1023 | | |
1004 | 1024 | | |
1005 | 1025 | | |
1006 | | - | |
1007 | | - | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
1008 | 1030 | | |
1009 | 1031 | | |
1010 | 1032 | | |
| |||
1059 | 1081 | | |
1060 | 1082 | | |
1061 | 1083 | | |
| 1084 | + | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
1062 | 1088 | | |
1063 | 1089 | | |
1064 | 1090 | | |
| |||
0 commit comments