gh-109096: Deprecate `http.server.CGIHTTPRequestHandler` #109387

gpshead · 2023-09-13T22:23:27Z

So far I haven't found any practical users of this 1990s era functionality. Given we don't encourage the use of http.server as anyones main serving stack and that using the old CGI child process and environment method fell out of favor ages ago... Lets see if we can deprecate it.

Issue: test_httpservers causes os.fork DeprecationWarning, can we deprecate CGIHTTPRequestHandler? #109096

📚 Documentation preview 📚: https://cpython-previews--109387.org.readthedocs.build/

vstinner · 2023-09-15T13:14:43Z

Since the cgi module was removed in Python 3.13, it makes sense to deprecate CGIHTTPRequestHandler.

wsgiref has BaseCGIHandler, CGIHandler and IISCGIHandler classes. Are they still relevant nowadays? WSGI isn't suppose to replace CGI?

wsgiref.simple_ref uses env['GATEWAY_INTERFACE'] = 'CGI/1.1'. Is it WSGI or CGI? I'm confused :-(

gpshead · 2023-09-15T19:14:57Z

WSGI used much of the original CGI web server to application interfaces to make migrating from old CGI scripts child processes to in-server-process WSGI calls easy. https://peps.python.org/pep-3333/#specification-details

Doc/library/http.server.rst

JelleZijlstra

Couple of wording suggestions. Deprecating CGI sounds like a good idea

Lib/http/server.py

Doc/library/http.server.rst

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Doc/library/http.server.rst

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

vstinner

Can you please document the deprecation at https://docs.python.org/dev/whatsnew/3.13.html#deprecated and the future removal at https://docs.python.org/dev/whatsnew/3.13.html#pending-removal-in-python-3-15 ?

vstinner

LGTM. I dislike the _self hack in the test but it's not a big deal.

gpshead · 2023-09-15T21:25:40Z

I dislike the _self hack in the test but it's not a big deal.

Agreed. If it weren't in test code set to be deleted in two years I'd have refactored further.

bedevere-bot · 2023-09-15T21:37:54Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 3.x has failed when building commit 59073c9.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/179/builds/5489) and take a look at the build logs.
Check if the failure is related to this commit (59073c9) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/179/builds/5489

Failed tests:

test_tools

Failed subtests:

test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1004, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1100, in get_code
  File "<frozen importlib._bootstrap_external>", line 1199, in get_data
TypeError: descriptor 'close' for '_io.BufferedReader' objects doesn't apply to a '_io.FileIO' object


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/threading.py", line 1059, in _bootstrap_inner
    self.run()
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/threading.py", line 996, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_interpreters.py", line 483, in task
    interp = interpreters.create()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/support/interpreters.py", line 25, in create
    id = _interpreters.create(isolated=isolated)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: interpreter creation failed
k


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 146, in prepare
    copy_source_tree(srcdir, SRCDIR)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 95, in copy_source_tree
    shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_ctrl/ustar/fifotype', '/tmp/test_python_f41cjpzy/tmp92oau7y2/cpython/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_ctrl/ustar/fifotype', '`/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_ctrl/ustar/fifotype` is a named pipe'), ('/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_none/ustar/fifotype', '/tmp/test_python_f41cjpzy/tmp92oau7y2/cpython/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_none/ustar/fifotype', '`/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/build/test_python_16368æ/@test_16368_tmpæ-tardir/extractall_none/ustar/fifotype` is a named pipe')]

bedevere-bot · 2023-09-15T21:39:12Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 3.x has failed when building commit 59073c9.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/509/builds/4921) and take a look at the build logs.
Check if the failure is related to this commit (59073c9) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/509/builds/4921

Failed tests:

test.test_asyncio.test_subprocess

Failed subtests:

test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: ['process_exited', ('pipe_data_received', 1, b'stdout')] != [('pipe_data_received', 1, b'stdout'), ('p[95 chars]ted']

bedevere-bot · 2023-09-15T23:28:17Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO + PGO 3.x has failed when building commit 59073c9.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/101/builds/3754) and take a look at the build logs.
Check if the failure is related to this commit (59073c9) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/101/builds/3754

Failed tests:

test_venv

Failed subtests:

test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_venv.py", line 580, in test_zippath_from_non_installed_posix
    shutil.copytree(fn, os.path.join(libdir, name))
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_json/__pycache__/test_float.cpython-313.pyc.140735984064752', '/tmp/test_python_jv6l7pxc/tmpviaat877/lib/python3.13/test/test_json/__pycache__/test_float.cpython-313.pyc.140735984064752', "[Errno 2] No such file or directory: '/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_json/__pycache__/test_float.cpython-313.pyc.140735984064752'")]

…n#109387) Deprecate `http.server.CGIHTTPRequestHandler`. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Peterl777 · 2023-10-19T23:15:38Z

Only just came across this. As a teacher of Python, where I usually only have access to core Python (no internet access), I have found the CGIHTTPRequestHandler really useful for teaching basic web processes. I can't go to a fully-fledged web framework. CGI is really useful for students to get the whole POST/run/output process with minimal effort and maximal engagement.
Disappointed about this deprecation.

vstinner · 2023-10-19T23:56:01Z

Only just came across this. As a teacher of Python, where I usually only have access to core Python (no internet access)

You can provide Python with a old (non deprecated) copy of CGIHTTPRequestHandler to your students.

Peterl777 · 2023-10-20T04:56:11Z

Yes, that would work. But often I can't bring any external code in either; they've got core Python and that's all I can work with.

Deprecate http.server.CGIHTTPRequestHandler.

6ea9d69

bedevere-app bot mentioned this pull request Sep 13, 2023

test_httpservers causes os.fork DeprecationWarning, can we deprecate CGIHTTPRequestHandler? #109096

Closed

gpshead added 3.13 bugs and security fixes tests Tests in the Lib/test dir stdlib Python modules in the Lib dir labels Sep 13, 2023

satisfy older sphinx?

fd3dbe5

gpshead added the docs Documentation in the Doc dir label Sep 15, 2023

serhiy-storchaka reviewed Sep 15, 2023

View reviewed changes

Doc/library/http.server.rst Outdated Show resolved Hide resolved

JelleZijlstra reviewed Sep 15, 2023

View reviewed changes

Doc/library/http.server.rst Outdated Show resolved Hide resolved

Doc/library/http.server.rst Outdated Show resolved Hide resolved

appease Older Sphinx?

24ebc80

JelleZijlstra reviewed Sep 15, 2023

View reviewed changes

Lib/http/server.py Outdated Show resolved Hide resolved

Lib/http/server.py Outdated Show resolved Hide resolved

Doc/library/http.server.rst Outdated Show resolved Hide resolved

gpshead and others added 3 commits September 15, 2023 12:49

docstring fixup.

d1f0c73

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

docstring typo fixup

3e3a5d3

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Update Doc/library/http.server.rst

e1e9b22

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

JelleZijlstra reviewed Sep 15, 2023

View reviewed changes

Doc/library/http.server.rst Outdated Show resolved Hide resolved

unindent for older sphinx

0c44758

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

vstinner reviewed Sep 15, 2023

View reviewed changes

gpshead marked this pull request as ready for review September 15, 2023 20:32

bedevere-app bot added the awaiting review label Sep 15, 2023

gpshead self-assigned this Sep 15, 2023

gpshead added 2 commits September 15, 2023 13:43

what's new entries.

7cd0b6f

Merge branch 'main' into deprecate/CGI

d95be85

vstinner approved these changes Sep 15, 2023

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Sep 15, 2023

gpshead merged commit 59073c9 into python:main Sep 15, 2023

bedevere-app bot removed the awaiting core review label Sep 15, 2023

gpshead deleted the deprecate/CGI branch September 15, 2023 21:26

csm10495 pushed a commit to csm10495/cpython that referenced this pull request Sep 28, 2023

pythongh-109096: Deprecate http.server.CGIHTTPRequestHandler (pytho…

99d114e

…n#109387) Deprecate `http.server.CGIHTTPRequestHandler`. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Uh oh!

gh-109096: Deprecate http.server.CGIHTTPRequestHandler #109387

gh-109096: Deprecate http.server.CGIHTTPRequestHandler #109387

Uh oh!

Conversation

gpshead commented Sep 13, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Sep 15, 2023

Uh oh!

gpshead commented Sep 15, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

gpshead commented Sep 15, 2023

Uh oh!

bedevere-bot commented Sep 15, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 15, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 15, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Peterl777 commented Oct 19, 2023

Uh oh!

vstinner commented Oct 19, 2023

Uh oh!

Peterl777 commented Oct 20, 2023

Uh oh!

Uh oh!

gh-109096: Deprecate `http.server.CGIHTTPRequestHandler` #109387

gh-109096: Deprecate `http.server.CGIHTTPRequestHandler` #109387

gpshead commented Sep 13, 2023 •

edited by github-actions bot

Loading