Move pyface.ui.qt4 to pyface.ui.qt (#1223)

This moves `pyface.ui.qt4.*` to `pyface.ui.qt` and helps with gradually
deprecating all uses of "qt4". The current state is internal
consistency, optional hooks for backwards compatibility for applications
that depend on Pyface.

Changes for downstream libraries and applications should be fairly
simple: replacing `qt` with `qt4` in appropriate import statements.

But there are some circumstances where this may not be feasible:

- where the end-user doesn't control the code-base (eg. someone who
pip-installs Mayavi and gets the new version of Pyface automatically)
- where an application developer needs to make a newer version of pyface
work with code they don't control that imports from `pyface.ui.qt4.*`
- where a library developer wants to be compatible with newer and older
versions of pyface

This PR solves two of these problems by providing opt-in import hooks
that replace imports of `pyface.ui.qt4.*` with corresponding imports of
`pyface.ui.qt.*` so that `sys.modules` points to the new location
(avoiding issues with duplicated objects and modules which would happen
with some other strategies). These hooks are not available by default,
but can be accessed in a number of ways:

- end-users can set environment variables to turn it on:
`ETS_QT4_IMPORTS` to directly turn on the import hooks, and
`ETS_TOOLKIT=qt4` (as opposed to `ETS_TOOLKIT=qt`) with the assumption
that this is an older environment
- application developers can either set `ETSConfig.toolkit = "qt4"` (as
opposed to `ETSConfig.toolkit = "qt"`) with the assumption that this is
flagging that this is an application made with older assumptions about
import locations; or if they need more control, they can install the
import hooks into `sys.meta_path` in a way that works best for them and
any other import hooks that they might be using.

The third case of a library developer who wants to support newer and
older versions should be done through the usual mechanisms (eg. try
importing `pyface.ui.qt.foo` and if it fails import `pyface.ui.qt4.foo`;
or looking at pyface version numbers). An alternative which will work in
almost all cases is to instead use `toolkit_object()` to get the thing
that they want, which will automatically get it from the right place.

Done:
- [x] check about pickle compatibility with mementos: Tasks and
Workbench mementos are independent of toolkit
- [x] migration documentation (added to toolkits section)

Note:

This ended up taking about a day longer than expected because the
original working version of the code was using a deprecated API that
would be removed in Python 3.12, so it needed re-implementation after
digging in to understand what the new importlib API was doing. In doing
so it has been generalized a bit, since we expect to also do this in
TraitsUI at a minimum.

Fixes #560

---------

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>

docs/source/index.rst

@@ -43,7 +43,7 @@ that signifies the GUI toolkit.
 
 The supported values of **ETSConfig.toolkit** are:
 
-* 'qt4' or 'qt': PySide2, PySide6 or `PyQt <http://riverbankcomputing.co.uk/pyqt/>`_,
+* 'qt' or 'qt4': PySide2, PySide6 or `PyQt <http://riverbankcomputing.co.uk/pyqt/>`_,
   which provide Python bindings for the `Qt <http://www.qt.io>`_ framework.
 * 'wx': `wxPython 4 <http://www.wxpython.org>`_, which provides Python bindings
   for the `wxWidgets <http://wxwidgets.org>`_ toolkit.

docs/source/overview.rst

@@ -45,7 +45,7 @@ interface, which in turn inherits from :py:class:`.IDialog`,
 is the combination of these.  The :py:class:`.MFileDialog` class provides some
 mix-in capabilities around the expression of filters for file types.  And then
 there are two concrete implementations, :py:class:`pyface.ui.wx.file_dialog.FileDialog`
-for wxPython and :py:class:`pyface.ui.qt4.file_dialog.FileDialog` for the Qt
+for wxPython and :py:class:`pyface.ui.qt.file_dialog.FileDialog` for the Qt
 backends.  These toolkit classes in turn inherit from the appropriate toolkit's
 :py:class:`Dialog`, :py:class:`Window`, and :py:class:`Widget` classes, as well
 as the :py:class:`.MFileDialog` mixin.

docs/source/toolkits.rst

@@ -21,7 +21,7 @@ ways:
   attribute appropriately::
 
     from tratis.etsconfig.api import ETSConfig
-    ETSConfig.toolkit = 'qt4'
+    ETSConfig.toolkit = 'qt'
 
   This must be done _before_ any widget imports in your application, including
   importing :py:mod:`pyface.api`.  Precisely, this must be set before the
@@ -30,7 +30,7 @@ ways:
 If for some reason Pyface can't load a deliberately specified toolkit, then it
 will raise an exception.
 
-If the toolkit is not specified, Pyface will try to load the ``qt4`` or ``wx``
+If the toolkit is not specified, Pyface will try to load the ``qt`` or ``wx``
 toolkits, in that order, and then any other toolkits that it knows about
 other than ``null``.  If all of those fail, then it will try to load the
 ``null`` toolkit.
@@ -57,8 +57,8 @@ The API module for the new widget class typically looks something like this::
 
 The base toolkits use the identifier to select which module to import the
 toolkit object by constructing a full module path from the partial path and
-importing the object.  For example the ``qt4`` backend will look for the
-concrete implementation in :py:mod:`pyface.ui.qt4.my_package.my_widget`
+importing the object.  For example the ``qt`` backend will look for the
+concrete implementation in :py:mod:`pyface.ui.qt.my_package.my_widget`
 while the ``wx`` backend will look for
 :py:mod:`pyface.ui.wx.my_package.my_widget`.
 
@@ -73,6 +73,34 @@ second trait provides a hook where an application can insert other packages
 into the search path to override the default implementations of a toolkit's
 widgets, if needed.
 
+The "qt4" Toolkit
+-----------------
+
+The "qt4" toolkit is the same as the "qt" toolkit in almost all respects:
+in older versions of Pyface it was the standard name for all the Qt-based
+toolkits whether or not they were actually using Qt4.
+
+However it does trigger some backwards-compatibility code that may be useful
+for legacy applications. In particular it installs import hooks that makes the
+``pyface.ui.qt4.*`` package namespace an alias for ``pyface.ui.qt.*`` modules.
+
+This backwards-compatibility code can also be invoked by setting the
+``ETS_QT4_IMPORTS`` environment variable to any non-empty value, or adding
+an instance of the :py:class:`pyface.ui.ShadowedModuleFinder` module finder
+to :py:attr:`sys.meta_path` list.
+
+..  warning::
+
+    Library code which imports from ``pyface.ui.qt4.*`` should not use this
+    compatibility code.  Instead it should be updated to import from
+    ``pyface.ui.qt.*`` as soon as practical.  Backwards-compatibility can be
+    achieved fairly easily by using :py:attr:`pyface.toolkit.toolkit` to access
+    objects rather than direct imports.
+
+This backwards-compatibility code will be removed in Pyface 9, and applications
+which rely on the particulars of the implementation are encouraged to
+migrate to the newer import locations as soon as practical.
+
 Toolkit Entrypoints
 ===================
 

etstool.py

@@ -141,7 +141,10 @@
 doc_ignore = {
     "pyface/wx/*",
     "pyface/qt/*",
-    "pyface/ui/*",
+    "pyface/ui/null/*",
+    "pyface/ui/qt/*",
+    "pyface/ui/qt4/*",
+    "pyface/ui/wx/*",
     "pyface/dock/*",
     "pyface/util/fix_introspect_bug.py",
     "pyface/grid/*",

examples/python_editor.py

@@ -68,7 +68,7 @@ def __init__(self, **traits):
             )
         )
 
-        # Add a tool bar if we are using qt4 - wx has layout issues
+        # Add a tool bar if we are using qt - wx has layout issues
         if toolkit_object.toolkit.startswith("qt"):
             from pygments.styles import STYLE_MAP
 

examples/tasks/advanced/python_editor_qt4.py

@@ -126,7 +126,7 @@ def _show_line_numbers_updated(self, event=None):
     def _create_control(self, parent):
         """ Creates the toolkit-specific control for the widget.
         """
-        from pyface.ui.qt4.code_editor.code_widget import AdvancedCodeWidget
+        from pyface.ui.qt.code_editor.code_widget import AdvancedCodeWidget
 
         self.control = control = AdvancedCodeWidget(parent)
         self._show_line_numbers_updated()

examples/tasks/advanced/run.py

@@ -3,7 +3,7 @@
 components.
 
 Note: Run it with
-$ ETS_TOOLKIT='qt4' python run.py
+$ ETS_TOOLKIT='qt' python run.py
 as the wx backend is not supported yet for the TaskWindow.
 """
 

examples/tasks/basic/run.py

@@ -3,7 +3,7 @@
 components.
 
 Note: Run it with
-$ ETS_TOOLKIT='qt4' python run.py
+$ ETS_TOOLKIT='qt' python run.py
 as the wx backend is not supported yet for the TaskWindow.
 """
 

pyface/base_toolkit.py

@@ -57,7 +57,7 @@
 - after that, we try every 'pyface.toolkit' plugin we can find.  If one
   succeeds, we consider ourselves good, and set the ETSConfig.toolkit
   appropriately.  The order is configurable, and by default will try to load
-  the `qt4` toolkit first, `wx` next, then all others in arbitrary order,
+  the `qt` toolkit first, `wx` next, then all others in arbitrary order,
   and `null` last.
 
 - finally, if all else fails, we try to load the null toolkit.
@@ -81,7 +81,7 @@
 logger = logging.getLogger(__name__)
 
 
-TOOLKIT_PRIORITIES = {"qt4": -2, "wx": -1, "null": float("inf")}
+TOOLKIT_PRIORITIES = {"qt": -2, "wx": -1, "null": float("inf")}
 default_priorities = lambda plugin: TOOLKIT_PRIORITIES.get(plugin.name, 0)
 
 

pyface/mimedata.py

@@ -10,6 +10,6 @@
 # Import the toolkit specific version.
 from pyface.toolkit import toolkit_object
 
-# WIP: Currently only supports qt4 backend. API might change without
+# WIP: Currently only supports qt backend. API might change without
 # prior notification
 PyMimeData = toolkit_object("mimedata:PyMimeData")

pyface/tasks/tests/test_dock_pane_toggle_group.py

@@ -19,7 +19,7 @@
 from traits.etsconfig.api import ETSConfig
 
 
-USING_WX = ETSConfig.toolkit not in ["", "qt4"]
+USING_WX = ETSConfig.toolkit not in {"", "qt", "qt4"}
 
 
 class BogusTask(Task):

pyface/tasks/tests/test_editor_area_pane.py

@@ -18,7 +18,7 @@
 GuiTestAssistant = toolkit_object("util.gui_test_assistant:GuiTestAssistant")
 no_gui_test_assistant = GuiTestAssistant.__name__ == "Unimplemented"
 
-USING_WX = ETSConfig.toolkit not in ["", "qt4"]
+USING_WX = ETSConfig.toolkit not in {"", "qt", "qt4"}
 
 
 class EditorAreaPaneTestCase(unittest.TestCase):

pyface/tasks/tests/test_enaml_dock_pane.py

@@ -13,7 +13,7 @@
 
 # Skip tests if Enaml is not installed or we're using the wx backend.
 SKIP_REASON = None
-if ETSConfig.toolkit not in ["", "qt4"]:
+if ETSConfig.toolkit not in {"", "qt", "qt4"}:
     SKIP_REASON = "Enaml does not support WX"
 else:
     try:

pyface/tasks/tests/test_enaml_editor.py

@@ -13,7 +13,7 @@
 
 # Skip tests if Enaml is not installed or we're using the wx backend.
 SKIP_REASON = None
-if ETSConfig.toolkit not in ["", "qt4"]:
+if ETSConfig.toolkit not in {"", "qt", "qt4"}:
     SKIP_REASON = "Enaml does not support WX"
 else:
     try:

pyface/tasks/tests/test_enaml_task_pane.py

@@ -13,7 +13,7 @@
 
 # Skip tests if Enaml is not installed or we're using the wx backend.
 SKIP_REASON = None
-if ETSConfig.toolkit not in ["", "qt4"]:
+if ETSConfig.toolkit not in {"", "qt", "qt4"}:
     SKIP_REASON = "Enaml does not support WX"
 else:
     try:

pyface/tests/test_base_toolkit.py

@@ -27,12 +27,17 @@ def test_missing_toolkit(self):
 
     def test_find_current_toolkit_no_etsconfig(self):
         old_etsconfig_toolkit = ETSConfig._toolkit
+        expected_toolkit = (
+            "qt"
+            if old_etsconfig_toolkit == "qt4"
+            else old_etsconfig_toolkit
+        )
         ETSConfig._toolkit = ""
         try:
             toolkit = find_toolkit("pyface.toolkits", old_etsconfig_toolkit)
             self.assertEqual(toolkit.package, "pyface")
-            self.assertEqual(toolkit.toolkit, old_etsconfig_toolkit)
-            self.assertEqual(ETSConfig.toolkit, old_etsconfig_toolkit)
+            self.assertEqual(toolkit.toolkit, expected_toolkit)
+            self.assertEqual(ETSConfig.toolkit, expected_toolkit)
         finally:
             ETSConfig._toolkit = old_etsconfig_toolkit
 

pyface/toolkit.py

@@ -19,5 +19,5 @@
 from .base_toolkit import find_toolkit
 
 
-# The toolkit function.
+#: The callable toolkit object.
 toolkit = toolkit_object = find_toolkit("pyface.toolkits")

pyface/ui/__init__.py

@@ -0,0 +1,113 @@
+# (C) Copyright 2005-2023 Enthought, Inc., Austin, TX
+# All rights reserved.
+#
+# This software is provided without warranty under the terms of the BSD
+# license included in LICENSE.txt and may be redistributed only under
+# the conditions described in the aforementioned license. The license
+# is also available online at http://www.enthought.com/licenses/BSD.txt
+#
+# Thanks for using Enthought open source!
+
+from importlib import import_module
+from importlib.abc import MetaPathFinder, Loader
+from importlib.machinery import ModuleSpec
+from importlib.util import find_spec
+import sys
+
+
+# Import hooks for loading pyface.ui.qt.* in place of a pyface.ui.qt4.*
+# This is just the implementation, it is not connected in this module, but
+# is available for applications which want to install it themselves.
+# It is here rather than in pyface.ui.qt4 so it can be imported and used
+# without generating the warnings from pyface.ui.qt4
+#
+# To use manually:
+#
+#     import sys
+#     sys.meta_path.append(ShadowedModuleFinder())
+
+class ShadowedModuleLoader(Loader):
+    """This loads another module into sys.modules with a given name.
+
+    Parameters
+    ----------
+    fullname : str
+        The full name of the module we're trying to import.
+        Eg. "pyface.ui.qt4.foo"
+    new_name : str
+        The full name of the corresponding "real" module.
+        Eg. "pyface.ui.qt.foo"
+    new_spec : ModuleSpec instance
+        The spec object for the corresponding "real" module.
+    """
+
+    def __init__(self, fullname, new_name, new_spec):
+        self.fullname = fullname
+        self.new_name = new_name
+        self.new_spec = new_spec
+
+    def create_module(self, spec):
+        """Create the module object.
+
+        This doesn't create the module object directly, rather it gets the
+        underlying "real" module's object, importing it if needed.  This object
+        is then returned as the "new" module.
+        """
+        if self.new_name not in sys.modules:
+            import_module(self.new_name)
+        return sys.modules[self.new_name]
+
+    def exec_module(self, module):
+        """Execute code for the module.
+
+        This is given a module which has already been executed, so we don't
+        need to execute anything.  However we do need to remove the __spec__
+        that the importlibs machinery has injected into the module and
+        replace it with the original spec for the underlying "real" module.
+        """
+        # patch up the __spec__ with the true module's original __spec__
+        if self.new_spec:
+            module.__spec__ = self.new_spec
+            self.new_spec = None
+
+
+class ShadowedModuleFinder(MetaPathFinder):
+    """MetaPathFinder for shadowing modules in a package
+
+    This finds modules with names that match a package but arranges loading
+    from a different package.  By default this is  matches imports from any
+    path starting with pyface.ui.qt4. and returns a loader which will instead
+    load from pyface.ui.qt.*
+
+    The end result is that sys.modules has two entries for pointing to the
+    same module object.
+
+    This may be hooked up by code in pyface.ui.qt4, but it can also be
+    installed manually with::
+
+        import sys
+        sys.meta_path.append(ShadowedModuleFinder())
+
+    Parameters
+    ----------
+    package : str
+        The prefix of the "shadow" package.
+    true_package : str
+        The prefix of the "real" package which contains the actual code.
+    """
+
+    def __init__(self, package="pyface.ui.qt4.", true_package="pyface.ui.qt."):
+        self.package = package
+        self.true_package = true_package
+
+    def find_spec(self, fullname, path, target=None):
+        if fullname.startswith(self.package):
+            new_name = fullname.replace(self.package, self.true_package, 1)
+            new_spec = find_spec(new_name)
+            if new_spec is None:
+                return None
+            return ModuleSpec(
+                name=fullname,
+                loader=ShadowedModuleLoader(fullname, new_name, new_spec),
+                is_package=(new_spec.submodule_search_locations is not None),
+            )

pyface/ui/qt4/code_editor/tests/test_code_widget.py → pyface/ui/qt/code_editor/tests/test_code_widget.py

@@ -16,7 +16,7 @@
 from pyface.qt.QtTest import QTest
 
 
-from pyface.ui.qt4.code_editor.code_widget import (
+from pyface.ui.qt.code_editor.code_widget import (
     CodeWidget,
     AdvancedCodeWidget,
 )

pyface/ui/qt4/data_view/data_view_widget.py → pyface/ui/qt/data_view/data_view_widget.py

@@ -19,7 +19,7 @@
 from pyface.data_view.i_data_view_widget import (
     IDataViewWidget, MDataViewWidget
 )
-from pyface.ui.qt4.layout_widget import LayoutWidget
+from pyface.ui.qt.layout_widget import LayoutWidget
 from .data_view_item_model import DataViewItemModel
 
 # XXX This file is scaffolding and may need to be rewritten

pyface/ui/qt4/data_view/tests/test_data_view_item_model.py → pyface/ui/qt/data_view/tests/test_data_view_item_model.py

@@ -20,7 +20,7 @@
 from pyface.data_view.exporters.row_exporter import RowExporter
 from pyface.data_view.data_formats import table_format
 from pyface.data_view.value_types.api import FloatValue
-from pyface.ui.qt4.data_view.data_view_item_model import DataViewItemModel
+from pyface.ui.qt.data_view.data_view_item_model import DataViewItemModel
 
 
 @requires_numpy

pyface/ui/qt4/data_view/tests/test_data_wrapper.py → pyface/ui/qt/data_view/tests/test_data_wrapper.py

@@ -11,7 +11,7 @@
 from unittest import TestCase
 
 from pyface.qt.QtCore import QMimeData
-from pyface.ui.qt4.data_view.data_wrapper import DataWrapper
+from pyface.ui.qt.data_view.data_wrapper import DataWrapper
 
 
 class TestDataWrapper(TestCase):

pyface/ui/qt4/fields/field.py → pyface/ui/qt/fields/field.py

@@ -14,7 +14,7 @@
 from traits.api import Any, provides
 
 from pyface.fields.i_field import IField, MField
-from pyface.ui.qt4.layout_widget import LayoutWidget
+from pyface.ui.qt.layout_widget import LayoutWidget
 
 
 @provides(IField)

pyface/ui/qt4/fields/time_field.py → pyface/ui/qt/fields/time_field.py

@@ -16,7 +16,7 @@
 from pyface.qt.QtGui import QTimeEdit
 
 from pyface.fields.i_time_field import ITimeField, MTimeField
-from pyface.ui.qt4.util.datetime import pytime_to_qtime, qtime_to_pytime
+from pyface.ui.qt.util.datetime import pytime_to_qtime, qtime_to_pytime
 from .field import Field