Incompatibility with EmPy 4.0

[wentasah]

Bug report

rosidl is not compatible with recently released EmPy 4.0.x.

This manifests with errors like (the last few lines are the most important):

...
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
CMake Error at /nix/store/f6hkk0fbrjnwibg1gxr285cqr66pnx86-ros-humble-rosidl-adapter-3.1.5-r2/share/rosidl_adapter/cmake/rosidl_adapt_interfaces.cmake:59 (message):
  
  execute_process(/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6/bin/python3.11
  -m rosidl_adapter --package-name builtin_interfaces --arguments-file
  /build/rcl_interfaces-release-release-humble-builtin_interfaces-1.2.1-1/build/rosidl_adapter__arguments__builtin_interfaces.json
  --output-dir
  /build/rcl_interfaces-release-release-humble-builtin_interfaces-1.2.1-1/build/rosidl_adapter/builtin_interfaces
  --output-file
  /build/rcl_interfaces-release-release-humble-builtin_interfaces-1.2.1-1/build/rosidl_adapter/builtin_interfaces.idls)
  returned error code 1:

  AttributeError processing template 'msg.idl.em'

  Traceback (most recent call last):

    File "/nix/store/f6hkk0fbrjnwibg1gxr285cqr66pnx86-ros-humble-rosidl-adapter-3.1.5-r2/lib/python3.11/site-packages/rosidl_adapter/resource/__init__.py", line 51, in evaluate_template
      em.BUFFERED_OPT: True,
      ^^^^^^^^^^^^^^^

  AttributeError: module 'em' has no attribute 'BUFFERED_OPT'

I was able to get rid of the errors after applying the following patch (for humble version):

diff --git a/rosidl_adapter/rosidl_adapter/resource/__init__.py b/rosidl_adapter/rosidl_adapter/resource/__init__.py
index ce33358..4a0c2e2 100644
--- a/rosidl_adapter/rosidl_adapter/resource/__init__.py
+++ b/rosidl_adapter/rosidl_adapter/resource/__init__.py
@@ -44,19 +44,20 @@ def evaluate_template(template_name, data):
     template_path = os.path.join(os.path.dirname(__file__), template_name)
 
     output = StringIO()
+    config = em.Configuration()
+    config.deleteOnError = True
+    config.rawErrors = True
     try:
         _interpreter = em.Interpreter(
             output=output,
-            options={
-                em.BUFFERED_OPT: True,
-                em.RAW_OPT: True,
-            })
+            config=config,
+        )
 
         with open(template_path, 'r') as h:
             content = h.read()
         _interpreter.invoke(
             'beforeFile', name=template_name, file=h, locals=data)
-        _interpreter.string(content, template_path, locals=data)
+        _interpreter.string(content, locals=data)
         _interpreter.invoke('afterFile')
 
         return output.getvalue()
@@ -78,7 +79,7 @@ def _evaluate_template(template_name, **kwargs):
             'beforeInclude', name=template_path, file=h, locals=kwargs)
         content = h.read()
     try:
-        _interpreter.string(content, template_path, kwargs)
+        _interpreter.string(content, kwargs)
     except Exception as e:  # noqa: F841
         print(
             f"{e.__class__.__name__} processing template '{template_name}': "
diff --git a/rosidl_cmake/rosidl_cmake/__init__.py b/rosidl_cmake/rosidl_cmake/__init__.py
index 0c7d1ed..0b4e680 100644
--- a/rosidl_cmake/rosidl_cmake/__init__.py
+++ b/rosidl_cmake/rosidl_cmake/__init__.py
@@ -125,12 +125,12 @@ def expand_template(
 
     global interpreter
     output = StringIO()
+    config = em.Configuration()
+    config.deleteOnError = True
+    config.rawErrors = True
     interpreter = em.Interpreter(
         output=output,
-        options={
-            em.BUFFERED_OPT: True,
-            em.RAW_OPT: True,
-        },
+        config=config,
     )
 
     global template_prefix_path
@@ -146,7 +146,7 @@ def expand_template(
             template_content = h.read()
             interpreter.invoke(
                 'beforeFile', name=template_name, file=h, locals=data)
-        interpreter.string(template_content, template_path, locals=data)
+        interpreter.string(template_content, locals=data)
         interpreter.invoke('afterFile')
     except Exception as e:  # noqa: F841
         if os.path.exists(output_file):
@@ -195,7 +195,7 @@ def _expand_template(template_name, **kwargs):
             'beforeInclude', name=str(template_path), file=h, locals=kwargs)
         content = h.read()
     try:
-        interpreter.string(content, str(template_path), kwargs)
+        interpreter.string(content, kwargs)
     except Exception as e:  # noqa: F841
         print(f"{e.__class__.__name__} in template '{template_path}': {e}",
               file=sys.stderr)

I'm not really sure whether the change to interpreter.string() is correct, but I was able to compile my project with it. Unfortunately, I'm not experienced with using EmPy and the lack of EmPy source code repository with development history doesn't make it easy to figure out what changes would be correct. Therefore, I hope somebody more experienced would be able to follow from here.

[JamesTann]

Is there a way to downgrade it to use an older empy? I am new to rosidl and trying to build a package of message definitions for the first time

Edit: downgraded empy manually to empy==3.3.4

[ahcorde]

Related PR #821

[fujitatomoya]

closing this in favor of #821