Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] restructure rosidl to work with IDL files #298

Closed
wants to merge 39 commits into from
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
cf56be9
copy ROS msg/srv parser
dirk-thomas Sep 18, 2018
48635bc
remove message/service specific stuff from the other module
dirk-thomas Sep 19, 2018
b1caf0d
add rosidl_adapter package
dirk-thomas Sep 19, 2018
bd944f8
use rosidl_parser in rosidl_cmake, allow passing interface files with…
dirk-thomas Sep 19, 2018
552b08c
add lark grammar and parsing functions
dirk-thomas Sep 19, 2018
9149d16
update rosidl_generator_c to use IDL files
dirk-thomas Sep 19, 2018
9b0e9d0
fix Python 3.5 issue
dirk-thomas Sep 21, 2018
43a9052
only require either messages or services
dirk-thomas Sep 21, 2018
2425af5
add u16string type and functions and use for wstring, add long double
dirk-thomas Sep 22, 2018
166d457
fix for 43a90521
dirk-thomas Sep 22, 2018
e177d0b
[rosidl_generator_c] fix (w)string with upper bound
dirk-thomas Sep 22, 2018
66f9129
get_filename_component(... EXT) includes '.'
sloretz Sep 24, 2018
34b5c46
Try to convert message and service files
sloretz Sep 24, 2018
4393367
Further work on IDL.
mjcarroll Sep 24, 2018
0a4af16
Include message name in constants module
sloretz Sep 25, 2018
028b117
1.125 is a legal floating point literal
sloretz Sep 25, 2018
75593a4
multiple annotation_appl per member
sloretz Sep 25, 2018
b5df207
add string literal escaping
sloretz Sep 25, 2018
0bee962
First attempt at char literal escaping
sloretz Sep 25, 2018
88569ee
More informative error message.
mjcarroll Sep 25, 2018
94dbeeb
Change char mapping to int8.
mjcarroll Sep 25, 2018
9cc7654
String escaping fixup.
mjcarroll Sep 25, 2018
a069cbc
Check escape before graphic char
sloretz Sep 25, 2018
7aa8d9d
.srv use string literal escape function
sloretz Sep 25, 2018
0202f3d
Allow verbatim rosidl_array_init
sloretz Sep 25, 2018
d4e108b
Remove to_character_literal since type changed to uint8
sloretz Sep 25, 2018
a5cc511
array values use verbatim annotation
sloretz Sep 25, 2018
208e063
Merge branch 'shane/rosidl-reloaded' into rosidl-reloaded
dirk-thomas Oct 15, 2018
0aee9b7
add additional rule to grammar to distinguish relevants parts of a ma…
dirk-thomas Oct 19, 2018
499028e
add priorities / weights to prefer certain rules
dirk-thomas Oct 19, 2018
25b4f1e
add new object representation for interfaces
dirk-thomas Oct 19, 2018
5fdbd8c
add parser to parse .idl files using lark and transform the AST into …
dirk-thomas Oct 19, 2018
bc9578f
add test for .idl parser
dirk-thomas Oct 19, 2018
a9fd325
working copy dump
dirk-thomas Nov 9, 2018
4635a93
extract comments and units
dirk-thomas Nov 9, 2018
6bd82b0
replace function pointer with import
dirk-thomas Nov 9, 2018
49463a6
rm obsolete file
dirk-thomas Nov 10, 2018
9116396
migrate rosidl_generator_cpp, fix rosidl_generator_c
dirk-thomas Nov 13, 2018
5ee3cb0
avoid duplicate logic
dirk-thomas Nov 14, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions rosidl_adapter/package.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format2.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="2">
<name>rosidl_adapter</name>
<version>0.5.1</version>
<description>
Scripts to convert .msg/.srv files to .idl.
</description>
<maintainer email="dthomas@osrfoundation.org">Dirk Thomas</maintainer>
<license>Apache License 2.0</license>

<exec_depend>python3-empy</exec_depend>
<exec_depend>rosidl_parser</exec_depend>

<test_depend>ament_copyright</test_depend>
<test_depend>ament_flake8</test_depend>
<test_depend>ament_pep257</test_depend>
<test_depend>python3-pytest</test_depend>

<export>
<build_type>ament_python</build_type>
</export>
</package>
52 changes: 52 additions & 0 deletions rosidl_adapter/rosidl_adapter/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copyright 2018 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from enum import Enum


class InterfaceType(Enum):
UNKNOWN = 0
MESSAGE = 1
SERVICE = 2


def convert_to_idl(
package_dir, package_name, interface_file, output_dir,
interface_type=InterfaceType.UNKNOWN
):
print('convert_to_idl', interface_file)
# TODO use plugin approach rather than hard coded alternatives
if interface_file.suffix == '.idl':
assert interface_type != InterfaceType.UNKNOWN, \
'.idl files must be passed explicitly as a message or service file'
# just pass through for now
# TODO support e.g. multiple entities in a single input file?
return (interface_type, interface_file)

if interface_file.suffix == '.msg':
assert interface_type in (InterfaceType.UNKNOWN, InterfaceType.MESSAGE)
from rosidl_adapter.msg import convert_msg_to_idl
return (InterfaceType.MESSAGE, convert_msg_to_idl(
package_dir, package_name, interface_file,
output_dir / package_name / 'msg'))

if interface_file.suffix == '.srv':
assert interface_type in (InterfaceType.UNKNOWN, InterfaceType.SERVICE)
from rosidl_adapter.srv import convert_srv_to_idl
return (InterfaceType.SERVICE, convert_srv_to_idl(
package_dir, package_name, interface_file,
output_dir / package_name / 'srv'))

# unknown / unhandled for now
return (InterfaceType.UNKNOWN, interface_file)
19 changes: 19 additions & 0 deletions rosidl_adapter/rosidl_adapter/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 2018 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys

from rosidl_adapter.main import main

sys.exit(main())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be an entry point to work on windows?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't since it is invoked with python -m rosidl_adapter.

83 changes: 83 additions & 0 deletions rosidl_adapter/rosidl_adapter/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Copyright 2018 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import argparse
import os
import pathlib
import sys


from rosidl_adapter import convert_to_idl
from rosidl_adapter import InterfaceType


def main(argv=sys.argv[1:]):
parser = argparse.ArgumentParser(
description='Convert interface files to .idl')
parser.add_argument(
'--package-dir', required=True,
help='The base directory of the package')
parser.add_argument(
'--package-name', required=True,
help='The name of the package')
parser.add_argument(
'--message-files', nargs='*',
help='The message files to convert to .idl')
parser.add_argument(
'--service-files', nargs='*',
help='The service files to convert to .idl')
parser.add_argument(
'--interface-files', nargs='*',
help='The interface files to convert to .idl')
parser.add_argument(
'--output-dir', required=True,
help='The base directory to create .idl files in')
parser.add_argument(
'--output-messages-file', required=True,
help='The output file containing the absolute paths to the message '
'.idl files')
parser.add_argument(
'--output-services-file', required=True,
help='The output file containing the absolute paths to the service '
'.idl files')
args = parser.parse_args(argv)
output_dir = pathlib.Path(args.output_dir)

message_files = []
service_files = []
for interface_file in args.interface_files:
interface_file = pathlib.Path(interface_file)

interface_type, idl_file = convert_to_idl(
args.package_dir, args.package_name, interface_file,
output_dir)
if interface_type == InterfaceType.MESSAGE:
message_files.append(idl_file)
elif interface_type == InterfaceType.SERVICE:
service_files.append(idl_file)
elif interface_type == InterfaceType.UNKNOWN:
raise RuntimeError(
'Unsupported interface file format: {interface_file}'
.format_map(locals()))
else:
assert False

os.makedirs(os.path.dirname(args.output_messages_file), exist_ok=True)
with open(args.output_messages_file, 'w') as h:
for message_file in message_files:
h.write('{message_file}\n'.format_map(locals()))
os.makedirs(os.path.dirname(args.output_services_file), exist_ok=True)
with open(args.output_services_file, 'w') as h:
for service_file in service_files:
h.write('{service_file}\n'.format_map(locals()))
81 changes: 81 additions & 0 deletions rosidl_adapter/rosidl_adapter/msg/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Copyright 2018 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from rosidl_adapter.msg.parser import parse_message_string
from rosidl_adapter.resource import expand_template


def convert_msg_to_idl(package_dir, package_name, input_file, output_dir):
assert input_file.suffix == '.msg'

print('Reading input file: {input_file}'.format_map(locals()))
content = input_file.read_text(encoding='utf-8')
msg = parse_message_string(package_name, input_file.stem, content)

output_file = output_dir / input_file.with_suffix('.idl').name
print('Writing output file: {output_file}'.format_map(locals()))
data = {
'pkg_name': package_name,
'relative_input_file': input_file.absolute().relative_to(package_dir),
'msg': msg,
'get_idl_type': get_idl_type,
'get_include_file': get_include_file,
}
expand_template('msg.idl.em', data, output_file)
return output_file


MSG_TYPE_TO_IDL = {
'bool': 'boolean',
'byte': 'octet',
'char': 'char',
'int8': 'int8',
'uint8': 'uint8',
'int16': 'int16',
'uint16': 'uint16',
'int32': 'int32',
'uint32': 'uint32',
'int64': 'int64',
'uint64': 'uint64',
'float32': 'float',
'float64': 'double',
'string': 'string',
}


def get_include_file(base_type):
if base_type.is_primitive_type():
return None
return '{base_type.pkg_name}/msg/{base_type.type}.idl'.format_map(locals())


def get_idl_type(type_):
if isinstance(type_, str):
identifier = MSG_TYPE_TO_IDL[type_]
elif type_.is_primitive_type():
identifier = MSG_TYPE_TO_IDL[type_.type]
else:
identifier = '{type_.pkg_name}::msg::{type_.type}' \
.format_map(locals())

if isinstance(type_, str) or not type_.is_array:
return identifier

if type_.is_fixed_size_array():
return '{identifier}[{type_.array_size}]'.format_map(locals())

if not type_.is_upper_bound:
return 'sequence<{identifier}>'.format_map(locals())

return 'sequence<{identifier}, {type_.array_size}>'.format_map(locals())
50 changes: 50 additions & 0 deletions rosidl_adapter/rosidl_adapter/msg/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Copyright 2018 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import argparse
import pathlib
import sys

from catkin_pkg.package import package_exists_at
from catkin_pkg.package import parse_package

from rosidl_adapter.msg import convert_msg_to_idl


def main(argv=sys.argv[1:]):
parser = argparse.ArgumentParser(
description='Convert .msg files to .idl')
parser.add_argument(
'interface_files', nargs='+',
help='The interface files to convert')
args = parser.parse_args(argv)

for interface_file in args.interface_files:
interface_file = pathlib.Path(interface_file)
package_dir = interface_file.parent.absolute()
while (
len(package_dir.parents) and
not package_exists_at(str(package_dir))
):
package_dir = package_dir.parent
if not package_dir.parents:
print(
"Could not find package for '{interface_file}'"
.format_map(locals()), file=sys.stderr)
continue
warnings = []
pkg = parse_package(package_dir, warnings=warnings)

convert_msg_to_idl(
package_dir, pkg.name, interface_file, interface_file.parent)
Loading