Skip to content

Commit

Permalink
Introduce rosbag2_transport layer and CLI (ros2#38)
Browse files Browse the repository at this point in the history
* rosbag2_transport package with python interface

* use cpp for python extension

* use rosbag2_transport cpp API

* use rosbag2_transport API in cli

* linters

* ros2GH-25 Rename target librosbag2 to rosbag2

CMake already prepends libraries with `lib`, so the old name resulted
in `liblibrosbag2`

* ros2GH-21 Initial call of rosbag2.record() from rosbag2_transport

* ros2GH-21 Add missing copyright header

* ros2GH-21 Cleanup clang tidy issues

* ros2GH-21 Remove rclcpp dependency from rosbag2

* ros2GH-21 Wire rosbag play into CLI

* ros2GH-21 Add missing test_depend in rosbag2_transport package.xml

* ros2GH-21 Unify name of python import

* ros2GH-21 Enable -a in CLI, show help on wrong args

* ros2GH-85 Introduce topic and type struct for readability

* ros2GH-85 Do not export sqlite3 as dependency from default plugins

- not referenced in header, therefore unnecessary

* ros2GH-85 Move rosbag2 except typesupport to rosbag2_transport

* ros2GH-85 Add rosbag2 wrapper

* ros2GH-85 Change signature of create_topic to take TopicWithType

* ros2GH-85 Use rosbag2 in rosbag2_transport

- Don't link against rosbag2_storage anymore

* ros2GH-84 Cleanup package.xmls and CMakeLists everywhere

* ros2GH-21 Add missing init() and shutdown() in record

* ros2GH-85 Fix Windows build

* ros2GH-85 Add visibility control to rosbag2

* ros2GH-85 Cleanup and documentation

* ros2GH-87 Add test package rosbag2_tests

* ros2GH-87 [WIP] Add first working prototype of an end-to-end test

* ros2GH-87 Use test_msgs instead of std_msgs/String in end-to-end test

* ros2GH-87 Use SIGTERM instead of SIGKILL and refactor test

* ros2GH-87 Make end-to-end test work on Windows

* ros2GH-87 Fix uncrustify

* ros2GH-87 Refactor end-to-end test fixture

* ros2GH-21 Extend transport python module interface

The python interface should accept all options that can be passed to rosbag2_transport

* ros2GH-87 Fix test fixture for Windows

* ros2GH-87 Refactor test fixture

* ros2GH-87 Separate record from play end-to-end test

* ros2GH-87 Make record end-to-end test work

* ros2GH-87 Publish before recording to create topic

* ros2GH-87 Fix record all on Windows

* ros2GH-87 Check for topics instead of all

* ros2GH-87 Wait until rosbag record opened database

* ros2GH-87 Delete directory recursively

* ros2GH-87 Delete directories recursively on Linux

* ros2GH-87 Reset ROS_DOMAIN_ID to protect against concurrent tests

* ros2GH-89 Make rosbag2 interfaces virtual and add explicit open() method

This allows downstream packages (e.g. rosbag2_transport) to mock these
interfaces in tests.

* ros2GH-87 Improve test and refactoring

* ros2GH-87 Minor refactoring to increase test readability

* ros2GH-87 Fix environmental variable behaviour on Mac

* ros2GH-87 Fix Windows build

* ros2GH-89 Use mock reader and writer in rosbag2_transport tests

* ros2GH-87 Add play end_to_end test

* ros2GH-87 Improvements of test

* ros2GH-87 Fix Windows build

* ros2GH-89 Cleanup: small documentation fixes.

* ros2GH-89 [WIP] Test if Writer and Reader work with class visibility

* ros2GH-87 Stabilize rosbag2_play test

* ros2GH-87 Minor refactoring of tests

* ros2GH-87 Rename end to end tests

* add license agreement

* ros2GH-89 Simplification of writing to in-memory storage

* ros2GH-89 Stabilize transport tests

* ros2GH-87 Refactoring of tests

- Extract temporary file handling
- Extract subscription management

* ros2GH-87 Add pytest cache to gitignore

* ros2GH-87 Refactoring of play test

- Extract Publisher manager

* ros2GH-87 Extract record test fixture for readability

* ros2GH-89 Refactor transport tests

- Use subscription and publisher manager just as e2e tests
- Use options in recording

* ros2GH-89 Use temporary directory fixture in sqlite tests

* ros2GH-89 Conform to naming standard for tests

* ros2GH-89 Prevent burst publishing of all messages

- Improves test stability

* ros2GH-89 Improve play stability

- Sometimes the first message is lost (discovery)

* ros2GH-25 Fix package.xmls

* Consistently use project name in CMakeLists

* Minor cleanup

- make rosbag2_transport description more expressive
- hide unnecessary methods in typesupport_helpers
- fix incorrect logging in tests
- minor cleanup

* Change name of nodes in rosbag2_transport

* Cleanup folder structure in rosbag2_storage and rosbag2_tests

- use src/<package_name>/ and test/<package_name>/ folders everywhere
- harmonises with all other packages
- results in better header guards

* Export sqlite3 dependency as package dependency

* Create node in Rosbag2Transport always

* Only hold one node in rosbag2_transport

* Move all duplicate files to common package

* Adapt namespacing in test commons package

- use "using namespace" declaratives for tests
- use package name as namespace

* Replace "Waiting for messages..." message

* ros2GH-25 rename rosbag2_test_commons -> rosbag2_test_common

* ros2GH-25 Overwrite already existing test.bag when recording

This is a temporary solution and will be handled properly once a
file path can be passed via the cli.

* ros2GH-25 Cleanups

- Log every subscription
- move all dependencies onside BUILD_TESTING for rosbag2_test_common

* fix cmake typo for test_common

* Remove superfluous loop in rosbag2 transport

* Delete superfluous test_msgs dependency

* Add rclcpp to test dependencies

- Apparently ament_export_dependencies does not work in rosbag2_test_common

* Fix rosbag2 node test

- Clock topic is no longer present on all nodes
- Remove assumptions on foreign ros topics

* Fix dependencies by exporting them explicitly
  • Loading branch information
Martin-Idel-SI authored and Karsten1987 committed Sep 28, 2018
1 parent bc822f1 commit 1370c7f
Show file tree
Hide file tree
Showing 87 changed files with 2,666 additions and 1,232 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ cmake-build-debug/
cmake-build-release/
build/
venv/
**/.pytest_cache/
Empty file removed ros2bag/COLCON_IGNORE
Empty file.
1 change: 1 addition & 0 deletions ros2bag/package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
<license>Apache License 2.0</license>

<depend>ros2cli</depend>
<depend>rosbag2_transport</depend>

<test_depend>ament_copyright</test_depend>
<test_depend>ament_flake8</test_depend>
Expand Down
4 changes: 3 additions & 1 deletion ros2bag/ros2bag/verb/play.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

from ros2bag.verb import VerbExtension

from rosbag2_transport import rosbag2_transport_py


class PlayVerb(VerbExtension):
"""ros2 bag play."""
Expand All @@ -24,4 +26,4 @@ def add_arguments(self, parser, cli_name): # noqa: D102

def main(self, *, args): # noqa: D102
bag_file = args.bag_file
print('calling ros2 bag play on', bag_file)
rosbag2_transport_py.play(uri=bag_file, storage_id='sqlite3')
32 changes: 20 additions & 12 deletions ros2bag/ros2bag/verb/record.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,22 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import sys
import time
import os

from ros2bag.verb import VerbExtension

from ros2cli.node.strategy import add_arguments
from ros2cli.node.strategy import NodeStrategy
from ros2cli.node.strategy import add_arguments

from ros2bag.verb import VerbExtension
from rosbag2_transport import rosbag2_transport_py


class RecordVerb(VerbExtension):
"""ros2 bag record."""

def add_arguments(self, parser, cli_name): # noqa: D102
self._subparser = parser

add_arguments(parser)
parser.add_argument(
'-a', '--all', action='store_true', help='recording all topics')
Expand All @@ -36,11 +39,16 @@ def main(self, *, args): # noqa: D102
print('invalid choice: Can not specify topics and -a at the same time')
return

with NodeStrategy(args) as node:
if args.all:
t_and_n = node.get_topic_names_and_types()
print(t_and_n)
topics = [t for t,n in node.get_topic_names_and_types()]
if args.topics:
topics = args.topics
print('topics to be recorded:', topics)
uri = 'test.bag'
if os.path.exists(uri):
os.remove(uri)
print('warning: Overwriting already existing \'test.bag\'!')

storage_id = 'sqlite3'

if args.all:
rosbag2_transport_py.record(uri=uri, storage_id=storage_id, all=True)
elif args.topics and len(args.topics) > 0:
rosbag2_transport_py.record(uri=uri, storage_id=storage_id, topics=args.topics)
else:
self._subparser.print_help()
116 changes: 15 additions & 101 deletions rosbag2/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,143 +19,57 @@ find_package(ament_cmake REQUIRED)
find_package(ament_index_cpp REQUIRED)
find_package(poco_vendor)
find_package(Poco COMPONENTS Foundation)
find_package(rcl REQUIRED)
find_package(rclcpp REQUIRED)
find_package(rcutils REQUIRED)
find_package(rosbag2_storage REQUIRED)
find_package(rosidl_generator_cpp REQUIRED)
find_package(shared_queues_vendor REQUIRED)

add_library(
librosbag2
SHARED
src/rosbag2/rosbag2.cpp
src/rosbag2/player.cpp
add_library(${PROJECT_NAME} SHARED
src/rosbag2/sequential_reader.cpp
src/rosbag2/typesupport_helpers.cpp
src/rosbag2/generic_publisher.cpp
src/rosbag2/generic_subscription.cpp
src/rosbag2/rosbag2_node.cpp)
src/rosbag2/writer.cpp)

ament_target_dependencies(librosbag2
ament_target_dependencies(${PROJECT_NAME}
ament_index_cpp
Poco
rcl
rclcpp
rcutils
rosbag2_storage
rosidl_generator_c
shared_queues_vendor
rosidl_generator_cpp
)

target_include_directories(librosbag2
target_include_directories(${PROJECT_NAME}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)

# Causes the visibility macros to use dllexport rather than dllimport,
# which is appropriate when building the dll but not consuming it.
target_compile_definitions(librosbag2 PRIVATE "ROSBAG2_BUILDING_DLL")

add_executable(${PROJECT_NAME}_record src/rosbag2/demo_record.cpp)
target_link_libraries(${PROJECT_NAME}_record librosbag2)

add_executable(${PROJECT_NAME}_play src/rosbag2/demo_play.cpp)
target_link_libraries(${PROJECT_NAME}_play librosbag2)
target_compile_definitions(${PROJECT_NAME} PRIVATE "ROSBAG2_BUILDING_DLL")

install(
DIRECTORY include/
DESTINATION include)

install(
TARGETS librosbag2
TARGETS ${PROJECT_NAME}
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib)
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin)

ament_export_include_directories(include)
ament_export_libraries(${PROJECT_NAME})
ament_export_dependencies(rosbag2_storage)

if(BUILD_TESTING)
find_package(ament_cmake_gmock REQUIRED)
find_package(ament_lint_auto REQUIRED)
find_package(rosbag2_storage_default_plugins REQUIRED)
find_package(test_msgs REQUIRED)
ament_lint_auto_find_test_dependencies()

ament_add_gmock(rosbag2_record_integration_test
test/rosbag2/rosbag2_record_integration_test.cpp
test/rosbag2/test_memory_management.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_record_integration_test)
target_link_libraries(rosbag2_record_integration_test librosbag2
${test_msgs_LIBRARIES}
${rosbag2_storage_default_plugins_LIBRARIES})
target_include_directories(rosbag2_record_integration_test PRIVATE
${test_msgs_INCLUDE_DIRS}
${rosbag2_storage_default_plugins_INCLUDE_DIRS})
endif()

ament_add_gmock(rosbag2_record_all_integration_test
test/rosbag2/rosbag2_record_all_integration_test.cpp
test/rosbag2/test_memory_management.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_record_all_integration_test)
target_link_libraries(rosbag2_record_all_integration_test librosbag2
${test_msgs_LIBRARIES}
${rosbag2_storage_default_plugins_LIBRARIES})
target_include_directories(rosbag2_record_all_integration_test PRIVATE
${test_msgs_INCLUDE_DIRS}
${rosbag2_storage_default_plugins_INCLUDE_DIRS})
endif()

ament_add_gmock(rosbag2_play_integration_test
test/rosbag2/rosbag2_play_integration_test.cpp
test/rosbag2/test_memory_management.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_play_integration_test)
target_link_libraries(rosbag2_play_integration_test librosbag2)
ament_target_dependencies(rosbag2_play_integration_test test_msgs)
endif()

ament_add_gmock(rosbag2_play_timing_integration_test
test/rosbag2/rosbag2_play_timing_integration_test.cpp
test/rosbag2/test_memory_management.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_play_timing_integration_test)
target_link_libraries(rosbag2_play_timing_integration_test librosbag2)
ament_target_dependencies(rosbag2_play_timing_integration_test test_msgs)
endif()

ament_add_gmock(rosbag2_typesupport_helpers_test
test/rosbag2/rosbag2_typesupport_helpers_test.cpp
src/rosbag2/typesupport_helpers.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_typesupport_helpers_test)
ament_target_dependencies(rosbag2_typesupport_helpers_test
ament_index_cpp
Poco
rcl
rosidl_generator_cpp
)
endif()

ament_add_gmock(rosbag2_rosbag_node_test
test/rosbag2/rosbag2_rosbag_node_test.cpp
test/rosbag2/test_memory_management.cpp
src/rosbag2/generic_publisher.cpp
src/rosbag2/generic_subscription.cpp
src/rosbag2/rosbag2_node.cpp
src/rosbag2/typesupport_helpers.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
if(TARGET rosbag2_rosbag_node_test)
target_include_directories(rosbag2_rosbag_node_test
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)
ament_target_dependencies(rosbag2_rosbag_node_test
ament_index_cpp
Poco
rclcpp
test_msgs
)
target_link_libraries(rosbag2_typesupport_helpers_test rosbag2)
endif()
endif()

Expand Down
88 changes: 0 additions & 88 deletions rosbag2/include/rosbag2/rosbag2.hpp

This file was deleted.

Loading

0 comments on commit 1370c7f

Please sign in to comment.