Skip to content

Commit

Permalink
Split converters (ros2#70)
Browse files Browse the repository at this point in the history
* ros2GH-134 Split converter interface into Serializer and Deserializer

- Allow plugins which can only read or write
- Most important example: plugin for old rosbags

* ros2GH-134 Switch to using serializer and deserializer in factory

* ros2GH-134 Add test for serializer plugin

* ros2GH-134 Try to load Serializer and Deserializer

- When loading a serializer, try to load both serializer and converter
- Similar for deserializers

* ros2GH-134 Fix e2e test after improving error message for missing converters

* ros2GH-134 Remove duplicate code in converter factory

* ros2GH-134 Change namespace of converter interfaces

- adapt namespaces to folder structure
- folder structure similar to rosbag2_storage

* ros2GH-134 Hide pluginlib import via pimpl

- We want to use template functions that require the pluginlib import
- The pluginlib import should not be exported (this creates issues with
  downstream packages)
- Similar to the storage factory, use a pimpl

* ros2GH-134 Adapt documentation

* Minor documentation updates

Co-Authored-By: Martin-Idel-SI <external.Martin.Idel@bosch-si.com>

* ros2GH-134 Rename converter interface to drop "interface"

- already visible from namespace
  • Loading branch information
Martin-Idel-SI authored and Karsten1987 committed Dec 6, 2018
1 parent 366d750 commit ffbca3e
Show file tree
Hide file tree
Showing 28 changed files with 424 additions and 115 deletions.
19 changes: 15 additions & 4 deletions docs/converter_plugin_development.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ To simplify conversion between all formats each plugin provides functions to con

Rosbag2 is shipped with a default converter plugin to convert between ROS 2 messages and serialized messages in the CDR format (this is the general serialization format specified by DDS).

## Writing converter plugins
## Writing converter plugins for both serialization and deserialization

To write a plugin `MyConverter`, implement the interface `rosbag2::SerializationFormatConverterInterface`.
Most converter plugins are used to convert between a custom serialization format in both directions, i.e. serialization and deserialization.
To write a plugin `MyConverter` supporting conversion to and from a custom serialization format, implement the interface `rosbag2::converter_interfaces::SerializationFormatConverter`.

The plugin interface provides two functions:

Expand All @@ -22,7 +23,7 @@ Add the following lines in the `my_converter.cpp`:

```
#include "pluginlib/class_list_macros.hpp"
PLUGINLIB_EXPORT_CLASS(MyConverter, rosbag2::SerializationFormatConverterInterface)
PLUGINLIB_EXPORT_CLASS(MyConverter, rosbag2::converter_interfaces::SerializationFormatConverter)
```

Furthermore, we need some meta-information in the form of a `plugin_description.xml` file.
Expand All @@ -32,7 +33,7 @@ In the case of `MyConverter` this would look like:
<class
name="my_storage_format_converter"
type="MyConverter"
base_class_type="rosbag2::SerializationFormatConverterInterface"
base_class_type="rosbag2::converter_interfaces::SerializationFormatConverter"
>
<description>This is a converter plugin for my storage format.</description>
</class>
Expand All @@ -48,6 +49,16 @@ pluginlib_export_plugin_description_file(rosbag2 plugin_description.xml)

The first argument `rosbag2` denotes the ament index key we add our plugin to (this will always be `rosbag2` for converter plugins), while the second argument is the path to the plugin description file.

## Writing converter plugins for only serialization or deserialization

It is possible to write a plugin which only supports converting to OR from a custom serialization format.
For example, the default plugin for legacy ROS1 rosbags supports only conversion from a ROS 1 to a ROS 2 message format (deserialization), since rosbag2 is not intended to be used to write rosbags in the old format.
When a similar use case applies, it is possible to provide a plugin which supports only one direction (serialization or deserialization).

In order to write a plugin `MyDeserializer` supporting conversion from a custom serialization format, implement the interface `rosbag2::converter_interfaces::SerializationFormatDeserializer.
This interface only provides the `deserialize` method mentioned above.
Similarly, there exists a `rosbag2::converter_interfaces::SerializationFormatSerializer` for serialization plugins.

## How to choose conversion at runtime

The conversion will be chosen automatically according to the storage format specified in the bagfile or specified by the user.
Expand Down
7 changes: 4 additions & 3 deletions rosbag2/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -82,12 +82,13 @@ if(BUILD_TESTING)
ament_lint_auto_find_test_dependencies()

add_library(
converter_test_plugin
converter_test_plugins
SHARED
test/rosbag2/serializer_test_plugin.cpp
test/rosbag2/converter_test_plugin.cpp)
target_link_libraries(converter_test_plugin rosbag2)
target_link_libraries(converter_test_plugins rosbag2)
install(
TARGETS converter_test_plugin
TARGETS converter_test_plugins
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin)
Expand Down
6 changes: 3 additions & 3 deletions rosbag2/include/rosbag2/converter.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
#include "rosbag2/converter_options.hpp"
#include "rosbag2/serialization_format_converter_factory.hpp"
#include "rosbag2/serialization_format_converter_factory_interface.hpp"
#include "rosbag2/serialization_format_converter_interface.hpp"
#include "rosbag2/converter_interfaces/serialization_format_converter.hpp"
#include "rosbag2/types.hpp"
#include "rosbag2/visibility_control.hpp"

Expand Down Expand Up @@ -78,8 +78,8 @@ class ROSBAG2_PUBLIC Converter

private:
std::shared_ptr<SerializationFormatConverterFactoryInterface> converter_factory_;
std::unique_ptr<SerializationFormatConverterInterface> input_converter_;
std::unique_ptr<SerializationFormatConverterInterface> output_converter_;
std::unique_ptr<converter_interfaces::SerializationFormatDeserializer> input_converter_;
std::unique_ptr<converter_interfaces::SerializationFormatSerializer> output_converter_;
std::unordered_map<std::string, ConverterTypeSupport> topics_and_types_;
};

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// Copyright 2018, Bosch Software Innovations GmbH.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_
#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_

#include <memory>
#include <string>

#include "rosbag2/types/introspection_message.hpp"
#include "rosbag2/types.hpp"
#include "rcutils/types.h"
#include "rosbag2_storage/serialized_bag_message.hpp"
#include "rosidl_typesupport_cpp/message_type_support.hpp"
#include "rosbag2/converter_interfaces/serialization_format_serializer.hpp"
#include "rosbag2/converter_interfaces/serialization_format_deserializer.hpp"

/**
* This is a convenience class for plugin developers. When developing a plugin to both write and
* read a specified serialization format, inherit from this class
*/
namespace rosbag2
{

namespace converter_interfaces
{

class SerializationFormatConverter
: public SerializationFormatSerializer, public SerializationFormatDeserializer
{};

} // namespace converter_interfaces
} // namespace rosbag2

#endif // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_CONVERTER_HPP_
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,33 @@
// See the License for the specific language governing permissions and
// limitations under the License.

#ifndef ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
#define ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_
#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_

#include <memory>
#include <string>

#include "rosbag2/types/introspection_message.hpp"
#include "rosbag2/types.hpp"
#include "rcutils/types.h"
#include "rosbag2_storage/serialized_bag_message.hpp"
#include "rosidl_typesupport_cpp/message_type_support.hpp"

namespace rosbag2
{

class SerializationFormatConverterInterface
namespace converter_interfaces
{

class SerializationFormatDeserializer
{
public:
virtual ~SerializationFormatConverterInterface() = default;
virtual ~SerializationFormatDeserializer() = default;

virtual void deserialize(
std::shared_ptr<const rosbag2::SerializedBagMessage> serialized_message,
const rosidl_message_type_support_t * type_support,
std::shared_ptr<rosbag2_introspection_message_t> ros_message) = 0;

virtual void serialize(
std::shared_ptr<const rosbag2_introspection_message_t> ros_message,
const rosidl_message_type_support_t * type_support,
std::shared_ptr<rosbag2::SerializedBagMessage> serialized_message) = 0;
};


} // namespace converter_interfaces
} // namespace rosbag2

#endif // ROSBAG2__SERIALIZATION_FORMAT_CONVERTER_INTERFACE_HPP_
#endif // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_DESERIALIZER_HPP_
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Copyright 2018, Bosch Software Innovations GmbH.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#ifndef ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_
#define ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_

#include <memory>

#include "rosbag2/types/introspection_message.hpp"
#include "rosbag2/types.hpp"

namespace rosbag2
{

namespace converter_interfaces
{

class SerializationFormatSerializer
{
public:
virtual ~SerializationFormatSerializer() = default;

virtual void serialize(
std::shared_ptr<const rosbag2_introspection_message_t> ros_message,
const rosidl_message_type_support_t * type_support,
std::shared_ptr<rosbag2::SerializedBagMessage> serialized_message) = 0;
};

} // namespace converter_interfaces
} // namespace rosbag2

#endif // ROSBAG2__CONVERTER_INTERFACES__SERIALIZATION_FORMAT_SERIALIZER_HPP_
21 changes: 9 additions & 12 deletions rosbag2/include/rosbag2/serialization_format_converter_factory.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@

#include <memory>
#include <string>
#include <vector>

#include "rosbag2/serialization_format_converter_interface.hpp"
#include "rosbag2/visibility_control.hpp"

// This is necessary because of using stl types here. It is completely safe, because
Expand All @@ -31,17 +31,11 @@
# pragma warning(disable:4251)
#endif

namespace pluginlib
{

template<class T>
class ClassLoader;

} // namespace pluginlib

namespace rosbag2
{

class SerializationFormatConverterFactoryImpl;

class ROSBAG2_PUBLIC SerializationFormatConverterFactory
: public SerializationFormatConverterFactoryInterface
{
Expand All @@ -50,11 +44,14 @@ class ROSBAG2_PUBLIC SerializationFormatConverterFactory

~SerializationFormatConverterFactory() override;

std::unique_ptr<SerializationFormatConverterInterface>
load_converter(const std::string & format) override;
std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
load_deserializer(const std::string & format) override;

std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
load_serializer(const std::string & format) override;

private:
std::unique_ptr<pluginlib::ClassLoader<SerializationFormatConverterInterface>> class_loader_;
std::unique_ptr<SerializationFormatConverterFactoryImpl> impl_;
};

} // namespace rosbag2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
#include <memory>
#include <string>

#include "rosbag2/serialization_format_converter_interface.hpp"
#include "rosbag2/converter_interfaces/serialization_format_converter.hpp"
#include "rosbag2/visibility_control.hpp"

namespace rosbag2
Expand All @@ -29,8 +29,11 @@ class ROSBAG2_PUBLIC SerializationFormatConverterFactoryInterface
public:
virtual ~SerializationFormatConverterFactoryInterface() = default;

virtual std::unique_ptr<SerializationFormatConverterInterface>
load_converter(const std::string & format) = 0;
virtual std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
load_deserializer(const std::string & format) = 0;

virtual std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
load_serializer(const std::string & format) = 0;
};

} // namespace rosbag2
Expand Down
4 changes: 2 additions & 2 deletions rosbag2/src/rosbag2/converter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ Converter::Converter(
const rosbag2::ConverterOptions & converter_options,
std::shared_ptr<rosbag2::SerializationFormatConverterFactoryInterface> converter_factory)
: converter_factory_(converter_factory),
input_converter_(converter_factory_->load_converter(
input_converter_(converter_factory_->load_deserializer(
converter_options.input_serialization_format)),
output_converter_(converter_factory_->load_converter(
output_converter_(converter_factory_->load_serializer(
converter_options.output_serialization_format))
{
if (!input_converter_) {
Expand Down
42 changes: 13 additions & 29 deletions rosbag2/src/rosbag2/serialization_format_converter_factory.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -16,45 +16,29 @@

#include <memory>
#include <string>
#include <vector>

#include "pluginlib/class_loader.hpp"
#include "rosbag2/logging.hpp"
#include "./serialization_format_converter_factory_impl.hpp"

namespace rosbag2
{

SerializationFormatConverterFactory::SerializationFormatConverterFactory()
{
try {
class_loader_ = std::make_unique<pluginlib::ClassLoader<SerializationFormatConverterInterface>>(
"rosbag2", "rosbag2::SerializationFormatConverterInterface");
} catch (const std::exception & e) {
ROSBAG2_LOG_ERROR_STREAM("Unable to create class loader instance: " << e.what());
throw e;
}
}
: impl_(std::make_unique<SerializationFormatConverterFactoryImpl>())
{}

SerializationFormatConverterFactory::~SerializationFormatConverterFactory() = default;

std::unique_ptr<SerializationFormatConverterInterface>
SerializationFormatConverterFactory::load_converter(const std::string & format)
std::unique_ptr<converter_interfaces::SerializationFormatDeserializer>
SerializationFormatConverterFactory::load_deserializer(const std::string & format)
{
return impl_->load_deserializer(format);
}

std::unique_ptr<converter_interfaces::SerializationFormatSerializer>
SerializationFormatConverterFactory::load_serializer(const std::string & format)
{
auto converter_id = format + "_converter";

const auto & registered_classes = class_loader_->getDeclaredClasses();
auto class_exists = std::find(registered_classes.begin(), registered_classes.end(), converter_id);
if (class_exists == registered_classes.end()) {
ROSBAG2_LOG_ERROR_STREAM("Requested converter id '" << converter_id << "' does not exist");
return nullptr;
}

try {
return std::unique_ptr<SerializationFormatConverterInterface>(
class_loader_->createUnmanagedInstance(converter_id));
} catch (const std::runtime_error & ex) {
ROSBAG2_LOG_ERROR_STREAM("Unable to load instance of converter interface: " << ex.what());
return nullptr;
}
return impl_->load_serializer(format);
}

} // namespace rosbag2
Loading

0 comments on commit ffbca3e

Please sign in to comment.