Skip to content

feat: use rapidjson for osrm_route serializer#5462

Open
MohamedM216 wants to merge 19 commits intovalhalla:masterfrom
MohamedM216:osrm-routeapi-to-rapidjson
Open

feat: use rapidjson for osrm_route serializer#5462
MohamedM216 wants to merge 19 commits intovalhalla:masterfrom
MohamedM216:osrm-routeapi-to-rapidjson

Conversation

@MohamedM216
Copy link
Contributor

@MohamedM216 MohamedM216 commented Aug 18, 2025

Issue

This PR addresses issue #5190 by migrating the osrm_route API serializer from the legacy json:: implementation to rapidjson for improved performance and consistency.

This PR should close this issue

Tasklist

  • Add tests
  • Add #fixes with the issue number that this PR addresses
  • Update the docs with any new request parameters or changes to behavior described
  • Update the changelog
  • If you made changes to the lua files, update the taginfo too
  • If you made changes to a translation file, update transifex too

Requirements / Relations

#5190

@MohamedM216
Copy link
Contributor Author

MohamedM216 commented Aug 18, 2025

After merging #5429 we need to remove this function in this PR

baldr::json::ArrayPtr serializeWarnings(const valhalla::Api& api);

EDIT: Done

Comment on lines 1853 to 1859
(prev_maneuver->type() == DirectionsLeg_Maneuver_Type_kRoundaboutEnter) &&
!prev_step_json.empty()) {
rapidjson::writer_wrapper_t prev_step_writer;
prev_step_writer(prev_step_json);
prev_step_writer("destinations", dest);
prev_step_json = prev_step_writer.get_buffer();
}
Copy link
Contributor Author

@MohamedM216 MohamedM216 Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests are failing due to wrong serialization here and in the next 2 ifs. The reason behind that is writing prev_step_json with the writer directly which means writing it as a string instead of proper json elements. I will work on finding a way to fix this. Probably using rapidjson::Document might work.

Comment on lines +325 to +331
inline void raw_json(const std::string& json_string) {
writer.Flush();
for (char c : json_string) {
buffer.Put(c);
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these tests were failing due to writing prev_step_json as a string instead of proper json elements.

test_zero_length_route.cc
test_via_waypoints.cc
test_use.cc
test_use_direction_on_ways.cc
test_turn_lanes.cc
test_turns.cc
test_traffic.cc
test_tagged_values.cc
test_simple_restrictions.cc
test_search_filter.cc
test_route.cc
test_restricted_area.cc
test_recost.cc
test_osrm_serializer.cc
test_node_type.cc
test_multi_layer_loki.cc
test_maxspeed.cc
test_instructions_uturn.cc
test_instructions_collapse_double_turn.cc
test_conditional_restrictions.cc
test_admin.cc
test_route_summary.cc
test_access.cc

Now after adding this function raw_json, writer can write any valid json string as proper json elements.

!prev_step_json.empty()) {
std::string inner_json = prev_step_json;
if (inner_json.front() == '{' && inner_json.back() == '}') {
inner_json = inner_json.substr(1, inner_json.size() - 2);
Copy link
Contributor Author

@MohamedM216 MohamedM216 Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raw_json works correctly only for embedding inside an object or array, or it will throw !has_root exception coming fromprefix function. The reason behind that is writer expects only one root. So I removed the step object notations {} and then created a new one to be able to write inner_json and any other content.

@nilsnolde nilsnolde self-requested a review September 15, 2025 14:29
@MohamedM216
Copy link
Contributor Author

MohamedM216 commented Sep 16, 2025

Debug build was failing because of the unused variable is_rest_stop_closed (I forgot to remove it after the last update 🙂). That should be fixed now.

As for the Release build failure, it was caused by a missed refactor in route_serializer_osrm.cc:

#ifdef INLINE_TEST

I will start refactoring that part now.

EDIT:

Errors are coming from these lines:
auto intersection_doc = json::Jmap();
auto intersection_doc = json::Jmap();
auto intersection_doc = json::Jmap();
json::MapPtr annotations = serialize_annotations(leg);
ss << *annotations;
auto annotations = serialize_annotations(leg);
auto annotations = serialize_annotations(leg);
json::ArrayPtr indications_1 = true, kTurnLaneReverse | kTurnLaneSharpLeft);
json::ArrayPtr indications_2 =

ASSERT_EQ(indications_1->size(), 2);
  ASSERT_STREQ(boost::get<std::string>(indications_1->at(0)).c_str(), "uturn");
  ASSERT_STREQ(boost::get<std::string>(indications_1->at(1)).c_str(), "sharp left");

  ASSERT_EQ(indications_2->size(), 3);
  ASSERT_STREQ(boost::get<std::string>(indications_2->at(0)).c_str(), "straight");
  ASSERT_STREQ(boost::get<std::string>(indications_2->at(1)).c_str(), "right");
  ASSERT_STREQ(boost::get<std::string>(indications_2->at(2)).c_str(), "sharp right");

I think that there are only 2 reasons behind those errors:

  • json header is missing
  • using the legacy json version of the serializing functions instead of the rapidjson version

Before I dive into the work, what changes do you suggest @nilsnolde? Should I restore the json header again for this file and rewrite the current tests to be able use the rapidjson version?

@SiarheiFedartsou
Copy link
Member

@nilsnolde should we may be merge it? WDYT?

@nilsnolde
Copy link
Member

Last time I looked at it (a while ago admittedly), there was some strange json string parsing. Wanted to have a look if that can't be done a bit less hacky.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use rapidjson in serializers

3 participants