Incrementally generate raptor transfers from constrained transfers for new pattern

[slvlirnoff]

Summary

This aims at incrementally updating the generated raptor transfers for the transit layer from the constrained transfers. It keeps track of the last RoutingPattern index and only generate transfers that are alighting or boarding newly created patterns.

Issue

Closes #6190

[codecov]

Codecov Report

Attention: Patch coverage is 65.71429% with 12 lines in your changes missing coverage. Please review.

Project coverage is 69.90%. Comparing base (cee960f) to head (dd394a5).

Files with missing lines	Patch %	Lines
...it/constrainedtransfer/TransferIndexGenerator.java	65.71%	8 Missing and 4 partials ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             dev-2.x    #6191      +/-   ##
=============================================
- Coverage      69.90%   69.90%   -0.01%     
- Complexity     17723    17725       +2     
=============================================
  Files           1998     1998              
  Lines          75443    75468      +25     
  Branches        7718     7727       +9     
=============================================
+ Hits           52740    52754      +14     
- Misses         20025    20031       +6     
- Partials        2678     2683       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[slvlirnoff]

In our deployment (40k pattern, 60k constrained transfer) updating the transit layer goes from systematically ~6s to ~10-20ms in average and a up to a few hundreds ms when new pattern are created.

[slvlirnoff]

@t2gran I've investigated further where the performance issue comes from for switzerland's setup. Out of the 50k constrained transfers, the majority of them are trip to trip and aren't causing an issue but there is around 8'000 stations to stations transfers (for minimum transfer time) that are creating 40'000'000 TransferForPattern object.

[t2gran]

@t2gran I've investigated further where the performance issue comes from for switzerland's setup. Out of the 50k constrained transfers, the majority of them are trip to trip and aren't causing an issue but there is around 8'000 stations to stations transfers (for minimum transfer time) that are creating 40'000'000 TransferForPattern object.

I am not sure if you are allowed to do it or not, but to me a sensible thing would be to specify a minTransferTime per stop (on the target stop). If there are additional constraints, like only apply the minWaitTime to longDistance rail, we could add that. If you want I would like to discuss this in a meeting. I think the GTFS specification is impossible to implement in a scalable way, and that we should NOT do it. Instead we should come up with some features that cover the needs of the industry. We should probably also inform the GTFS community what we think and what we intend to do.

Is there a time witch we can discuss this?

[leonardehrenfried]

My experience is that often legacy systems required explicit transfers to be defined for them to work at all. When these data sets are being converted to GTFS they are mapped to minimum_transfer_times when in fact it just means that a transfer is possible. In GTFS and OTP it's not necessary to explicitly say that the transfer is possible and the better option is to micromap your stations so that you get good, flexible walking instructions with detailed paths.

I often completely drop the minimum_transfer_times from my German data sets. I suspect the source of @slvlirnoff's data is a German software vendor.

[slvlirnoff]

Yep, unfortunately we currently have a need to use this information. In many cases it is difficult to micromap the station level for several reasons, the main one is that we aren't always in contact with the producer/owner of the specific station data.

They also use these constraints to sometimes prevent/discourage short transfer between specific stops, for instance there are nearby bus stops up the mountains were you could theoretically switch bus in under a minute, however if your bus is late or the other bus is running early you'll be stuck in the middle of nowhere for an hour and it'd be safer to consider only transfers with 10mn slack.

In other cases the train platforms are very long and you could reach them from multiple connecting bus stops relatively quickly, to the front, the middle or respectively the back of the platform. This is harder to map and historically done with these transfer time. Would you map these with pathways?

In any case, there are two issues:

• if you create a constrained transfer between stops it'll have a significant performance impact that is somehow hidden to the operator of OTP (no logs, etc.). OTP displays the number of constrained transfers however it doesn't display the number of computed raptor transfers and here it generates 40Mio objects from 8k constraints. This practice should be potentially discouraged in the documentation.
• The raptor transfers are recomputed for each transit layer update cycle, this PR tries to address this.