Multi fetch for objects histories

[deevroman]

Problem

Currently, there is no way to request the history of multiple objects in a single request.

Description

Let's say you want to get the full history of changes to the line geometry. To do this, you need to get the history of the line + the history of all points that have ever entered the line. There seem to be only two ways to do this right now:

1. Call in the worst case 2000 /history for nodes (in one of the line versions)
2. Make a Multi fetch of the current state of points, and then generate queries like: nodes?nodes=1v1,1v2,...2v1,....

In theory, if the nodes versions are small (1-5), then loading with Multi fetch (~600 versions at a time), the second method will only need 4-17 requests

In practice, most likely you will not be able to get a boost due to #5005

---

But why all these difficulties? Why not have multi fetch for /history?

Something like: /nodes_histories?nodes=1,2,3

p.s. judging by, most likely, this method would also be useful for undo edits tools

Screenshots

No response

[tomhughes]

Why would an editor want to get the history for lots of objects?

[deevroman]

Probably to check for conflicts. For example, try reverting 141361687 in JOSM and you will see that the last step it tries is to download the history of all thousands of objects

[mmd-osm]

get the full history of changes to the line geometry

At least this requirement looks like a duplicate of #130

[deevroman]

At least this requirement looks like a duplicate of

This particular example. But this issue is also true for relations. For example, in route relations that often have many node members.

I also have a suspicion that /:id/full/history will sometimes run into a limit on the transmitted data. Especially if you want to implement relation/:id/full/history

[mmd-osm]

#130 only talks about way/:id/full/history. It doesn't really make a lot of sense to extend this idea to relation/:id/full/history like you've also noted, and that was not discussed there.

I'm also wondering if it makes sense to request the full history in all cases. Maybe returning the last n versions would be more than sufficient.

[gravitystorm]

Currently, there is no way to request the history of multiple objects in a single request.

Can I just clarify here why multiple HTTP requests are a bad thing? With the API being served over HTTP/3, there's not much overhead for multiple separate parallel requests (a world of difference from HTTP 1.0 and 1.1 pipelining), and early responses (rather than waiting for an entire merged document to be assembled) might be useful to allow the client application to start processing the individual responses as they come in. Are there some disadvantages that I'm missing?

[bhousel]

Can I just clarify here why multiple HTTP requests are a bad thing? With the API being served over HTTP/3, there's not much overhead for multiple separate parallel requests

Each fetch does carry some overhead in the browser. Firing off a thousand fetches in a browser app like Rapid or iD would fill up the browser's microtask queue. I guess I can do this though, if there's no other way.

Thinking about it more, #5005 might be the cause of the "infiinite conflict checking" bug.
facebook/Rapid#1361
openstreetmap/iD#7199

[deevroman]

With the API being served over HTTP/3,

I'll add that even though I use the latest version of Firefox, *iD editors only use http2. In http2, however, there is still parallelization, but what about JOSM, in which http2 is enabled by a separate plugin?

[mmd-osm]

We could also extend the current definition of how a version looks like:

nodes?nodes=1v1,1v2,...2v10,2v11,2v12,....

Could be enhanced towards:

• nodes?nodes=1v1..2,2v10..12 -> start..end are version ranges,
  Start/end might be optional: e.g. 1v1.. for full history
• nodes?nodes=1v-3,2v-3 -> return 3 most recent versions.
  Maybe also include start/end versions for negative range values, e.g. nodes?nodes=1v-20..-10
  Instead of - other characters might also work, like ^ (similar to git HEAD^2)