lib, zebra: support incremental json output [Draft] #20166
Draft
+568
−14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
Nice proposal 👍 |
Contributor
Author
|
looks like the versions of the json-c lib have some meaningful differences - I'll see about walling off the older versions... |
mjstapp
force-pushed
the
json_incremental
branch
from
ded70be to
98176b4
Compare
Contributor
Author
|
I've pushed some changes that should help with libjson-c compatibility |
added 2 commits
December 2, 2025 15:28
Add some new lib apis that allow us to incrementally output a json object hierarchy. Many leaf objects can be freed as soon as they've been output. Parent container objects can be kept "open" during long-running iterations, and then "closed" once their children have been processed. This incremental approach reduces the number of json objects in memory; only "open" containers and a limited number of leaf objects need to be in memory. Signed-off-by: Mark Stapp <mjs@cisco.com>
Demonstrate incremental json output with the zebra "show nexthop-group rib" command. Signed-off-by: Mark Stapp <mjs@cisco.com>
mjstapp
force-pushed
the
json_incremental
branch
from
98176b4 to
994b0d4
Compare
Member
|
Could you add some documentation to the doc/developer directory on how this should be used. That way I can point people at this when we say use this functionality. |
Add some doc about the new json lib apis for incremental output. Signed-off-by: Mark Stapp <mjs@cisco.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are a number of scaling issues with the current FRR json output approach. In general, we have to produce the entire json hierarchy of objects and then emit a single string to the vty. At scale, this means a large memory footprint. This PR proposes some new lib apis that allow us to incrementally output a json object hierarchy. Many leaf objects can be freed as soon as they've been output. Parent container objects can be kept "open" during long-running iterations, and then "closed" once their children have been processed. This incremental approach reduces the number of json objects in memory; only "open" parents/containers and a limited number of leaf objects need to be in memory.
The library changes required are a little ... invasive, because we have to have our own wrapper that replaces the generic json-c lib code. Maybe that's a bridge too far, so I've made this a draft PR initially.
I've used a fairly simple zebra show command, "show nexthop-group rib", as an example to illustrate the use of the new apis.