Design for moving the app graph to actual graph storage #97

sylvainsf · 2025-06-05T17:42:43Z

This design adds a new component to Radius, the graph abstraction layer, and allows for app graph functionality to leverage the performance and simplicity of a graph db rather than the imperative graph traversal logic currently implemented in CoreRP.

architecture/2025-06-non-k8s-controlplane/2025-06-graph-db-replace-planes.md

brooke-hamilton

📈 🚀

architecture/2025-06-non-k8s-controlplane/2025-06-graph-db-replace-planes.md

ytimocin

I will need to do another round of review for this doc.

architecture/2025-06-non-k8s-controlplane/2025-06-graph-db-replace-planes.md

ytimocin · 2025-06-11T18:21:50Z

Do we need to write all the resources to Kuzu too? Are we using Kuzu as the main data store now?

architecture/2025-06-non-k8s-controlplane/2025-06-graph-db-replace-planes.md

Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

…fic radius scenarios Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

…clearly, made migration tool a requirement. Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

…oing all data storage via graph today with multiple RPs writing to the same data store. Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

Signed-off-by: Sylvain Niles <sylvain.niles@gmail.com>

nithyatsu · 2025-06-25T18:34:40Z

ucp/aws/handle-non-idempotent-resources/2023-09-handle-aws-non-idempotent-resources.md

@@ -1,6 +1,6 @@
 # Handling Non-Idempotent AWS Resources in UCP

-**Note: This design doc has been ported over from an old design doc and might not match the template completely**
+**Note: This design doc has been ported over from an old design doc and might not match Note that here we are using the Radius resource group to construct the key of the tracking entry instead of the AWS `resourceID`. The deployment engine provides the resource group information from the deployment's resource group during the POST call that it makes to UCP to create/update the resource. This will associate the AWS resource with the Radius resource group, and thus allow the user to list AWS resources deployed within a Radius resource group.he template completely**

 * **Author**: Vinaya Damle (vinayada)



I think this doc was added by mistake?

Yes sorry I said that at the beginning of the meeting my copilot accidentally modified, will remove changes :)

nithyatsu · 2025-06-25T18:52:44Z

architecture/2025-06-graph-db-replace-planes.md

+
+### User Scenarios (optional)
+
+* **Scenario 1 (Developer):** A Radius developer needs to address performance issues for application graphs containing many resources. Retrieving all resources connected to a specific environment, which is a very expensive operation. Using Cypher through the GAL would be more expressive and orders of magnitude faster than multiple etcd calls and client-side filtering.


We would still take care to support only the read operations correct?

nithyatsu · 2025-06-25T18:55:15Z

architecture/2025-06-graph-db-replace-planes.md

+    ```bash
+    rad resource inspect gateway api-gateway --show-dependencies --type secret
+    ```
+


--show-dependencies is awesome :)

willtsai · 2025-06-25T18:59:07Z

architecture/2025-06-graph-db-replace-planes.md

+4.  **Schema Definition:** A formal schema for application graph entities (Applications, Environments, Resources as nodes) and their relationships (as edges with types and properties) will be defined and enforced in PostgreSQL + AGE.
+5.  **Component Updates:** Radius components that currently perform application graph operations (traversals, relationship queries) will be updated to use the GAL for graph queries while maintaining existing functionality through current storage mechanisms during the transition period.
+
+**Architecture Rationale:** PostgreSQL with Apache AGE provides a strategic foundation that can serve both immediate graph database needs and future storage consolidation. Unlike embedded solutions, a containerized PostgreSQL deployment offers: (1) Production-ready operational patterns familiar to most teams; (2) Native support for both relational and graph data models; (3) Established ecosystem of monitoring, backup, and scaling tools; (4) The flexibility to migrate additional Radius storage needs to the same technology stack over time, reducing operational complexity. **Note: This proposal is scoped specifically to graph operations via the GAL. Future work would involve migrating key-value operations from etcd to PostgreSQL and renaming the GAL to DAL (Data Access Layer) to reflect its expanded role as the unified data access interface for all Radius storage operations.**


Note: This proposal is scoped specifically to graph operations via the GAL. Future work would involve migrating key-value operations from etcd to PostgreSQL and renaming the GAL to DAL (Data Access Layer) to reflect its expanded role as the unified data access interface for all Radius storage operations.

Will data be duplicated across both etcd and the graph database in the meantime?

sylvainsf requested review from a team as code owners June 5, 2025 17:42

nicolejms reviewed Jun 11, 2025

View reviewed changes