Managed CCF Apps - CPP application

[bsenthilr]

Managed CCF Apps
Could not find any azure support ticket category to post this question to trying out here for some guidelines.

Evaluation managed CCF preview version and it seems there is only support to deploy customer JS APP. Is it feasible to deploy custom CPP application on managed CCF apps on azure?

From sample cpp repository it implies one would need to redeploy CCF node since node enclave configuration parameter indicates required open enclave shared object. Assuming its not feasible to dynamically load a CPP application for above reason.

Can someone please clarify? Also please suggest a better forum where managed CCF question should be posted.

[achamayou]

Hello @bsenthilr, there is currently no support, or plan that I am aware of to support native/C++ apps in Managed CCF. You would need to deploy these applications yourself, on top of AKS for example.

[bsenthilr]

thanks @achamayou

[bsenthilr]

hi @achamayou referring to this document

   "deploymentType": {
      "languageRuntime": "CPP",
      "appSourceUri": "https://myaccount.blob.core.windows.net/storage/mccfsource?sv=2022-02-11%st=2022-03-11"
    }

Seem to indicate CPP app can be loaded.

Im not clear how creating CCF under ACL would provision member nodes etc.,

Please let me know if it makes sense to move this discussion on Azure support channel for ACL?

Thanks

[andpiccione]

Hi @bsenthilr, I'm from the Managed CCF team. I confirm that C++ apps are currently not supported by the service.

That CPP language runtime option you see in the API specifications was originally added to support the scenario of deploying a CCF app from a custom application source (including C++ source code), but the implementation for this feature has not been completed yet.

If you would like this feature to be prioritized, I suggest contacting Azure support to find the right channels for this to be escalated. Otherwise, I can pass this message to our Product Managers and keep you posted if/when there are any updates on this specific topic.

[andpiccione]

Sounds good, please feel free to reach out for any other question or clarification.

What are some crucial criteria for a APP to choose CPP over JS (performance?). Any insights here would help with evaluation.

I'll let the CCF team (cc @achamayou) answer this question as they can provide more details on the difference between the two options from a CCF perspective.

[achamayou]

Mainly performance, there is some variation depending on the amount of code, the fraction that really calls into C++ etc, but TS/JS apps tend to be about an order of magnitude slower. The API selection is also a little more constrained than in C++ (notably, there is no way to write indexing strategies in C++ at the moment).

On the flip side, there are a number of sandbox controls (heap, stack, runtime limits) that don't exist in C++, plus obviously memory safety, and strong isolation between execution contexts, which generally make building safe applications easier. NPM libraries that target a browser environment, or don't require Node APIs, can often be imported with little or no change as well, whereas adding C++ libraries is more involved.

Lastly, the upgrade process is different. JS app updates are just another KV transaction, there is literally zero throughput/latency impact during an upgrade. C++ code upgrades mean spinning up new nodes, and reconfiguration. Done optimally, there is little impact because CCF's consensus has a handover mechanism to a new likely primary, but it's at least a new term, and it's very possible to see some consensus-level rollbacks as a result.

[bsenthilr]

Thanks for your information @achamayou

Regarding performance metrics is this a good reference:
long_stress seems to be based on c++. Do not have access to visualization but are these right metrics to correlate to? Based on below JS would probably end up 2X of these numbers is understanding correct?

2024-07-15T03:55:13.7467941Z 80: 03:55:13.745 | INFO     | __main__:print_memory_stats:17 - [0] GET /node/memory
2024-07-15T03:55:13.7468502Z 80: 03:55:13.746 | INFO     | __main__:print_memory_stats:17 - 200 {"current_allocated_heap_size":6307840,"max_total_heap_size":204800000,"peak_allocated_heap_size":6307840}
2024-07-15T03:55:13.7469279Z 80: 03:55:13.746 | WARNING  | __main__:print_memory_stats:18 - {'current_allocated_heap_size': 6307840, 'max_total_heap_size': 204800000, 'peak_allocated_heap_size': 6307840}
2024-07-15T03:55:16.8852260Z 80: Requests      [total, rate, throughput]         135000, 50.00, 50.00
2024-07-15T03:55:16.8852617Z 80: Duration      [total, attack, wait]             45m0s, 45m0s, 1.22ms
2024-07-15T03:55:16.8853132Z 80: Latencies     [min, mean, 50, 90, 95, 99, max]  204.352µs, 1.212ms, 1.21ms, 1.24ms, 1.259ms, 1.356ms, 90.995ms
2024-07-15T03:55:16.8853330Z 80: Bytes In      [total, mean]                     2126250, 15.75
2024-07-15T03:55:16.8853481Z 80: Bytes Out     [total, mean]                     2531250, 18.75
2024-07-15T03:55:16.8853634Z 80: Success       [ratio]                           100.00%
2024-07-15T03:55:16.8853811Z 80: Status Codes  [code:count]                      200:135000  

Regarding JS my understanding is there are 2 main favorable factors:

• Manage CCF / ACL only supports JS
• Upgrade of application is hitless. But JRE is not. Not clear what would generally be cadence of JRE upgrade but if security scans forces upgrades that would imply more upgrades. Been noticing in general azure VM are always patched once a week and its not a rare event. Hope JRE cadence is not similar but would help to gather that information.

Sandbox controls are interesting. Let me catchup on indexing strategy.

Are there any hints on how to migrate prevailing cpp to JS. (not sure emscript wasm is a way to go). Mostly translation should be reasonable rewrite since APP are generally not that complex. But if your familiar with any tooling to facilitate this would be helpful to quickly translate and prototype with managed ACL.
Was able to run the cookie cutter sample app that comes with managed ACL and mapped the endpoints to bank samples (since it is not obvious and documented). But works well. If there are samples CTEST to stress test that app, can give a shot or will build a quick curl test.

Azure Managed application would ease a lots of CCF operations to Azure. JS APP upgrade is straightforward there. Wondering if Azure will deal with JRE upgrades for security workflows. The cost seems very reasonable compared to hosting N confidential VM. Is my cost understanding right?

Please share your thoughts.
Thanks a lot again.

[achamayou]

@bsenthilr I suggest taking a look at this Bencher graph and selecting only the first two (Basic/Basic JS) to get a sense of the difference with a very simple handler that just writes a single value to the KV. But yes, the impact will vary per application.

I am not sure what you mean by JRE? Do you mean the JavaScript runtime? That is part of CCF/the node binary, and will be upgraded and kept up to date by the operator (Azure in the case of Managed CCF). There isn't a specific amount of downtime other than 1.5 network round trip time, the node replacement happens on-line. But because the old primary needs to hand over to a new node, there is always a term change, and that can cause some transaction to roll back if they race with the handover. This is observable by the user (the Tx status changes to invalid) who can retry them.

There is no tooling I am aware of for migration from C++ to TS/JS (or the other way around for that matter). One thing that we have in JS is a polyfill for some CCF APIs, to allow you to unit test the JS handlers.

In terms of cost, I suggest using the Azure cost calculator, things vary per region etc. I would obviously recommend using the Azure hosted service :)

[bsenthilr]

Thanks @achamayou very helpful information !
Assuming virtual metrics is posted because enclaves are anticipated to be lower. Is that the reason there are no non-virtual metrics? If SGX based metrics are available please share a pointer. Tried to review all metrics and didn't non virtual metrics.

Do you mean the JavaScript runtime? => yes

Surprised to see almost 10X differential on throughput with basic app. 2x is not concerning, but such high differentials would impact cost.

Also as you indicate application impacts throughput although not clear what aspects of those application create those side effects. (log data movement, historic or range access, commit waits etc.,). Are there any JS reference to cross check logging application differentials between cpp / js. Assuming typescript metrics can be assumed canonical to JS.

Also JWT vs cert authentication for logging has 5x difference. Why is that? The only overhead should be based on token signature validation.

Regarding service scaling: either with managed service or premise based CCF, the network can only scale so much vertically right? When capacity hits bounds assuming additional ACL needs to provisioned and data shard strategy is required right? In such case does ACL offer some strategy to manage shards or its onus of users?

Was about to ask about leader/view change. Thanks for commenting on that.

Are there some design article regarding how JS app integration works under covers? Does execution/runtime load the application once? Noticed c++ wrapper generation around JS and not clear how the flow integrates. Any insights here would be helpful to understand the integration better to evaluate.

Thanks again

[achamayou]

@bsenthilr we have stopped maintaining continuous performance testing infrastructure on SGX for practical reasons, but you can find numbers until April here: #6126 (comment). The ratios between JS and C++ are similar there.

Surprised to see almost 10X differential on throughput with basic app. 2x is not concerning, but such high differentials would impact cost.

We use what we believe to be the fastest available interpreter, but it's still an interpreter. The isolation and sandboxing guarantees are also vastly different from C++. Keep in mind that the measured throughputs are for long-lived, pipelined connections. In a more typical usage, with lots of clients running short lived connections, the cost of the TLS handshake will outweigh the cost of the transaction.

Assuming typescript metrics can be assumed canonical to JS.

Yes. We don't have an extensive reference of costs, you will need to benchmark your specific use case.

Regarding service scaling: either with managed service or premise based CCF, the network can only scale so much vertically right? When capacity hits bounds assuming additional ACL needs to provisioned and data shard strategy is required right? In such case does ACL offer some strategy to manage shards or its onus of users?

Yes, there is no built-in data sharding support at the moment. We have done some work in the KV to prepare for it, but it's a future item, which will be prioritised according to customer demand.

Are there some design article regarding how JS app integration works under covers? Does execution/runtime load the application once?

There are no design articles, but happy to answer questions. By default a fresh runtime is created for every transaction, and completely isolated from others. There is a way to specify through configuration that one or more endpoints should re-use interpreters if desired, to trade off isolation against transaction startup latency.

[bsenthilr]

Hi @achamayou thanks again and really appreciate your help to understand the details.
Tried sample app deployment it went smooth, evaluating porting some internal apps and overheads. Still early on the process.

At a high level trying to understand how JS app code gets integrated on CCF execution layer. Seems the wrap layer implements the KVMap. Seems the wrapper layer creates runtime (quickjs) and sandboxes and dispatches the JS endpoint application callback. Is this a proper understanding if not please clarify.

Trying to compare and update API on C++ using KV API.
Does the wrapper eventually calls into C++ core logic for KV / Txn etc., ? Basically the local KV states would still exist within C++ context even though sandbox of the endpoint dispatch gets wiped off with fresh runtime on each dispatch.
Is this a right understanding if not can you please clarify the design?

Referring to kv.ts for KV is the proper reference? The API seems to missing pieces compared with C++ KV.
Also what is equivalent of get_globally_committed() on JS?

Not clear how to compare with "ccf::Tx" (ro / rw) context. The invoked endpoint is a transaction context and it seems to implicitly determine the mode via "mode" attribute of the endpoint. In event a transaction needs to invoke a readonly access eg., tx.ro<ccf::UserCerts>(Tables::USER_CERTS) from a readwrite mode endpoint how to express that on JS?