Skip to content

canton v2.10.0

Latest
Compare
Choose a tag to compare
@canton-machine canton-machine released this 12 Feb 20:41
· 6 commits to main since this release
4e07aac

Release of Canton 2.10.0

Canton 2.10.0 has been released on February 12, 2025. You can download the Daml Open Source edition from the Daml Connect Github Release Section. The Enterprise edition is available on Artifactory.
Please also consult the full documentation of this release.

Summary

Release 2.10.0 has three key enhancements.

First, the long awaited Smart Contract Upgrade (SCU) is now Generally Available. So, fixing application bugs or extending Daml models is possible without downtime or breaking Daml clients. This feature also eliminates the need to hardcode package IDs which increases developer efficiency. The steps to enable this feature are described below. It was introduced as Beta in v2.9.1 and has since undergone a lot of hardening based on customer feedback so it transitioned to GA status.

Secondly, the new “Tri-State Vetting” feature complements SCU by allowing a DAR to be unvetted in production. This is helpful to disable a DAR that has a bug in its business logic and then upgrade it without downtime.

Lastly, this is the first Daml Enterprise release to be designated a Long Term Support (LTS) release. An LTS release focuses on long term stability and may forgo feature enhancements. It is supported for multiple years and can be upgraded to with full data continuity, but it does have some caveats:
Limited cross-version compatibility, specifically no guaranteed support for old Canton Protocol versions.
No long-term support for deprecated features.
Focuses on supporting LTS versions of environment components (java, dbs, etc).
LTS releases are only guaranteed to have Critical or High issues fixed. Lower priority fixes are possible but not mandatory.

What’s New

Introduction of protocol version 7

Background

There is a new Canton protocol version (PV=7) in this release and it also supports PV=5.

Specific Changes

Protocol version 6 has been marked as deleted and should not be used.
Protocol version 7 has been introduced as its stable replacement. There is also a new Daml LF version (LF 1.17).
Some features have been deprecated in support of this being a LTS release and also to enable SCU.

Impact and Migration

Please remember that since version 2.9, you must set the protocol version explicitly. In prior releases, the domain protocol version was set to the latest protocol version by default. To specify the protocol version for a domain:

myDomain {
init.domain-parameters.protocol-version = 7
}

Specifying for a domain manager is:

domainManager {
init.domain-parameters.protocol-version = 7
}

You can read more about protocol versions in the public docs. If you are unsure which protocol version to pick, use the latest one supported by your binary (see docs).

Please ensure all your environments use the same protocol version: you should not use one protocol version in your test environment and another one in production.

If a protocol version is not provided, then an error message like this will be generated:

ERROR c.d.c.CantonEnterpriseApp$ - CONFIG_VALIDATION_ERROR(8,0): Failed to validate the configuration due to: Protocol version is not defined for domain `mydomain`. Define protocol version at key `init.domain-parameters.protocol-version` …

Daml-LF versions 1.14, 1.15, and 1.17 are available in this release. LF 1.15 is the default Daml compiler setting. LF 1.16 has been deleted and should not be used. Use LF 1.17 to enable the SCU feature.

The compatibility matrix is:

  • PV5 is compatible with LF 1.14 and LF 1.15.
  • PV7 is compatible with LF 1.14, LF 1.15, and LF 1.17.
  • PV7 and LF 1.17 enable SCU and SCU is disabled with any other configuration.

Smart Contract Upgrading (SCU)

The SCU feature debuted as a Beta level feature in 2.9.1 and it is now GA. These release notes are similar to the 2.9.1 release notes with some updates.

Background

SCU allows Daml models (packages in DAR files) to be updated on Canton transparently, provided guidelines in making the changes are followed. For example, you can fix a Daml model bug by uploading the DAR that has the fixed package version. This feature requires LF 1.17 and Canton Protocol versions 7. The detailed documentation is available here with the reference documentation available here. Please note that enabling this feature may require code updates for rarely used language idioms. For example, it does require that the daml.yaml files set the package version and that the version is increasing as new versions are developed.

Details

This feature is well-suited for developing and rolling out incremental template updates. There are guidelines to ensure upgrade compatibility between DAR files. The compatibility is checked at compile time, DAR upload time, and runtime. This is to ensure data backwards compatibility and forward compatibility (subject to the guidelines being followed) so that DARs can be safely upgraded to new versions. It also prevents unexpected data loss if a runtime downgrade occurs (e.g., a ledger client is using template version 1.0.0 while the participant node has the newer version 1.1.0).

A general guideline is that additive model changes are allowed but items cannot be removed. A summary of the allowed changes in templates are:

  • A template can add new optional fields at the end of the list of fields;
  • A record datatype can add new optional fields at the end of the list of fields, and a variant/enum datatype can add new constructors at the end;
  • The ensure predicate can be changed and it is reevaluated at interpretation;
  • A choice signature can be changed by adding optional parameters at the end;
  • The controller of a choice can be changed;
  • The observer of a choice can be changed
  • The body of a choice can be changed;
  • A new choice can be added to a template;
  • The implementation of an interface instance can be changed;

The Beta version of this feature allowed a new interface instance to be added to a template but this ability is not available in this GA release. Please consult the documentation for more information.

The package name associates a series of DAR versions where the newest version is the default version to use. The package name and version (e.g., “version: 1.1.0”) are specified in the daml.yaml file. Package name is now part of the Fully Qualified Name instead of the package ID. Internally, the package ID is still available and used at run time where the package name and version are resolved to a package ID. This allows backwards compatibility.There is flexibility where the package ID can still be specified (prior approach) or the package name can be used (new approach). A side effect is that the package name provides a namespace scope where modules, templates, and data belong to the namespace of a package.

To prevent unexpected behavior, this feature enforces that a DAR being uploaded to a participant node has a unique package name and version. This closes a loophole where the PN allowed uploading multiple DARs with the same package name and version. For backward compatibility, this restriction only applies for packages compiled with LF >= 1.17. If LF <= 1.15 is used, there can be several packages with the same name and version but this should be corrected as it will not be supported further.

Compilation support for smart contract upgrade is enabled by adding following fields to the daml.yaml:

  • --target=1.17
  • ‘upgrades: ’

For additional type checking, use the ‘dry-run’ option which simulates the checks a PN will run during the upload step. The format of the command is “daml ledger upload-dar --dry-run” which can be included as part of a CI/CD process.

The JSON API server is compatible with the smart contract upgrade feature by:

  • Supporting package names for commands and queries;
  • Allowing use of an optional packageIdSelectionPreference field to specify a preferred package ID to use. This allows the client to specify the package ID like in prior releases but it is not a best practice;
  • Requiring either a package ID or package name to be present to disambiguate the partially-qualified form of template/interface ids.

Previously JSON API had supported partially qualified template IDs, (i.e. simply <module>:<entity>) as an interactive convenience which fails if there is more than one package with matching template names. Since this format was not supported for production use and will not work with smart contract upgrades, it is now unavailable.

The Java and TypeScript codegen allow the use of package name and package ID (if needed).

Impact and Migration

This feature is not enabled by default. There are four steps that are required to enable this feature:

  • Compile the Daml model(s) into LF 1.17 (LF 1.15 is the default). When using the daml build command to compile a Daml project, make sure the LF-version is set to 1.17. To do this, set this field in the daml.yaml

    build-options:
    ---target=1.17
    

    Additionally, use the following field to enable compile time checking of LF 1.17 upgrade DARs:

    upgrades: <path to dar files of prior versions>
    
  • The Canton protocol version needs to be set to 7. (This is the default version for daml sandbox and daml start.) See here (the parameter protocolVersion) for domain parameter configuration.

  • Use the script library called daml-script-lts instead of the older daml-script.

  • Domain migration. For existing systems, the protocol version change requires a domain migration that is discussed in “Change the Canton Protocol Version.”

This feature is not compatible with some deprecated coding patterns which may require some code changes.

  1. Retroactive interface instances are not supported.
  2. Using divulged contracts during interpretation, is deprecated since Daml 1.14, and does not work with upgrades.
  3. Interface definitions are not upgradable by SCU. So, to allow upgrading of templates, move co-mingled interfaces into a different package that can be referenced by all current and future versions of the template.
  4. Exception definitions are not upgradable by SCU. So, to allow upgrading of templates, move co-mingled exception definitions into a different package.
  5. Test scripts are not upgradable by the SCU. Move your tests to a separate dedicated package and make sure they are not uploaded to the ledger.

Please refer to the documentation for more details.

Known limitations

Compile time compatibility checking in the IDE is not reflected directly in the IDE (Daml Studio).

Triggers are incompatible with the DAML-LF 1.17. As a consequence, you cannot use upgrading (available in 1.17) and triggers (available in 1.15) in combination.

A participant running 2.10 and protocol version 5 will not be able to connect to a domain running on versions 2.9.0 through 2.9.3). The issue is fixed on 2.9.4.

Tri-state vetting

Background

Starting with protocol version 7, topology management is extended with full support for unvetting which disables all Choices and bare contract creates for the unvetted DAR. You would want to use this if there was a DAR that had a serious logic error in a Choice and you did not want anyone to execute that Choice so you unvett the DAR. In particular, this feature:

  • Allows a participant to disable a DAR for new transactions, while still allowing the validation of existing contracts that were created with that DAR.
  • Allows upgrading contracts originally created with packages no longer vetted.

Specific changes

A new CheckOnlyPackages topology transaction is introduced, alongside the existing VettedPackages, that allows a participant to specify a collection of Daml packages that can only be used for validating pre-existing contracts on the ledger and not for executing new Daml transactions.

The currently existing DAR disabling/unvetting operations become supported for production usage when:

  • Issuing a specific request against the gRPC participant Admin API: PackageService.unvetDar
  • Using the Canton console command: participant.dars.vetting.disable

By default CheckOnlyPackages transactions will be published automatically when a DAR is disabled. In case low level management of CheckOnlyPackages topology transactions is needed, the following new topology management endpoints have been introduced:

  • For adding new transactions: TopologyManagerWriteService.AuthorizeCheckOnlyPackages or via Canton console: participant.topology.check_only_packages.authorize
  • For querying the current state: TopologyManagerReadService.ListCheckOnlyPackages or via Canton console: participant.topology.check_only_packages.list

Impact and migration

There is no migration required since this is a new protocol version 7 feature. In participants running with prior protocol versions, unvetting of DARs is supported for production use-cases.

Keep Alive configuration in Ledger API

Background

gRPC subsystem comes with a built-in mechanism for sending keep alive signals. All Canton endpoints support
this concept. However, so far only Canton's Public and Admin APIs could be configured to change the default
behavior. Now, also the Ledger API gains this capability.

Specific Changes

The keep alive behavior of the Ledger API can be configured through

canton.participants.participant.ledger-api.keep-alive-server.*

The default values of the keep alive configuration for the ledger api has been set to

time: 10m
timeout: 20s
permitKeepAliveTime: 10s
permitKeepAliveWithoutCalls: true

The default values of the keep alive configuration for all the other endpoints are set to

time: 40s
timeout: 20s
permitKeepAliveTime: 20s
permitKeepAliveWithoutCalls: false

The effective settings are reported by the Participant Node at the initialization time with a logline:

2024-10-31 18:09:34,258 [canton-env-ec-35] INFO  c.d.c.p.a.LedgerApiService:participant=participant - Listening on localhost:5001 over plain text with LedgerApiKeepAliveServerConfig(10m,20s,10s,true).

A new parameter value for permitKeepAliveWithoutCalls has been introduced to all keep alive configurations.
When set, it allows the clients to send keep alive signals outside any ongoing grpc call.

Impact and Migration

No impact.

Enhanced status admin command

The status of all Canton nodes now includes the Canton release version. Additionally, the participant status lists the
supported protocol versions while domain nodes such as the sequencer show the protocol version.

The additional status information is only available from Canton nodes running on this or later versions.

BREAKING: Please note that this change may break Canton scripts if they import NodeStatus and derived classes,
as they have been moved into a new package. This will not happen in the default case but only if the operator
has made explicit use of the types. The following import statement needs to be updated:

-import com.digitalasset.canton.health.admin.data.{NodeStatus, ParticipantStatus}
+import com.digitalasset.canton.admin.api.client.data.{NodeStatus, ParticipantStatus}

Command rejection logging

When a command is rejected due to conflict (e.g. usage of an inactive contract),
every participant detecting the conflict will now log the resource causing the conflict at INFO level.
This change affects the following error codes:

  • LOCAL_VERDICT_LOCKED_CONTRACTS
  • LOCAL_VERDICT_LOCKED_KEYS
  • LOCAL_VERDICT_INACTIVE_CONTRACTS
  • LOCAL_VERDICT_DUPLICATE_KEY
  • LOCAL_VERDICT_INCONSISTENT_KEY
  • LOCAL_VERDICT_CREATES_EXISTING_CONTRACTS

Remote console TLS configuration improvements

Previously correctly configuring TLS for remote node management console in sections canton.remote-domains.<mydomain>,
canton.remote-sequencers.<sequencer> for public-api used different configuration keys from the rest of the system.
One had to specify transport-security = true and custom-trust-certificates.pem-file = "<pem file>" options, while
admin-api, ledger-api sections used tls.trust-collection-file = "<pem file>".
This has been unified, so now all sections use the same configuration key tls.trust-collection-file.
Old keys are still supported for backward compatibility and will produce an INFO level log message.
Keep alive and client TLS authentication are not supported for public-api part of remote console configuration.
Migration: Update your configuration to replace custom-trust-certificates.pem-file keys with tls.trust-collection-file.

TLS client configuration without custom truststore

It is now possible to configure TLS client connections in Canton configuration without providing a truststore,
i.e. when the server's certificate is signed by a well-known CA and can be verified against the OS/environment trust store.
To enable this, set tls.enabled under the respective admin-api, ledger-api or public-api section to true.

Memory check during node startup

A memory check has been introduced when starting the node. This check compares the memory allocated to the container with the -Xmx JVM option.
The goal is to ensure that the container has sufficient memory to run the application.
To configure the memory check behavior, add one of the following to your configuration:

canton.parameters.startup-memory-check-config = warn  // Default behavior: Logs a warning.
canton.parameters.startup-memory-check-config = crash // Terminates the node if the check fails.
canton.parameters.startup-memory-check-config = ignore // Skips the memory check entirely.

Minor Improvements

  • Additional metrics added: Two new metrics have been added that count the number of created and archived contracts observed by a participant.
    Contracts created as part of the standard Canton ping workflow are excluded from the tally.
    participant_daml_parallel_indexer_creates
    participant_daml_parallel_indexer_archivals
    
  • When a Grpc channel is open or closed on the Ledger API, a message is logged at a debug level:
[..] DEBUG c.d.c.p.a.GrpcConnectionLogger:participant=participant - Grpc connection open: {io.grpc.Grpc.TRANSPORT_ATTR_LOCAL_ADDR=/127.0.0.1:5001, io.grpc.internal.GrpcAttributes.securityLevel=NONE, io.grpc.Grpc.TRANSPORT_ATTR_REMOTE_ADDR=/127.0.0.1:49944}
[..] DEBUG c.d.c.p.a.GrpcConnectionLogger:participant=participant - Grpc connection closed: {io.grpc.Grpc.TRANSPORT_ATTR_LOCAL_ADDR=/127.0.0.1:5001, io.grpc.internal.GrpcAttributes.securityLevel=NONE, io.grpc.Grpc.TRANSPORT_ATTR_REMOTE_ADDR=/127.0.0.1:49944}
  • KMS key management This is a type change which may break scripts:
    • Changed the name parameter of rotate_node_key from Option to String.
    • Added a name: String parameter to rotate_kms_node_key, allowing operators to specify a name for the new key.
  • Performance improvements:
    • We reduced the latency on the sequencer for processing and sequencing events from other nodes.
    • We introduced contract key prefetching / bulk loading to improve workloads that fetch many contract keys.
  • Removed deprecated Java bindings: Java binding functionality deprecated in 2.5 for command submission has been permanently removed. This affected following methods
    • redundant constructors of SubmitCommandsRequest
    • redundant serialization methods of the form SubmitCommandsRequest.toProto
    • redundant serialization methods of the form SubmitAndWaitRequest.toProto
    • redundant serialization methods of the form SubmitRequest.toProto
  • Additional monitoring: Several monitoring enhancements are available:
    • Canton can now include JVM Garbage collection events in the log file. This can be enabled using the configuration option canton.monitoring.logging.log-gc-events = true
    • Canton now checks if a Postgres database connection becomes read-only.
    • Added a new command health.count_in_flight that returns the number of pending submissions and transactions on a domain
  • Query cost logging: Periodic query cost logging now also covers the standard deviation of query times alongside the previously supported mean and total times.
  • Transaction Trace: This feature provides a trace to help developers debug and diagnose issues in their DAML model more effectively. It is similar to a stack trace but only tracks command and exercise calls.
    • Logging Level: The trace is outputted at the DEBUG level in the participant log.
    • Error Handling: The trace is output at the point of any interpretation error and when the interpretation is aborted due to a timeout
    • Trace Format: The transaction trace outputs in the following format:
      in choice XXXXXXXX:Mod2:Template2:Choice2 on contract 00XXXXXXXX (#3)
      in choice XXXXXXXX:Mod1:Template1:Choice1 on contract 00XXXXXXXX (#1)
      in create-and-exercise command XXXXXXXX:Mod1:Template1:Choice1.
      
      Such a trace indicates that the point where the error occurred or the execution was aborted happened inside the body of Choice2,
      which was exercised from the body of Choice1, which was itself part of an API create-and-exercise command.
      The trailing (#x) indicates the index of the corresponding exercise node within the transaction that would have been
      generated if the interpretation had passed through.
  • A participant will now crash in exceptional cases during transaction validation instead of remaining in a failed state.
  • Disabled the onboarding timeout for participants to support onboarding to domains with very large topology states
    without annoying warnings and timeouts.
  • Removed warnings about failing periodic acknowledgements during initial domain onboarding of participants.
  • Removed warnings about unhealthy sequencers during startup.

Bugfixes

  • A queue of tasks was unbounded and could use a large amount of heap memory which could result in
    an OutOfMemoryError. This has been optimized.
  • A ledger fork and participant could crash due to a retroactive interface due to the view reinterpretation
    of an exercise of a retroactive interface failing. This is fixed.
  • A participant replica fails to become active under certain database network conditions. The previously
    active replica fails to fully transition to passive due to blocked database connection health checks,
    which leaves the other replica to fail to transition to active. Eventually the database health checks
    get unblocked and the replica transitions to passive, but the other replica does not recover from
    the previous active transition failure, which leaves both replicas passive.
    This has been resolved.
  • A participant replica could hang during initialization due to a race condition that is fixed.
  • A rare occurrence could happen that would result in a participant being unable to reconnect to a domain.
    Contracts created on participants running PV3 have an unauthenticated contract ID. When these participants
    are upgraded to PV5 without setting the allow-for-unauthenticated-contract-ids flag to true, any submitted
    transaction that uses such unauthenticated contract IDs will produce warnings during validation, but also
    put the participants in an incorrect state. From then on, the participant will not output any ledger events
    anymore and fail to reconnect to the domain.
  • Since 2.9.0, the Hard Synchronization Domain Migration command repair.migrate_domain aborts when it detects
    in-flight submissions on the participant. It now also includes a check for in-flight transactions.

Patches applied to 2.8:

(24-022, Moderate): Participant replica does not clear package service cache

Issue Description

When a participant replica becomes active, it does not refresh the package dependency cache. If a vetting attempt is made on the participant that fails because the package is not uploaded, the "missing package" response is cached. If the package is then uploaded to another replica, and we switch to the original participant, this package service cache will still record the package as nonexistent. When the package is used in a transaction, we will get a local model conformance error as the transaction validator cannot find the package, whereas other parts of the participant that don't use the package service can successfully locate it.

Affected Deployments

Participant

Affected Versions

2.8.0-2.8.10, 2.9.0-2.9.4

Impact

Replica crashes during transaction validation.

Symptom

Validating participant emits warning:


LOCAL_VERDICT_FAILED_MODEL_CONFORMANCE_CHECK(5,a2b60642): Rejected transaction due to a failed model conformance check: UnvettedPackages

And then emits an error:

An internal error has occurred.
java.lang.IllegalStateException: Mediator approved a request that we have locally rejected

Workaround

Restart recently active replica

Likeliness

Likely to happen in any replicated participant setup with frequent vetting attempts and switches between active and passive replicated participants between those vetting attempts.

Recommendation

Users are advised to upgrade to the next patch release during their maintenance window.

(24-028, Medium): ACS export and party replication is broken after hard domain migration

Issue Description

The macros for the various steps for migrating a party look up domain parameters in the topology store, but don't filter out irrelevant domains. This results in the macro throwing an error because it finds multiple domain parameteres after a hard domain migration, even though one of them comes from an inactive domain.

Affected Deployments

Participant

Affected Versions

Participant nodes of versions 2.8.0-2.8.11, 2.9.0-2.9.5.

Impact

You cannot migrate a party to or from a participant that went through a hard domain migration.

Symptom

Calling repair.party_migration.step1_store_acs fails with the error "Found more than one (2) domain parameters set for the given domain and time!".

Workaround

The non-self-service work-around is to not call the party migration macros but replicate what these macros do.

Likeliness

The issue consistently occurs when calling the party migration macros after a hard domain migration.

Recommendation

Upgrade the involved participant nodes to the next patch release: 2.8.12 or 2.9.6.

(24-029, Low): Domain topology manager gets stuck on too large batches

Issue Description

An off by one check fails in the topology dispatcher of the domain manager as
batches are not limited to N but to N+1, while we check for N.

Affected Deployments

Domain topology manager
Domain

Affected Versions

All up to 2.7
2.8.0-2.8.11
2.9.0-2.9.5

Impact

Topology transactions stop being propagated through the system.

Symptom

Participants cannot onboard to domains, parties do not appear on the domain, uploaded dars cannot be used.

Workaround

Restart domain topology manager.

Likeliness

Can happen under high topology management load which is rather unusual (adding thousands of parties at full speed).

Recommendation

Update during the next maintenance window.

(25-001, Major): Newly onboarded participants may compute a wrong topology state during bootstrapping

Issue Description

When a participant is onboarded to a domain, the domain manager will send the topology state to the participant. The
topology state is split into batches of 100. If the state contains an add and a subsequent remove of a topology transaction,
and these two topology transactions are in the same batch (so less than 100 transactions apart), but the namespace certificate
or identifier delegation is in a previous batch, then the participant will miss the removal of the topology transaction.
In the common cases, the namespace delegation is always followed by a subsequent add, but it can happen.

Affected Deployments

Participant nodes.

Affected Versions

2.9.0-2.9.5
2.8.0-2.8.11
All versions before

Impact

Depends on the type of topology transaction, but the result is a fork in the topology state, which in a rare but theoretically
possible case (observer nodes and participant using previously removed parties) might create a ledger fork, leading to
participants disconnecting from the domain.

Symptom

If the missed transaction was a mediator domain state, then the participant will fail to submit transactions whenever it
randomly selects the non-existent mediator.

Workaround

No workaround available. Manually repairing the topology state is likely possible, but not recommended.

Likeliness

Happens deterministically if the conditions are met, but the conditions are rare and require a specific sequence of
events with removal of topology state.

Recommendation

Upgrade before removing topology state (disabling parties, rolling keys) or onboarding a new participant to a domain
with a larger number of topology transactions that includes removals.

(25-002, Medium): Intermediate certificate renewal will delete topology state

Issue Description

A Canton node uses topology keys to sign topology transactions. The ultimate trust is tied to the root node key,
which by default is held by the node, but can be moved offline. In such a case, the node may use an intermediate
certificate to manage the topology state. In order to renew such intermediate certificates, the topology state needs
to be re-issued in 2.x, which can be done using the convenience function node.topology.all.renew(oldKey, newKey).
The convenience function contains an error that will instead of renewing the topology state, delete topology transactions
of the type party to participant, mediator domain state and participant domain state (the ones that contain the
replaceExisting flag).

Affected Deployments

Domain, Domain manager, Participant nodes.

Affected Versions

2.9.0-2.9.5
2.8.0-2.8.11
All up to 2.7

Impact

Some of the topology state will be removed after running this operation.

Symptom

Parties, participants and mediators will be missing after running the operation.

Workaround

Manually re-add the missing parties, participants and mediators.

Likeliness

Deterministic if the convenience function is used.

Recommendation

Upgrade before renewing intermediate certificates.

(25-003, High): Identifier delegation cannot be renewed

Issue Description

A Canton node uses topology keys to sign topology transactions. The ultimate trust is tied to the root node key,
which by default is held by the node, but can be moved offline. In such a case, the node may use an intermediate
certificate to manage the topology state. If such an intermediate certificate is used to sign an identifier delegation
(used as an intermediate certificate for a specific uid), then the identifier delegation cannot be renewed,
as the renewal operation will remove the old and the new certificate from the in-memory state. Unfortunately,
after a restart, the certificate could be loaded again which can cause a ledger fork.

Affected Deployments

Domain, Domain manager, Participant nodes.

Affected Versions

2.9.0-2.9.5
2.8.0-2.8.11
All up to 2.7

Impact

The topology state signed with a particular key authorized by an identifier delegation will be removed from the state,
and the key cannot be used to sign new transactions. After a restart of a node, the key would be loaded again, leading
to a possible ledger fork.

Symptom

Topology state missing after an intermediate certificate renewal, with a possible subsequent ledger fork after a restart.

Workaround

Theoretically issue a new identifier delegation for a new key and re-create the topology state. In practice, upgrade
all nodes before renewing intermediate certificates.

Likeliness

Deterministic if several intermediate certificates are used and one of them is rolled in the chain.

Recommendation

Update all nodes to a version with a fix before renewing intermediate certificates.

Compatibility

The following Canton protocol versions are supported:

Dependency Version
Canton protocol versions 5, 7

Canton has been tested against the following versions of its dependencies:

Dependency Version
Java Runtime OpenJDK 64-Bit Server VM Zulu11.72+19-CA (build 11.0.23+9-LTS, mixed mode)
Postgres Recommended: PostgreSQL 12.22 (Debian 12.22-1.pgdg120+1) – Also tested: PostgreSQL 11.16 (Debian 11.16-1.pgdg90+1), PostgreSQL 11.16 (Debian 11.16-1.pgdg90+1), PostgreSQL 13.18 (Debian 13.18-1.pgdg120+1), PostgreSQL 13.18 (Debian 13.18-1.pgdg120+1), PostgreSQL 14.15 (Debian 14.15-1.pgdg120+1), PostgreSQL 14.15 (Debian 14.15-1.pgdg120+1), PostgreSQL 15.10 (Debian 15.10-1.pgdg120+1), PostgreSQL 15.10 (Debian 15.10-1.pgdg120+1)
Oracle 19.20.0