refactor extradata codec to unblock ccip ocr message optimization using protobuf #16402

huangzhen1997 · 2025-02-13T22:32:08Z

The decodec extra data map (map[string]any) become a difficulty for ccip gRPC optimization.
https://chainlink-core.slack.com/archives/C0779R3AH8R/p1739457726431579

Step 1, use the extra codec functions inside the codec and msgHasher, and get rid of the usage of decoded map.
Step 2, this will be another PR in chainlink-ccip, remove the map.

…ng protobuf

github-actions · 2025-02-13T22:55:10Z

AER Report: CI Core ran successfully ✅

aer_workflow , commit

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

winder · 2025-02-13T23:16:35Z

core/capabilities/ccip/ccipsolana/msghasher.go

+func NewMessageHasherV1(lggr logger.Logger, extraDataCodec cciptypes.ExtraDataCodec) *MessageHasherV1 {
 	return &MessageHasherV1{


not sure we need to pass this in, how about initializing it automatically

Suggested change

func NewMessageHasherV1(lggr logger.Logger, extraDataCodec cciptypes.ExtraDataCodec) *MessageHasherV1 {

return &MessageHasherV1{

func NewMessageHasherV1(lggr logger.Logger) *MessageHasherV1 {

extraDataCodec := ccipcommon.NewExtraDataCodec()

return &MessageHasherV1{

This does make mocking easier

Do you need to mock the encoder? It looks like the test is doing the full borsh encoding of the extra args, so the real one might work

If it's EVM->Solana, we would need ABI encoding, which will introduce some evm related logic in under the ccipsolana.

winder · 2025-02-13T23:17:38Z

core/capabilities/ccip/ccipsolana/msghasher.go

@@ -20,12 +20,14 @@ import (
 // Compatible with:
 // - "OnRamp 1.6.0-dev"
 type MessageHasherV1 struct {
-	lggr logger.Logger
+	lggr           logger.Logger
+	extraDataCodec cciptypes.ExtraDataCodec


Lets get rid of the public interface too

Suggested change

extraDataCodec cciptypes.ExtraDataCodec

extraDataCodec ccipcommon.NewExtraDataCodec

I assume Will meant "extraDataCodec ccipcommon.ExtraDataCodec". Use the type from this repo itself.

This is also because of the mocking, cciptypes.ExtraDataCodec interface gives more flexibility

Hmm, if we are going to remove the one from chainlink-ccip repo, perhaps just create it inside ccipcommon in this repo

prashantkumar1982

Looks good!

prashantkumar1982 · 2025-02-14T01:09:09Z

core/capabilities/ccip/ccipsolana/msghasher.go

@@ -20,12 +20,14 @@ import (
 // Compatible with:
 // - "OnRamp 1.6.0-dev"
 type MessageHasherV1 struct {
-	lggr logger.Logger
+	lggr           logger.Logger
+	extraDataCodec cciptypes.ExtraDataCodec


I assume Will meant "extraDataCodec ccipcommon.ExtraDataCodec". Use the type from this repo itself.

prashantkumar1982 · 2025-02-14T01:12:34Z

core/capabilities/ccip/oraclecreator/plugin.go

+		ExecutePluginCodec: ccipsolana.NewExecutePluginCodecV1(ccipcommon.NewExtraDataCodec()),
+		ExtraArgsCodec:     ccipcommon.NewExtraDataCodec(),
+		MessageHasher: func(lggr logger.Logger) cciptypes.MessageHasher {
+			return ccipsolana.NewMessageHasherV1(lggr, ccipcommon.NewExtraDataCodec())


nit: rather than create 2 instantiations of ExtraDataCodec, 1 for member ExtraArgsCodec, and other for member MessageHasher, I'd say, create just 1 and share it.

I think the other instantiation goes away completely when we remove it from the ccip reader and delete the decoded arg types.

prashantkumar1982 · 2025-02-14T16:49:07Z

core/capabilities/ccip/oraclecreator/plugin.go

+	chainsel.FamilyEVM: {
+		CommitPluginCodec:   ccipevm.NewCommitPluginCodecV1(),
+		ExecutePluginCodec:  ccipevm.NewExecutePluginCodecV1(),
+		ExtraArgsCodec:      ccipcommon.NewExtraDataCodec(),


This too should be extraDataCodec

…obuf-encoding

prashantkumar1982 · 2025-02-14T19:48:24Z

core/capabilities/ccip/oraclecreator/plugin.go

@@ -53,12 +53,12 @@ import (
 )

 var _ cctypes.OracleCreator = &pluginOracleCreator{}
-var extraDataCodec = ccipcommon.NewExtraDataCodec()
+var extraDataCodec = ccipcommon.NewExtraDataCodec(ccipevm.ExtraDataDecoder{}, ccipsolana.ExtraDataDecoder{})
 var plugins = map[string]plugin{
 	chainsel.FamilyEVM: {
 		CommitPluginCodec:   ccipevm.NewCommitPluginCodecV1(),
 		ExecutePluginCodec:  ccipevm.NewExecutePluginCodecV1(),


So EVM's ExecutePluginCodec and MessageHasher should also need access to extraDataCodec, similar to how solana is needing them right?

I think currently it is not there, because we haven't supported Solana -> EVM side codecs.
In EVM codec, I see we are decoding extraArgs and destinationGas directly, assuming ABI encoding, and not using the extraDataCodec.

Could we make the change here itself?

As we discussed offline, the evm changes can happen in another PR

cl-sonarqube-production · 2025-02-14T21:55:13Z

Quality Gate passed

Issues
2 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

* add workflow registry view * print instead of return from registry errors [CAPPL-442] add workflow registry view (#16336) * add workflow registry view * print instead of return from registry errors Added commit price only method config to Solana ChainWriter (#16401) * Added commit price only method config to Solana ChainWriter * Moved ATA config from commit method to execute * Upgraded chainlink-ccip dependency * Addressed feedback refactor extradata codec to unblock ccip ocr message optimization using protobuf (#16402) * refactor extradata codec to unblock ccip ocr message optimization using protobuf * goimport * revert * minor * minor * refactor * fix import * fix import * add comments * fix test * support Solana->EVM * fix * update * fix make [NONEVM-1319] - Solana Contract Reader Dest and Source cfg (#16308) * Implement Solana Contract Reader config and bump chainlink-solana * Update CR config * Resolve more TODOs from Contract Reader config * Bump solana * Progress on CR config * Add comments to CR config * Bump chainlink-ccip/chains/solana in core too * Use feeQuoterIDL * Fix CR config issues * Simplify CR config and add a TODO * Remove CR config resolved TODOs and add serialisation test * Add MethodNameOffRampLatestConfigDetails to CR config * Fix cfg * Upgraded chainlink-common dependency * Bump common and resolve OffRampLatestConfigDetails issues * Fixed linting --------- Co-authored-by: Blaž Hrastnik <blaz@mxxn.io> Co-authored-by: amit-momin <amit.momin@smartcontract.com> Add new RMN smoke test (#16385) * Add new RMN smoke test * Fix linting issues [TT-1991] use github.sha in integration-tests workflow (#16396) * use github.sha in integration-tests workflow * more Slack message debug * use reusable workflow sha from main branch TT-1956 Add support for e2e docker tests in Flakeguard #16376 (#16380) * Bump Flakeguard * wip * update test * update test cmd * bump * bump * bump * bump * fix * bump * fix * bump * bump * bump * bump * fail test * bump * Run nightly workflow with flakeguard * bump * bump * Add extraArgs input to E2E tests workflow for customizable arguments * bump * bump * bump * bump * Add extraArgs input to nightly E2E tests workflow for customizable arguments * bump * bump * bump * bump * bump * revert tests * bump * bump * bump * bump * bump * bump * bump * trigger tests * Revert "trigger tests" This reverts commit dbb37c4. fix: restore files go mod

refactor extradata codec to unblock ccip ocr message optimization usi…

b7dd72d

…ng protobuf

huangzhen1997 marked this pull request as ready for review February 13, 2025 22:32

huangzhen1997 requested review from a team as code owners February 13, 2025 22:32

huangzhen1997 requested review from winder, asoliman92, makramkd, dimkouv, mateusz-sekara, 0xAustinWang, 0xnogo, Atrax1 and prashantkumar1982 February 13, 2025 22:32

goimport

9dfd6e7

revert

7c4c51f

winder reviewed Feb 13, 2025

View reviewed changes

prashantkumar1982 reviewed Feb 14, 2025

View reviewed changes

huangzhen1997 added 2 commits February 14, 2025 09:27

minor

3d50aab

minor

30c54b3

huangzhen1997 requested review from winder and prashantkumar1982 February 14, 2025 15:27

prashantkumar1982 reviewed Feb 14, 2025

View reviewed changes

refactor

a5edbf9

huangzhen1997 requested a review from a team as a code owner February 14, 2025 18:37

huangzhen1997 requested a review from prashantkumar1982 February 14, 2025 18:40

huangzhen1997 added 2 commits February 14, 2025 12:46

fix import

38d04a8

Merge branch 'develop' into extra-data-codec-refactor-to-support-prot…

f93a040

…obuf-encoding

huangzhen1997 added 3 commits February 14, 2025 13:21

fix import

a3b22e1

add comments

2d4144f

fix test

817ff37

prashantkumar1982 reviewed Feb 14, 2025

View reviewed changes

prashantkumar1982 previously approved these changes Feb 14, 2025

View reviewed changes

support Solana->EVM

f1faab0

huangzhen1997 dismissed prashantkumar1982’s stale review via f1faab0 February 14, 2025 20:56

huangzhen1997 added 3 commits February 14, 2025 14:59

fix

7005577

update

6b01819

fix make

e8a969f

huangzhen1997 requested a review from prashantkumar1982 February 14, 2025 21:41

prashantkumar1982 approved these changes Feb 14, 2025

View reviewed changes

winder approved these changes Feb 14, 2025

View reviewed changes

prashantkumar1982 added this pull request to the merge queue Feb 14, 2025

Merged via the queue into develop with commit 13b7d5a Feb 15, 2025
200 checks passed

prashantkumar1982 deleted the extra-data-codec-refactor-to-support-protobuf-encoding branch February 15, 2025 00:17

github-actions bot mentioned this pull request Feb 15, 2025

[DO NOT MERGE] Changeset Release Preview - v2.21.0 #13148

Draft

github-actions bot mentioned this pull request Feb 18, 2025

[DO NOT MERGE] Changeset Release Preview - v2.21.0 mikeyhodl/chainlink#830

Draft

This was referenced Feb 18, 2025

remove extradata encoded map smartcontractkit/chainlink-ccip#628

Closed

remove extra data codec and encoded maps smartcontractkit/chainlink-ccip#627

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor extradata codec to unblock ccip ocr message optimization using protobuf #16402

refactor extradata codec to unblock ccip ocr message optimization using protobuf #16402

huangzhen1997 commented Feb 13, 2025 •

edited

Loading

github-actions bot commented Feb 13, 2025 •

edited

Loading

winder Feb 13, 2025

huangzhen1997 Feb 13, 2025

winder Feb 13, 2025

huangzhen1997 Feb 13, 2025 •

edited

Loading

winder Feb 13, 2025

prashantkumar1982 Feb 14, 2025

huangzhen1997 Feb 14, 2025

prashantkumar1982 Feb 14, 2025

huangzhen1997 Feb 14, 2025

prashantkumar1982 left a comment

prashantkumar1982 Feb 14, 2025

prashantkumar1982 Feb 14, 2025

winder Feb 14, 2025

prashantkumar1982 Feb 14, 2025

prashantkumar1982 Feb 14, 2025

prashantkumar1982 Feb 14, 2025

cl-sonarqube-production bot commented Feb 14, 2025

		func NewMessageHasherV1(lggr logger.Logger, extraDataCodec cciptypes.ExtraDataCodec) *MessageHasherV1 {
		return &MessageHasherV1{

	extraDataCodec cciptypes.ExtraDataCodec
	extraDataCodec ccipcommon.NewExtraDataCodec

refactor extradata codec to unblock ccip ocr message optimization using protobuf #16402

refactor extradata codec to unblock ccip ocr message optimization using protobuf #16402

Conversation

huangzhen1997 commented Feb 13, 2025 • edited Loading

github-actions bot commented Feb 13, 2025 • edited Loading

AER Report: CI Core ran successfully ✅

AER Report: Operator UI CI ran successfully ✅

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

huangzhen1997 Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prashantkumar1982 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Feb 14, 2025

Quality Gate passed

huangzhen1997 commented Feb 13, 2025 •

edited

Loading

github-actions bot commented Feb 13, 2025 •

edited

Loading

huangzhen1997 Feb 13, 2025 •

edited

Loading