Skip to content

Conversation

@asoliman92
Copy link
Contributor

No description provided.

require.NoError(t, err)

l.Info().Msg("Asserting baseline message reaches verifier but gets dropped due to global curse")
testCtx.AssertMessageReachedAndDroppedInVerifier(messageID01, 100*time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a plan to also add test that shows how we recover those message once the curse is lifted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this replay features e2e tests should be part of M3

CC @makramkd

@asoliman92 asoliman92 force-pushed the asoliman/chain-curse-e2e-test branch from 8782f5a to 1f90f75 Compare November 13, 2025 14:40
@github-actions
Copy link

Code coverage report:

Package main asoliman/chain-curse-e2e-test
aggregator 39.67% 48.66%
cmd 0.00% 0.00%
committee 100.00% 100.00%
common 53.95% 89.86%
executor 33.96% 56.90%
indexer 32.62% 35.83%
integration 6.26% 6.53%
protocol 41.06% 46.81%
verifier 57.35% 58.66%

@github-actions
Copy link

E2E Smoke Test Results

Test Case Status Duration
TestE2ESmoke/extra_args_v2/src->dst_msg_execution_eoa_receiver pass 6.02s
TestE2ESmoke/extra_args_v2/dst->src_msg_execution_eoa_receiver pass 6.02s
TestE2ESmoke/extra_args_v2/1337->3337_msg_execution_mock_receiver pass 6.02s
TestE2ESmoke/extra_args_v2 pass 18.07s
TestE2ESmoke/extra_args_v3_messaging/EOA_receiver_and_default_committee_verifier pass 6.02s
TestE2ESmoke/extra_args_v3_messaging/EOA_receiver_and_secondary_committee_verifier pass 5.03s
TestE2ESmoke/extra_args_v3_messaging/receiver_w/_secondary_verifier_required pass 5.02s
TestE2ESmoke/extra_args_v3_messaging/receiver_w/_secondary_required_and_tertiary_optional_threshold=1 pass 5.03s
TestE2ESmoke/extra_args_v3_messaging/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_all_three pass 6.03s
TestE2ESmoke/extra_args_v3_messaging/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_default_and_secondary pass 6.02s
TestE2ESmoke/extra_args_v3_messaging/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_default_and_tertiary pass 6.02s
TestE2ESmoke/extra_args_v3_messaging/EOA_receiver_and_default_committee_verifier#01 pass 6.02s
TestE2ESmoke/extra_args_v3_messaging/max_data_size pass 6.02s
TestE2ESmoke/extra_args_v3_messaging pass 51.22s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.6.1_[]_to_BurnMintTokenPool_1.6.1_[])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(LockReleaseTokenPool_1.7.0_[default]_to_LockReleaseTokenPool_1.7.0_[default])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.7.0_[default]_to_BurnMintTokenPool_1.7.0_[default])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.6.1_[]_to_BurnMintTokenPool_1.7.0_[default])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.7.0_[default]_to_BurnMintTokenPool_1.6.1_[])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.7.0_[default_secondary]_to_BurnMintTokenPool_1.7.0_[default_secondary])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.7.0_[]_to_BurnMintTokenPool_1.7.0_[])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_(TEST_(BurnMintTokenPool_1.7.0_[secondary]_to_BurnMintTokenPool_1.7.0_[secondary])) pass 7.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_1.7.0_(TEST_(LockReleaseTokenPool_1.7.0_[default]_to_LockReleaseTokenPool_1.7.0_[default]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_mock_receiver_and_token_transfer_1.7.0_(TEST_(LockReleaseTokenPool_1.7.0_[default]_to_LockReleaseTokenPool_1.7.0_[default]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[default]_to_BurnMintTokenPool_1.7.0_[default]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_mock_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[default]_to_BurnMintTokenPool_1.7.0_[default]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[default_secondary]_to_BurnMintTokenPool_1.7.0_[default_secondary]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_mock_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[default_secondary]_to_BurnMintTokenPool_1.7.0_[default_secondary]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[]_to_BurnMintTokenPool_1.7.0_[]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_mock_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[]_to_BurnMintTokenPool_1.7.0_[]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_EOA_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[secondary]_to_BurnMintTokenPool_1.7.0_[secondary]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer/src_dst_msg_execution_with_mock_receiver_and_token_transfer_1.7.0_(TEST_(BurnMintTokenPool_1.7.0_[secondary]_to_BurnMintTokenPool_1.7.0_[secondary]))_default_finality pass 8.03s
TestE2ESmoke/extra_args_v3_token_transfer pass 136.57s
TestE2ESmoke pass 205.94s
TestRMNCurseLaneVerifierSide fail 65.57s
TestRMNGlobalCurseVerifierSide fail 66.5s

Full logs are available in the workflow artifacts.

Comment on lines +135 to +139
// ApplyCurse applies a curse to the specified subjects on the given chain.
ApplyCurse(ctx context.Context, chainSelector uint64, subjects [][16]byte) error

// ApplyUncurse removes a curse from the specified subjects on the given chain.
ApplyUncurse(ctx context.Context, chainSelector uint64, subjects [][16]byte) error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can just be Curse and Uncurse as they are already verbs

Comment on lines +1701 to +1714
rmnRemoteAddr, err := m.getRMNRemoteAddress(chainSelector)
if err != nil {
return err
}

ethClient, ok := m.ethClients[chainSelector]
if !ok {
return fmt.Errorf("eth client not found for chain %d", chainSelector)
}

rmnRemote, err := rmn_remote_binding.NewRMNRemote(rmnRemoteAddr, ethClient)
if err != nil {
return fmt.Errorf("failed to create RMN Remote contract binding: %w", err)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull this out into a private helper method getRMNRemote so that you can use from the Curse and Uncurse methods

return fmt.Errorf("deployer key not found for chain %d", chainSelector)
}

// Set context for transaction
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useless comment?

Uint64("chain", chainSelector).
Str("tx", tx.Hash().Hex()).
Int("numSubjects", len(subjects)).
Msg("Applied curse on chain")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Msg("Applied curse on chain")
Msg("Cursed subjects on chain")

m.logger.Info().
Uint64("chain", chainSelector).
Str("tx", tx.Hash().Hex()).
Int("numSubjects", len(subjects)).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The subjects are pertinent to this function, would be nice to see them (maybe as uint64's and not [16]byte)

chain0, chain1, chain2 := selectors[0], selectors[1], selectors[2]
receiver := mustGetEOAReceiverAddress(t, c, chain1)

sentEvt := testCtx.MustSendMessage(chain0, chain1, receiver, 150) // Use custom finality to slow down picking for verification
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has the makings of a flakey test because if we somehow speed up the mining even more this test may start randomly failing?

More effective would be to use an env.toml file that doesn't have super fast finality?

Comment on lines +750 to +754
// We sleep here because in reality we'll need to replay events in case of curses to pick up the dropped tasks
// Increased wait time for CI environments where services may need more time to catch up
time.Sleep(15 * time.Second)

testCtx.MustExecuteMessage(chain0, chain1, receiver, 0) // finality=0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what happens to sentEvt, this first message sent in this test?

Comment on lines +734 to +735
l.Info().Msg("Asserting baseline message reaches verifier but gets dropped due to curse")
testCtx.AssertMessageReachedAndDroppedInVerifier(messageID, 100*time.Second)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like we need to come up with a better way to assert things like this more than ever, since this is a pretty long timeout (longer than half the other tests in TestSmoke combined)

Comment on lines +82 to +88
// DefaultMessageFields creates a simple message with test data.
func DefaultMessageFields(receiver protocol.UnknownAddress) cciptestinterfaces.MessageFields {
return cciptestinterfaces.MessageFields{
Receiver: receiver,
Data: []byte("test-msg"),
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't need to be exported

// MustSendMessage sends a message and waits for the sent event.
// If finality is 0, uses version 2 (no finality config).
// If finality is non-zero, uses version 3 with the specified finality config.
func (tc *TestingContext) MustSendMessage(srcChain, destChain uint64, receiver protocol.UnknownAddress, finality uint16) cciptestinterfaces.MessageSentEvent {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really like to not have these helpers - feels like this could be part of the top-level interface instead.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can define DefaultMessageFields as a test-local thing instead of something thats hardcoded in the method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants