Improve metadata bwc test for Logical Replication #354
Conversation
tests/bwc/test_rolling_upgrade.py
Outdated
| # Set up tables for logical replications | ||
| if int(path.from_version.split('.')[0]) >= 5 and int(path.from_version.split('.')[1]) >= 10: | ||
| c.execute("create table doc.x (a int) clustered into 1 shards with (number_of_replicas=0)") | ||
| expected_active_shards += 1 | ||
| c.execute("create publication p for table doc.x") | ||
| with connect(replica_cluster.node().http_url, error_trace=True) as replica_conn: | ||
| rc = replica_conn.cursor() | ||
| rc.execute("create table doc.rx (a int) clustered into 1 shards with (number_of_replicas=0)") | ||
| rc.execute("create publication rp for table doc.rx") | ||
| rc.execute(f"create subscription rs connection 'crate://localhost:{cluster.node().addresses.transport.port}?user=crate&sslmode=sniff' publication p") | ||
| wait_for_active_shards(rc) | ||
| c.execute(f"create subscription s connection 'crate://localhost:{replica_cluster.node().addresses.transport.port}?user=crate&sslmode=sniff' publication rp") | ||
| wait_for_active_shards(c) |
There was a problem hiding this comment.
If I remove the calls to wait_for_active_shards and move onto rolling upgrades immediately, I observe unexpected behaviours like UnavailableShardsException or the number of rows replicated do not add up correctly. But to my knowledge, users are recommended to wait for active shards before upgrading, so this is not an issue?
jeeminso
force-pushed
the
jeeminso/lr
branch
3 times, most recently
from
a47df1d to
e4b1b28
Compare
This comment was marked as resolved.
This comment was marked as resolved.
|
It is an intermittent behaviour(was able to reproduce but rarely on the latest master from 1 node cluster to 1 node cluster) that does seem |
tests/bwc/test_rolling_upgrade.py
Outdated
| c.execute("insert into doc.x values (1)") | ||
| rc.execute("insert into doc.rx values (1)") | ||
|
|
||
| rc.execute("select count(*) from doc.x") |
There was a problem hiding this comment.
crate.client.exceptions.ProgrammingError: RelationUnknown[Relation 'doc.x' unknown]
io.crate.exceptions.RelationUnknown: Relation 'doc.x' unknown
at io.crate.exceptions.RelationUnknown.of(RelationUnknown.java:46)
Guessing it means that the DROP stmt succeeded, looking into it.
|
The first commit tests LR during rolling upgrade
Hi @seut could you take a look? BTW, this problem is intermittent especially when tried to reproduce manually. |
|
@jeeminso |
|
I've looked into this and ran the related tests multiple times locally.
I do not understand yet what the real issue is but it feels very timing related. Such it does not break in general, all manual test I did are working as expected. I can also not see why #17960 would cause such issue, I think this was just coincidence and the same flaky failure can may be seen even without this change, also I did not test this. I'd followup on this at a later point to debug it more deeply, if no one else figured this out until then. |
jeeminso
force-pushed
the
jeeminso/lr
branch
2 times, most recently
from
5aa84d9 to
180e243
Compare
Summary of the changes / Why this is an improvement
Checklist