Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCORE-2213 Add CI builder that runs tests with log level set to all #7934

Merged
merged 3 commits into from
Aug 7, 2024

Conversation

jbreams
Copy link
Contributor

@jbreams jbreams commented Jul 30, 2024

What, How & Why?

This adds a builder that runs as part of the nightly build that runs our test suites with logging set to all. Because the log output can be 100s of MB per run, we only capture the logs if the tests fail, and we upload them directly to s3 rather than capturing them inline in the evergreen logs.

This also fixes some UBSAN failures and one bug that only manifested when trace level logging was enabled.

☑️ ToDos

  • 📝 Changelog update
  • 🚦 Tests (or not relevant)
  • C-API, if public C++ API changed
  • bindgen/spec.yml, if public C++ API changed

@@ -215,13 +215,16 @@ void ServerProtocol::insert_single_changeset_download_message(OutputBuffer& out,
entry.changeset.write_to(out);

if (logger.would_log(util::Logger::Level::trace)) {
util::AppendBuffer<char> changeset_buffer;
entry.changeset.copy_to(changeset_buffer);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compaction code removed in RCORE-2205 did a bunch of decoding/re-encoding of changesets before they got to be logged here, so entry.changeset.get_first_chunk() was only valid if the ChunkedBinaryData in changeset wrapped a single BinaryData. We just copy the whole (potentially chunked) changeset into a buffer here to simulate the old behavior.

int n = (4 * L) / 146097;
int i, j;
uint64_t L = jd + 68569;
uint64_t n = (4 * L) / 146097;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was undefined behavior where in some tests that log timestamps this multiplication could overflow an int. Switching to uint64_t's where overflowing is defined behavior.

@@ -242,7 +242,7 @@ TEST(Alloc_BadBuffer)
GROUP_TEST_PATH(path);

// Produce an invalid buffer
char buffer[32];
alignas(8) char buffer[32];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UBSAN was complaining about this while testing. The buffer gets reinterpret_cast'd into a pointer to a struct while trying to validate it as a realm buffer, so that storage needs to be aligned.

- name: ubuntu-trace-logging
display_name: "Ubuntu (Trace Logging Enabled)"
run_on: ubuntu2204-arm64-large
allowed_requesters: [ "patch", "ad_hoc" ]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now this runs in the nightly build. Bad behavior in trace-level logging is something we should catch but isn't something I think we need to try to catch on every PR patch build.

Copy link

coveralls-official bot commented Jul 30, 2024

Pull Request Test Coverage Report for Build jonathan.reams_3391

Details

  • 7 of 10 (70.0%) changed or added relevant lines in 3 files are covered.
  • 76 unchanged lines in 16 files lost coverage.
  • Overall coverage decreased (-4.3%) to 86.779%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/noinst/protocol_codec.cpp 0 3 0.0%
Files with Coverage Reduction New Missed Lines %
src/realm/sort_descriptor.cpp 1 94.06%
src/realm/sync/noinst/server/server_history.cpp 1 62.92%
src/realm/util/file.cpp 1 84.84%
test/test_dictionary.cpp 1 99.83%
test/test_table.cpp 1 99.51%
src/realm/cluster.cpp 2 75.85%
src/realm/query_expression.cpp 2 86.62%
src/realm/sync/network/http.hpp 2 82.27%
src/realm/sync/transform.cpp 2 61.05%
src/realm/sync/client.cpp 3 90.96%
Totals Coverage Status
Change from base Build 2546: -4.3%
Covered Lines: 40451
Relevant Lines: 46614

💛 - Coveralls

@jbreams jbreams requested review from tgoyne and jsflax July 31, 2024 15:36
@jbreams jbreams merged commit 603dcf5 into master Aug 7, 2024
46 checks passed
@jbreams jbreams deleted the jbr/all_logging_build_variant branch August 7, 2024 17:38
@github-actions github-actions bot mentioned this pull request Aug 9, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants