[pull] master from cockroachdb:master #1389
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Something I noticed while looking into other stuff. Release note: None
Add a comment to global functions in log package that log to the dev channel. In the future these will be removed and it will be required to specify the log channel that a log will be written to. This is intended to reduce the amount of logs that are sent to DEV by require engineers to be explicit about where they want to log Epic: CRDB-53410 Release note: None
Changes: * [`eaca9d0c`](cockroachdb/pebble@eaca9d0c) strparse: add (*Parser).Offset * [`f8b8a139`](cockroachdb/pebble@f8b8a139) db: include size of input, output files for blob file rewrites * [`d143b0d8`](cockroachdb/pebble@d143b0d8) compact: convert MERGE keys to SET when bottommost Release note: none. Epic: none.
Adds a safe and backwards compatible means of moving logs from one channel to another. The new log.Migrator and log.StructuredEventMigrator provide engineers with a way of safely migrating logs from one channel to another with minimal interruption to customers. Both of these migrators require a `shouldMigrate` callback which is used to determine whether to log to a new or old logging channel. The StructuredEventMigrator only requires a new channel to be specified, which will be used instead of the channel defined on the event itself. Note that the event proto doc string should still be updated to document that it may be logged to a different channel. The Migrator struct requires an old and new channel to write to. Both of the migrators will write to the "new" channel if `shouldMigrate` returns true, otherwise it writes to the "old" channel. Part of: CRDB-53412 Epic: CRDB-53410 Release note: None
…em.sql_instances Previously, DROP REGION cleanup logic for system.sql_instances was unintentionally applied to all databases when in multiregion setup, not just the system database. The original intent was to only perform this cleanup when extending `DROP REGION` for the system database. However, the logic leaked through, affecting user databases as well. This PR implements the fix described in #151789, and after a quick check of these changes with the blocked tests in the #150730, this fixes them plus passes the original tests created for testing drop region in system database. Epic: None Fixes: #151789 Release note: None Co-authored-by: Shubham Dhama <shubham.dhama@cockroachlabs.com>
Previously, backup fixtures were failing to generate due to a misnamed setting. The setting was originally correct, but became incorrect when I renamed the setting to fix the setting name lint. We don't have to backport this because the roachtest change hit a merge conflict during the backport and I removed the roachtest changes from the backport since they were non-essential. Release note: none Informs: #148334
Currently, the `CREATE USER` and the `GRANT role` operations use a bumped descriptor to invalidate caches of the user table. This blocked those operations until long-running transactions using the old table version committed. This change adds special handling of version bumps to wait for visibility of the change rather than convergence across the cluster. It no longer creates (blocking) jobs for the version bumps. Epic: CRDB-49398 Fixes: #138691 Release note (sql change): The `CREATE USER` and `GRANT role` operations now wait for full-cluster visibility of the new user table version rather than blocking on convergence.
151870: sql,multiregion: prevent DROP REGION in user DB from cleaning up `system.sql_instances` r=fqazi a=Nukitt Previously, DROP REGION cleanup logic for system.sql_instances was unintentionally applied to all databases when in multiregion setup, not just the system database. The original intent was to only perform this cleanup when extending `DROP REGION` for the system database. However, the logic leaked through, affecting user databases as well. This PR implements the fix described in #151789, and after a quick check of these changes with the blocked tests in the #150730, this fixes them plus passes the original tests created for testing drop region in system database. Epic: None Fixes: #151789 Release note: None 151936: go.mod: bump Pebble to eaca9d0cc5ea r=jbowens a=jbowens Changes: * [`eaca9d0c`](cockroachdb/pebble@eaca9d0c) strparse: add (*Parser).Offset * [`f8b8a139`](cockroachdb/pebble@f8b8a139) db: include size of input, output files for blob file rewrites * [`d143b0d8`](cockroachdb/pebble@d143b0d8) compact: convert MERGE keys to SET when bottommost Release note: none. Epic: none. 152014: roachtest: fix client retry token bucket setting r=jeffswenson a=jeffswenson Previously, backup fixtures were failing to generate due to a misnamed setting. The setting was originally correct, but became incorrect when I renamed the setting to fix the setting name lint. We don't have to backport this because the roachtest change hit a merge conflict during the backport and I removed the roachtest changes from the backport since they were non-essential. Release note: none Informs: #148334 Co-authored-by: Nukitt <nukit.tailor@cockroachlabs.com> Co-authored-by: Jackson Owens <jackson@cockroachlabs.com> Co-authored-by: Jeff Swenson <jeffswenson@betterthannull.com>
Previously this could queue up a large number of small changes to flush, particularly if flushing was slower than updates. Now all the updates that arrive over a 15s window are rolled up before being saved. If the channel becomes full while saving, additional messages may be dropped. Release note: none. Epic: none.
151693: log: deprecate log.* global functions r=kyle-a-wong a=kyle-a-wong Add a comment to global functions in log package that log to the dev channel. In the future these will be removed and it will be required to specify the log channel that a log will be written to. This is intended to reduce the amount of logs that are sent to DEV by require engineers to be explicit about where they want to log Epic: [CRDB-53410](https://cockroachlabs.atlassian.net/browse/CRDB-53410) Release note: None Co-authored-by: Kyle Wong <37189875+kyle-a-wong@users.noreply.github.com>
In pre-22.2 backups, incremental backups were written to the same directory as the full backup. This was removed in 22.2, but logic was kept in place to maintain backwards compatibility and allow later versions to restore from pre-22.2 backups. However, given that we are now on 25.4 and are many major versions past the removal of the old default location, the legacy logic is now being removed to facilitate the development of backup indexes. Epic: None Release note: None
150747: sql: unblock user table changes from long-running transactions r=bghal a=bghal sql: unblock user table changes from long-running transactions Currently, the `CREATE USER` and the `GRANT role` operations use a bumped descriptor to invalidate caches of the user table. This blocked those operations until long-running transactions using the old table version committed. This change adds special handling of version bumps to wait for visibility of the change rather than convergence across the cluster. It no longer creates (blocking) jobs for the version bumps. Epic: CRDB-49398 Fixes: #138691 Release note (sql change): The `CREATE USER` and `GRANT role` operations now wait for full-cluster visibility of the new user table version rather than blocking on convergence. 152015: backup: remove old incremental backup location lookup r=dt a=kev-cao In pre-22.2 backups, incremental backups were written to the same directory as the full backup. This was removed in 22.2, but logic was kept in place to maintain backwards compatibility and allow later versions to restore from pre-22.2 backups. However, given that we are now on 25.4 and are many major versions past the removal of the old default location, the legacy logic is now being removed to facilitate the development of backup indexes. Epic: None Release note: None Co-authored-by: Brendan Gerrity <brendan.gerrity@cockroachlabs.com> Co-authored-by: Kevin Cao <kevin.cao@cockroachlabs.com>
151806: log: add support for migrating logs to different channels r=kyle-a-wong a=kyle-a-wong Adds a safe and backwards compatible means of moving logs from one channel to another. The new log.Migrator and log.StructuredEventMigrator provide engineers with a way of safely migrating logs from one channel to another with minimal interruption to customers. Both of these migrators require a `shouldMigrate` callback which is used to determine whether to log to a new or old logging channel. The StructuredEventMigrator only requires a new channel to be specified, which will be used instead of the channel defined on the event itself. Note that the event proto doc string should still be updated to document that it may be logged to a different channel. The Migrator struct requires an old and new channel to write to. Both of the migrators will write to the "new" channel if `shouldMigrate` returns true, otherwise it writes to the "old" channel. Part of: CRDB-53412 Epic: CRDB-53410 Release note: None 151950: backup: only flush per-node progress every 15s r=dt a=dt Previously this could queue up a large number of small changes to flush, particularly if flushing was slower than updates. Now all the updates that arrive over a 15s window are rolled up before being saved. If the channel becomes full while saving, additional messages may be dropped. Release note: none. Epic: none. Co-authored-by: Kyle Wong <37189875+kyle-a-wong@users.noreply.github.com> Co-authored-by: David Taylor <tinystatemachine@gmail.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.3)
Can you help keep this open source service alive? 💖 Please sponsor : )