[pull] main from PeerDB-io:main #5
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Current design of this page where we await slot and stats data on the server side prevents us from editing peers when the connectivity to the peer is down - the page crashes Refactored the wiring here to still render the page in this scenario and show what can be shown --------- Co-authored-by: Kevin Biju <52661649+heavycrystal@users.noreply.github.com> Co-authored-by: Kevin Biju <kevin@peerdb.io>
Avoids postgres_fdw enriching queries with syntax MySQL doesn't allow since MySQL timestamps always have tzinfo DateTime is also the more suitable time type in MySQL, & StarRocks only has DateTime
identified this issue while debugging something completely different
If initial load only is selected in UI, allow non-mirrorable tables to be synced, and validation will not check replication and publication related things. Functionally tested One item I tried was also allowing excluding primary key columns when initial load only is selected, but seems like this requires some refactoring on the pull side. Could do it in this PR or follow up
 This PR: - Adds drop mirror stats (present in `peerdb_stats` schema) functionality to drop mirror. The failure of this step is not fatal to `drop_flow`. Exposed in UI as seen above. - Resync now drops mirror stats - In query layer, drop mirror does not drop stats. Extending the drop mirror command syntax to support this will be done if there's demand Also: - Adds a column - `parent_mirror_name (TEXT)` to `qrep_runs` and `qrep_partitions` tables. - Adds a field - `parentMirrorName (string)` to QRep Config proto - Adaps `monitoring.go` and initial load stats endpoint to use parent mirror name Parent mirror name is the name of the parent peer flow in case of initial load clone. It is the same as flow name for QRep and XMin mirrors - [ ] Functionally tested
### Overview Resync is becoming an increasingly utilized feature. Users of our connectors, particularly Clickhouse, sometimes wish to make small table definition changes on target or source and then want to repopulate data. This is alongside the usual recovery use-cases of resync. ### Failure points `RenameTables` (or resync in general) currently can fail in the following ways: 1. If, between original mirror kick off and resync, a column was added to the table and a row not inserted after, then that column would not have been added to the target via schema changes. In this case, upon resync, the `_resync` table and the original table have different schemas, causing the soft-delete transfer step to fail. This can then lead to: 2. It processes some tables but not all and hits a failure - upon retrying it tries to resync the first table again but the `_resync `table of it is dropped since it succeeded before. 3. Resyncing again midway through a resync (initial load) can result in duplicate data in the `_resync` table if initial load of that `_resync` table was done already. ### Fixes In light of these scenarios, this PR puts in place the following guards: - Perform `CREATE OR REPLACE` in `SetupNormalize()` if it is being called in a resync - If the `_resync` table does not exist, skip rename for the table - If the original target table does not exist, skip just the soft-delete transfer step TODO: - [x] Functionally test the failure points mentioned above for the warehouse peers. - [x] Also implement clearing of stats after resync : Done by #2029
Co-authored-by: Kaushik Iska <iska.kaushik@gmail.com>
forgot to commit in #2023
For table box in UI's table picker for CDC create flow: - Have a field for selecting ReplacingMergeTree vs MergeTree - Number field for each column. This will be used to order the columns in the ORDER BY clause while creating Clickhouse target tables Name/Type not exposed to ui for now Co-authored-by: Amogh-Bharadwaj <amogh@peerdb.io>
Mistake in #2029 where there's an extra where in the delete command
The `checkIfTableExists` function needs the input - schema name and table name - to be in the right case. Tested this on Snowflake and also with a mirror Also we should not skip `SetupNormalize` if the mirror is a resync mirror so that we can REPLACE the table downstream
Fixes #2037 Accounts for CH Cloud
Validation to existing table which peerdb created fails because in SetupNormalize we lower case synced_at column. So that needs to be done in validation as well because by default it is upper case PEERDB_SYNCED_AT
Account for mixed case table names in publication creation validation
For non-postgres destinations, type system must never be PG --------- Co-authored-by: Philip Dubé <serprex@users.noreply.github.com>
Resync currently does: 1. Drop mirror 2. Create mirror - which in turn does Validate mirror If validate mirror fails here for some reason (an example of which is fixed in this PR), then the UX is completely messed up because the mirror is dropped and the user does not have any idea of what went wrong This PR calls validate mirror explicitly in the Resync endpoint before dropping the mirror, returning any validation errors. The subsequent validate call in Create Mirror is put behind a !resync guard so that this new mirror gets created Second thing this PR does is not perform destination table emptiness and engine check for Clickhouse mirror validation in case of a resync, because those tables will get swapped out with the _resync tables whose structure and emptiness PeerDB fully controls because of CREATE OR REPLACE Thirdly, this PR improves UI/UX of resync in PeerDB UI to now keep the resync dialog open so that all user-facing messages are visible. Resync and close buttons are disabled until resync has been kicked off. Messages changed to be more accurate (Ex: Resyncing... changed to Preparing resync..) <img width="1549" alt="Screenshot 2024-09-05 at 4 10 29 AM" src="https://github.com/user-attachments/assets/826ad26c-7a83-4dd2-b899-a0a58d2c5df5"> <img width="1593" alt="Screenshot 2024-09-05 at 4 33 49 AM" src="https://github.com/user-attachments/assets/3abbda8d-0503-41f3-ad78-bbb327102460">
Do no skip table creation in case of resync mirror because we need it to be REPLACEd downstream
Adds a link to resync peerdb documentation: PeerDB-io/docs#174 Also keeps the close resync button enabled
value can be overwritten by the next iteration which creates an incorrect result.
Our otel manager currently breaks because of schema url version mismatch
<img width="1067" alt="Screenshot 2024-09-08 at 7 13 05 AM" src="https://github.com/user-attachments/assets/4c8a157a-7cab-4620-accf-a07e3fe46ba9">
There's engines like Distributed and other MergeTree family which could also be used in special workloads. Until we have more understanding on how these work and how common they are, don't block creation of mirrors.
bit of code cleanup along with adding an otel counter of postgres cdc payload sizes
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [rsa](https://redirect.github.com/RustCrypto/RSA) | dependencies | patch | `0.9.6` -> `0.9.7` | | [rustls](https://redirect.github.com/rustls/rustls) | dependencies | patch | `0.23.17` -> `0.23.18` | --- ### Release Notes <details> <summary>RustCrypto/RSA (rsa)</summary> ### [`v0.9.7`](https://redirect.github.com/RustCrypto/RSA/compare/v0.9.6...v0.9.7) [Compare Source](https://redirect.github.com/RustCrypto/RSA/compare/v0.9.6...v0.9.7) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xOS4wIiwidXBkYXRlZEluVmVyIjoiMzkuMTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | elasticsearch | service | patch | `8.16.0` -> `8.16.1` | | [github/codeql-action](https://redirect.github.com/github/codeql-action) | action | digest | `ea9e4e3` -> `f09c1c0` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xOS4wIiwidXBkYXRlZEluVmVyIjoiMzkuMTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Need to register tstsrange qvalue type which we newly introduced in gob
- add tags to telemetry to provide additional visibility - later these can be added to otel metrics etc
Currently flow-worker panics due to assignment to nil map entry at the getOrInitMetric call in the int64 counter because the map is nil
### Overview [A recent effort ](https://github.com/PeerDB-io/peerdb/pull/2256/files) was made to improve PeerDB's ingestion performance into ClickHouse by parallelizing the INSERT INTO SELECT queries which took data from the raw table and put it into the final tables. Just before this step, PeerDB moves data from S3 to the raw table. ### Problem In this implementation, every insert runs on a new ClickHouse session - which means the inserts could be running on different nodes/replicas. In this case, there exists a non-zero chance (albeit very unliikely) that the INSERT INTO SELECT does not read the rows in the raw table inserted by the first step. This is documented here: https://clickhouse.com/docs/en/cloud/reference/shared-merge-tree#consistency ### Solution This PR adds the setting `select_sequential_consistency = 1` in the `Connect` function which we use for connecting to ClickHouse
This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [@types/node](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/node) ([source](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node)) | [`22.9.0` -> `22.10.0`](https://renovatebot.com/diffs/npm/@types%2fnode/22.9.0/22.10.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [less](http://lesscss.org) ([source](https://redirect.github.com/less/less.js)) | [`4.2.0` -> `4.2.1`](https://renovatebot.com/diffs/npm/less/4.2.0/4.2.1) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [lucide-react](https://lucide.dev) ([source](https://redirect.github.com/lucide-icons/lucide/tree/HEAD/packages/lucide-react)) | [`^0.460.0` -> `^0.462.0`](https://renovatebot.com/diffs/npm/lucide-react/0.460.0/0.462.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [material-symbols](https://marella.github.io/material-symbols/demo/) ([source](https://redirect.github.com/marella/material-symbols/tree/HEAD/material-symbols)) | [`0.27.0` -> `0.27.1`](https://renovatebot.com/diffs/npm/material-symbols/0.27.0/0.27.1) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [postcss](https://postcss.org/) ([source](https://redirect.github.com/postcss/postcss)) | [`8.4.47` -> `8.4.49`](https://renovatebot.com/diffs/npm/postcss/8.4.47/8.4.49) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [prettier](https://prettier.io) ([source](https://redirect.github.com/prettier/prettier)) | [`3.3.3` -> `3.4.1`](https://renovatebot.com/diffs/npm/prettier/3.3.3/3.4.1) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [tailwindcss](https://tailwindcss.com) ([source](https://redirect.github.com/tailwindlabs/tailwindcss)) | [`3.4.14` -> `3.4.15`](https://renovatebot.com/diffs/npm/tailwindcss/3.4.14/3.4.15) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [typescript](https://www.typescriptlang.org/) ([source](https://redirect.github.com/microsoft/TypeScript)) | [`5.6.3` -> `5.7.2`](https://renovatebot.com/diffs/npm/typescript/5.6.3/5.7.2) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>less/less.js (less)</summary> ### [`v4.2.1`](https://redirect.github.com/less/less.js/blob/HEAD/CHANGELOG.md#v421-2024-09-26) [Compare Source](https://redirect.github.com/less/less.js/compare/v4.2.0...1df9072ee9ebdadc791bf35dfb1dbc3ef9f1948f) - [#​4237](https://redirect.github.com/less/less.js/pull/4237) Fix [#​4235](https://redirect.github.com/less/less.js/issues/4235) container style queries extra space resolved ([@​puckowski](https://redirect.github.com/puckowski)) </details> <details> <summary>lucide-icons/lucide (lucide-react)</summary> ### [`v0.462.0`](https://redirect.github.com/lucide-icons/lucide/releases/tag/0.462.0): New icons 0.462.0 [Compare Source](https://redirect.github.com/lucide-icons/lucide/compare/0.461.0...0.462.0) #### New icons 🎨 - `image-upscale` ([#​2462](https://redirect.github.com/lucide-icons/lucide/issues/2462)) by [@​jguddas](https://redirect.github.com/jguddas) #### Modified Icons 🔨 - `grid-2x2` ([#​2628](https://redirect.github.com/lucide-icons/lucide/issues/2628)) by [@​jguddas](https://redirect.github.com/jguddas) - `ship` ([#​2548](https://redirect.github.com/lucide-icons/lucide/issues/2548)) by [@​jguddas](https://redirect.github.com/jguddas) - `shuffle` ([#​2478](https://redirect.github.com/lucide-icons/lucide/issues/2478)) by [@​jguddas](https://redirect.github.com/jguddas) - `venetian-mask` ([#​1950](https://redirect.github.com/lucide-icons/lucide/issues/1950)) by [@​jguddas](https://redirect.github.com/jguddas) ### [`v0.461.0`](https://redirect.github.com/lucide-icons/lucide/releases/tag/0.461.0): New icons 0.461.0 [Compare Source](https://redirect.github.com/lucide-icons/lucide/compare/0.460.0...0.461.0) #### New icons 🎨 - `calendar-sync` ([#​2590](https://redirect.github.com/lucide-icons/lucide/issues/2590)) by [@​chessurisme](https://redirect.github.com/chessurisme) #### Modified Icons 🔨 - `scale-3d` ([#​2627](https://redirect.github.com/lucide-icons/lucide/issues/2627)) by [@​jguddas](https://redirect.github.com/jguddas) </details> <details> <summary>marella/material-symbols (material-symbols)</summary> ### [`v0.27.1`](https://redirect.github.com/marella/material-symbols/compare/v0.27.0...v0.27.1) [Compare Source](https://redirect.github.com/marella/material-symbols/compare/v0.27.0...v0.27.1) </details> <details> <summary>postcss/postcss (postcss)</summary> ### [`v8.4.49`](https://redirect.github.com/postcss/postcss/blob/HEAD/CHANGELOG.md#8449) [Compare Source](https://redirect.github.com/postcss/postcss/compare/8.4.48...8.4.49) - Fixed custom syntax without `source.offset` (by [@​romainmenke](https://redirect.github.com/romainmenke)). ### [`v8.4.48`](https://redirect.github.com/postcss/postcss/blob/HEAD/CHANGELOG.md#8448) [Compare Source](https://redirect.github.com/postcss/postcss/compare/8.4.47...8.4.48) - Fixed position calculation in error/warnings methods (by [@​romainmenke](https://redirect.github.com/romainmenke)). </details> <details> <summary>prettier/prettier (prettier)</summary> ### [`v3.4.1`](https://redirect.github.com/prettier/prettier/blob/HEAD/CHANGELOG.md#341) [Compare Source](https://redirect.github.com/prettier/prettier/compare/3.4.0...3.4.1) [diff](https://redirect.github.com/prettier/prettier/compare/3.4.0...3.4.1) ##### Remove unnecessary parentheses around assignment in `v-on` ([#​16887](https://redirect.github.com/prettier/prettier/pull/16887) by [@​fisker](https://redirect.github.com/fisker)) <!-- prettier-ignore --> ```vue <!-- Input --> <template> <button @​click="foo += 2">Click</button> </template> <!-- Prettier 3.4.0 --> <template> <button @​click="(foo += 2)">Click</button> </template> <!-- Prettier 3.4.1 --> <template> <button @​click="foo += 2">Click</button> </template> ``` ### [`v3.4.0`](https://redirect.github.com/prettier/prettier/blob/HEAD/CHANGELOG.md#340) [Compare Source](https://redirect.github.com/prettier/prettier/compare/3.3.3...3.4.0) [diff](https://redirect.github.com/prettier/prettier/compare/3.3.3...3.4.0) 🔗 [Release Notes](https://prettier.io/blog/2024/11/26/3.4.0.html) </details> <details> <summary>tailwindlabs/tailwindcss (tailwindcss)</summary> ### [`v3.4.15`](https://redirect.github.com/tailwindlabs/tailwindcss/releases/tag/v3.4.15) [Compare Source](https://redirect.github.com/tailwindlabs/tailwindcss/compare/v3.4.14...v3.4.15) - Bump versions for security vulnerabilities ([#​14697](https://redirect.github.com/tailwindlabs/tailwindcss/pull/14697)) - Ensure the TypeScript types for the `boxShadow` theme configuration allows arrays ([#​14856](https://redirect.github.com/tailwindlabs/tailwindcss/pull/14856)) - Set fallback for opacity variables to ensure setting colors with the `selection:*` variant works in Chrome 131 ([#​15003](https://redirect.github.com/tailwindlabs/tailwindcss/pull/15003)) </details> <details> <summary>microsoft/TypeScript (typescript)</summary> ### [`v5.7.2`](https://redirect.github.com/microsoft/TypeScript/compare/v5.6.3...d701d908d534e68cfab24b6df15539014ac348a3) [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.6.3...v5.7.2) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xOS4wIiwidXBkYXRlZEluVmVyIjoiMzkuMTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates: | Package | Update | Change | |---|---|---| | ghcr.io/peerdb-io/flow-api | pinDigest | -> `a759b2d` | | ghcr.io/peerdb-io/flow-snapshot-worker | pinDigest | -> `894c1fe` | | ghcr.io/peerdb-io/flow-worker | pinDigest | -> `4482314` | | ghcr.io/peerdb-io/peerdb-server | pinDigest | -> `c736500` | | ghcr.io/peerdb-io/peerdb-ui | pinDigest | -> `ffc4b59` | | postgres | digest | `0d96245` -> `e7897ba` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xOS4wIiwidXBkYXRlZEluVmVyIjoiMzkuMTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [docker/dockerfile](https://redirect.github.com/moby/buildkit) | syntax | minor | `1.11` -> `1.12` | | lukemathwalker/cargo-chef | stage | digest | `a539f69` -> `5b4cc6b` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS40Mi40IiwidXBkYXRlZEluVmVyIjoiMzkuNDIuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
### Why Unbounded numerics in Postgres databases statistically are unlikely to have values with dimensions larger than 76,38. Currently by default unbounded `numeric` in postgres is mapped to String in ClickHouse - an unideal mapping from data nativity point of view and only useful for the scarce cases mentioned above ### What This PR puts this mapping choice behind a dynamic setting flag whose default is false - thereby restoring the default mapping to numeric to Decimal in all cases Adapted the E2E test for this --------- Co-authored-by: Philip Dubé <philip@peerdb.io>
This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [@types/node](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/node) ([source](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node)) | [`22.10.0` -> `22.10.1`](https://renovatebot.com/diffs/npm/@types%2fnode/22.10.0/22.10.1) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS40Mi40IiwidXBkYXRlZEluVmVyIjoiMzkuNDIuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Philip Dubé <serprex@users.noreply.github.com>
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [bytes](https://redirect.github.com/tokio-rs/bytes) | dependencies | minor | `1.8.0` -> `1.9.0` | | [cargo-deb](https://lib.rs/crates/cargo-deb) ([source](https://redirect.github.com/kornelski/cargo-deb)) | dependencies | patch | `2.9.0` -> `2.9.1` | | [pgwire](https://redirect.github.com/sunng87/pgwire) | workspace.dependencies | minor | `0.26` -> `0.27` | | [rustls](https://redirect.github.com/rustls/rustls) | dependencies | patch | `0.23.18` -> `0.23.19` | | [tracing](https://tokio.rs) ([source](https://redirect.github.com/tokio-rs/tracing)) | workspace.dependencies | patch | `0.1.40` -> `0.1.41` | | [tracing-subscriber](https://tokio.rs) ([source](https://redirect.github.com/tokio-rs/tracing)) | dependencies | patch | `0.3.18` -> `0.3.19` | | [ureq](https://redirect.github.com/algesten/ureq) | dependencies | minor | `2.10.1` -> `2.11.0` | --- ### Release Notes <details> <summary>tokio-rs/bytes (bytes)</summary> ### [`v1.9.0`](https://redirect.github.com/tokio-rs/bytes/blob/HEAD/CHANGELOG.md#190-November-27-2024) [Compare Source](https://redirect.github.com/tokio-rs/bytes/compare/v1.8.0...v1.9.0) ##### Added - Add `Bytes::from_owner` to enable externally-allocated memory ([#​742](https://redirect.github.com/tokio-rs/bytes/issues/742)) ##### Documented - Fix typo in Buf::chunk() comment ([#​744](https://redirect.github.com/tokio-rs/bytes/issues/744)) ##### Internal changes - Replace BufMut::put with BufMut::put_slice in Writer impl ([#​745](https://redirect.github.com/tokio-rs/bytes/issues/745)) - Rename hex_impl! to fmt_impl! and reuse it for fmt::Debug ([#​743](https://redirect.github.com/tokio-rs/bytes/issues/743)) </details> <details> <summary>kornelski/cargo-deb (cargo-deb)</summary> ### [`v2.9.1`](https://redirect.github.com/kornelski/cargo-deb/compare/v2.9.0...v2.9.1) [Compare Source](https://redirect.github.com/kornelski/cargo-deb/compare/v2.9.0...v2.9.1) </details> <details> <summary>sunng87/pgwire (pgwire)</summary> ### [`v0.27.0`](https://redirect.github.com/sunng87/pgwire/blob/HEAD/CHANGELOG.md#0270---2024-11-30) [Compare Source](https://redirect.github.com/sunng87/pgwire/compare/v0.26.0...v0.27.0) ##### Added - `on_flush` handler for extended query handler \[[#​220](https://redirect.github.com/sunng87/pgwire/issues/220)] ##### Fixed - `Parse` message encoding \[[#​219](https://redirect.github.com/sunng87/pgwire/issues/219)] ##### Changed - MSRV to 1.75 - Made TLS an optional feature. `server-api` now provides no TLS functionality \[[#​218](https://redirect.github.com/sunng87/pgwire/issues/218)] </details> <details> <summary>tokio-rs/tracing (tracing)</summary> ### [`v0.1.41`](https://redirect.github.com/tokio-rs/tracing/releases/tag/tracing-0.1.41): tracing 0.1.41 [Compare Source](https://redirect.github.com/tokio-rs/tracing/compare/tracing-0.1.40...tracing-0.1.41) \[ [crates.io][crate-0.1.41] ] | \[ [docs.rs][docs-0.1.41] ] This release updates the `tracing-core` dependency to [v0.1.33][core-0.1.33] and the `tracing-attributes` dependency to [v0.1.28][attrs-0.1.28]. ##### Added - **core**: Add index API for `Field` ([#​2820]) - **core**: Allow `&[u8]` to be recorded as event/span field ([#​2954]) ##### Changed - Bump MSRV to 1.63 ([#​2793]) - **core**: Use const `thread_local`s when possible ([#​2838]) ##### Fixed - Removed core imports in macros ([#​2762]) - **attributes**: Added missing RecordTypes for instrument ([#​2781]) - **attributes**: Change order of async and unsafe modifier ([#​2864]) - Fix missing field prefixes ([#​2878]) - **attributes**: Extract match scrutinee ([#​2880]) - Fix non-simple macro usage without message ([#​2879]) - Fix event macros with constant field names in the first position ([#​2883]) - Allow field path segments to be keywords ([#​2925]) - **core**: Fix missed `register_callsite` error ([#​2938]) - **attributes**: Support const values for `target` and `name` ([#​2941]) - Prefix macro calls with ::core to avoid clashing with local macros ([#​3024]) [#​2762]: https://redirect.github.com/tokio-rs/tracing/pull/2762 [#​2781]: https://redirect.github.com/tokio-rs/tracing/pull/2781 [#​2793]: https://redirect.github.com/tokio-rs/tracing/pull/2793 [#​2820]: https://redirect.github.com/tokio-rs/tracing/pull/2820 [#​2838]: https://redirect.github.com/tokio-rs/tracing/pull/2838 [#​2864]: https://redirect.github.com/tokio-rs/tracing/pull/2864 [#​2878]: https://redirect.github.com/tokio-rs/tracing/pull/2878 [#​2879]: https://redirect.github.com/tokio-rs/tracing/pull/2879 [#​2880]: https://redirect.github.com/tokio-rs/tracing/pull/2880 [#​2883]: https://redirect.github.com/tokio-rs/tracing/pull/2883 [#​2925]: https://redirect.github.com/tokio-rs/tracing/pull/2925 [#​2938]: https://redirect.github.com/tokio-rs/tracing/pull/2938 [#​2941]: https://redirect.github.com/tokio-rs/tracing/pull/2941 [#​2954]: https://redirect.github.com/tokio-rs/tracing/pull/2954 [#​3024]: https://redirect.github.com/tokio-rs/tracing/pull/3024 [attrs-0.1.28]: https://redirect.github.com/tokio-rs/tracing/releases/tag/tracing-attributes-0.1.28 [core-0.1.33]: https://redirect.github.com/tokio-rs/tracing/releases/tag/tracing-core-0.1.33 [docs-0.1.41]: https://docs.rs/tracing/0.1.41/tracing/ [crate-0.1.41]: https://crates.io/crates/tracing/0.1.41 </details> <details> <summary>algesten/ureq (ureq)</summary> ### [`v2.11.0`](https://redirect.github.com/algesten/ureq/blob/HEAD/CHANGELOG.md#2110) [Compare Source](https://redirect.github.com/algesten/ureq/compare/2.10.1...2.11.0) - Fixes for changes to cargo-deny ([#​882](https://redirect.github.com/algesten/ureq/issues/882)) - Pin rustls dep on 0.23.19 to keep MSRV 1.67 ([#​878](https://redirect.github.com/algesten/ureq/issues/878)) - Bump MSRV 1.63 -> 1.67 due to time crate ([#​878](https://redirect.github.com/algesten/ureq/issues/878)) - Re-export rustls ([#​813](https://redirect.github.com/algesten/ureq/issues/813)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS40Mi40IiwidXBkYXRlZEluVmVyIjoiMzkuNDIuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Philip Dubé <philip@peerdb.io> Co-authored-by: Kunal Gupta <39487888+iamKunalGupta@users.noreply.github.com>
Co-authored-by: iamKunalGupta <iamKunalGupta@users.noreply.github.com>
closes #2312 however this can cause data to be missed if the schema is missing incorrectly (as opposed to table being dropped) --------- Co-authored-by: Kevin Biju <kevin@peerdb.io>
Co-authored-by: Philip Dubé <serprex@users.noreply.github.com>
Co-authored-by: Luke Judd <luke.judd@luxuryescapes.com>
1. log when we're remapping child table OID to parent table OID 2. partition fetching query also allows non-partitioned tables, extensions use this 3. in case we see an unknown table OID after checking `childToParentRelIDMapping`, check again to ensure the table wasn't created in the middle of CDC 4. remove log since it could log PII
Co-authored-by: iamKunalGupta <iamKunalGupta@users.noreply.github.com> Co-authored-by: Philip Dubé <serprex@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )