-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for equi_width_bins
function to compute histogram boundaries
#12574
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ptimizer can optimize branches away
…at the optimizer can optimize branches away" This reverts commit 4455c46.
This is awesome to have built into the database. However, I fid the numbers that Vega generates a bit easier to interpret. For example
Compared to Vega You can play with Vega's binning here: https://observablehq.com/@domoritz/histogram-parameterized. |
Thanks for the feedback and interactive link - I've played around with this for a bit and have made a PR to make the boundaries a bit nicer still - see #12878 |
renovate bot
referenced
this pull request
in d-issy/dotfiles
Sep 9, 2024
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [duckdb/duckdb](https://redirect.github.com/duckdb/duckdb) | minor | `v1.0.0` -> `v1.1.0` | --- ### Release Notes <details> <summary>duckdb/duckdb (duckdb/duckdb)</summary> ### [`v1.1.0`](https://redirect.github.com/duckdb/duckdb/releases/tag/v1.1.0): DuckDB 1.1.0 "Eatoni" [Compare Source](https://redirect.github.com/duckdb/duckdb/compare/v1.0.0...v1.1.0) This release of DuckDB is named "Eatoni" after Eaton's pintail (Anas Eatoni) from the southern Indian Ocean. Please also refer to the announcement blog post: https://duckdb.org/2024/09/09/announcing-duckdb-110 #### What's Changed - Add feature changes back in by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/11146](https://redirect.github.com/duckdb/duckdb/pull/11146) - Make `MultiFileReader` filename configurable by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/11178](https://redirect.github.com/duckdb/duckdb/pull/11178) - \[Dev] Fix compilation issues on `feature` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11082](https://redirect.github.com/duckdb/duckdb/pull/11082) - add query() and query_table() functions by [@​chrisiou](https://redirect.github.com/chrisiou) in [https://github.com/duckdb/duckdb/pull/10586](https://redirect.github.com/duckdb/duckdb/pull/10586) - \[Block Size] Move the block allocation size into the block manager by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/11176](https://redirect.github.com/duckdb/duckdb/pull/11176) - LIMIT pushdown below PROJECT by [@​jeewonhh](https://redirect.github.com/jeewonhh) in [https://github.com/duckdb/duckdb/pull/11112](https://redirect.github.com/duckdb/duckdb/pull/11112) - BUGFIX: IN () filter with one argument should translate to = filter. by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/11473](https://redirect.github.com/duckdb/duckdb/pull/11473) - Regression Script should calculate micro benchmark differences with the correct base branch by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/11762](https://redirect.github.com/duckdb/duckdb/pull/11762) - Pushdown filters on window partitions by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/10932](https://redirect.github.com/duckdb/duckdb/pull/10932) - Arrow ListView Type by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/10766](https://redirect.github.com/duckdb/duckdb/pull/10766) - Add scalar function support to the C API by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/11786](https://redirect.github.com/duckdb/duckdb/pull/11786) - Add TopN optimization in physical plan mapping by [@​kryonix](https://redirect.github.com/kryonix) in [https://github.com/duckdb/duckdb/pull/11290](https://redirect.github.com/duckdb/duckdb/pull/11290) - Join-dependent filter derivation by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/11272](https://redirect.github.com/duckdb/duckdb/pull/11272) - Implement `ROW_GROUPS_PER_FILE` for Parquet by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/11249](https://redirect.github.com/duckdb/duckdb/pull/11249) - Prefer Final projected columns on probe side if cardinalities are similar by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/11109](https://redirect.github.com/duckdb/duckdb/pull/11109) - Propagate unused columns to distinct on by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/11006](https://redirect.github.com/duckdb/duckdb/pull/11006) - Separate eviction queues by `FileBufferType` by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/11417](https://redirect.github.com/duckdb/duckdb/pull/11417) - Disable false positive for vector size nightly in test by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/11953](https://redirect.github.com/duckdb/duckdb/pull/11953) - Rework jemalloc extension by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/11891](https://redirect.github.com/duckdb/duckdb/pull/11891) - Tweak jemalloc config by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12034](https://redirect.github.com/duckdb/duckdb/pull/12034) - Httpfs test to nightly by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12196](https://redirect.github.com/duckdb/duckdb/pull/12196) - Removed three reinterpret casts and some rewriting by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12200](https://redirect.github.com/duckdb/duckdb/pull/12200) - Begin Profiling Rework to move towards Modularity by [@​maiadegraaf](https://redirect.github.com/maiadegraaf) in [https://github.com/duckdb/duckdb/pull/11101](https://redirect.github.com/duckdb/duckdb/pull/11101) - \[CLI] Add highlighting + limited auto-complete for shell dot commands by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12201](https://redirect.github.com/duckdb/duckdb/pull/12201) - Skip test to fix block size nightly and add more explicit error checking by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12211](https://redirect.github.com/duckdb/duckdb/pull/12211) - Remove BLOCK_ALLOC_SIZE from the column segment files by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/11474](https://redirect.github.com/duckdb/duckdb/pull/11474) - \[Julia] - Added optional `schema` input argument to `DuckDB.Appender` constructor by [@​curtd](https://redirect.github.com/curtd) in [https://github.com/duckdb/duckdb/pull/12174](https://redirect.github.com/duckdb/duckdb/pull/12174) - Fix Mark Index in the Bound Join Ref by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12263](https://redirect.github.com/duckdb/duckdb/pull/12263) - Fix for CI Regression Failure by [@​maiadegraaf](https://redirect.github.com/maiadegraaf) in [https://github.com/duckdb/duckdb/pull/12273](https://redirect.github.com/duckdb/duckdb/pull/12273) - 🦆 by [@​samansmink](https://redirect.github.com/samansmink) in [https://github.com/duckdb/duckdb/pull/12303](https://redirect.github.com/duckdb/duckdb/pull/12303) - Disable `JEMALLOC_RETAIN` by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12185](https://redirect.github.com/duckdb/duckdb/pull/12185) - Enforce compression extensions for CSV Files by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/11903](https://redirect.github.com/duckdb/duckdb/pull/11903) - Make spuriously failing test more robust by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12306](https://redirect.github.com/duckdb/duckdb/pull/12306) - Add new extensions to issue template by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12313](https://redirect.github.com/duckdb/duckdb/pull/12313) - \[Fix] Block size nightly run by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12283](https://redirect.github.com/duckdb/duckdb/pull/12283) - Spell Check | Nothing Major | Corrected base_scanner.cpp by [@​nj7](https://redirect.github.com/nj7) in [https://github.com/duckdb/duckdb/pull/12282](https://redirect.github.com/duckdb/duckdb/pull/12282) - add duckdb_bind_timestamp_tz function to C API by [@​karlseguin](https://redirect.github.com/karlseguin) in [https://github.com/duckdb/duckdb/pull/12151](https://redirect.github.com/duckdb/duckdb/pull/12151) - \[Python] Add some date/datetime functions to pyspark api by [@​mariotaddeucci](https://redirect.github.com/mariotaddeucci) in [https://github.com/duckdb/duckdb/pull/12075](https://redirect.github.com/duckdb/duckdb/pull/12075) - Fixes to Windows workflow and ubuntu\_18 action by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12308](https://redirect.github.com/duckdb/duckdb/pull/12308) - \[Extension Dev] Forward declare re2 in `hive_partitioning.hpp` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12302](https://redirect.github.com/duckdb/duckdb/pull/12302) - add expected errors to test/sql/copy/per_thread_output.test by [@​hmeriann](https://redirect.github.com/hmeriann) in [https://github.com/duckdb/duckdb/pull/12280](https://redirect.github.com/duckdb/duckdb/pull/12280) - Issue [#​12287](https://redirect.github.com/duckdb/duckdb/issues/12287): ICU Strptime Lists by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12295](https://redirect.github.com/duckdb/duckdb/pull/12295) - Issue [#​12171](https://redirect.github.com/duckdb/duckdb/issues/12171): Streaming Window FILTER by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12250](https://redirect.github.com/duckdb/duckdb/pull/12250) - \[Python] Update the Connection wrapper generation, now generates c++ code by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12216](https://redirect.github.com/duckdb/duckdb/pull/12216) - Use iterator buffer position when storing buffer handles by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12315](https://redirect.github.com/duckdb/duckdb/pull/12315) - Bump Julia client to v0.10.3 by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12323](https://redirect.github.com/duckdb/duckdb/pull/12323) - Fix [#​12286](https://redirect.github.com/duckdb/duckdb/issues/12286) - in the MetadataManager, prefer to allocate new blocks if the next free block id is smaller than the currently used metadata block by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12318](https://redirect.github.com/duckdb/duckdb/pull/12318) - \[Fix] Only read file size if file handle still exists by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12319](https://redirect.github.com/duckdb/duckdb/pull/12319) - Add support for APPEND argument to hive partitioned write by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12262](https://redirect.github.com/duckdb/duckdb/pull/12262) - Remove all reinterpret casts from the transformer by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12320](https://redirect.github.com/duckdb/duckdb/pull/12320) - Additional check for overlapping CTE names by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12305](https://redirect.github.com/duckdb/duckdb/pull/12305) - \[Dev] `STANDARD_VECTOR_SIZE` and `BLOCK_ALLOC_SIZE` can now be set through the Makefile by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12164](https://redirect.github.com/duckdb/duckdb/pull/12164) - \[Upsert] Fix issue with lambdas in `DO UPDATE SET` expressions by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11866](https://redirect.github.com/duckdb/duckdb/pull/11866) - \[Python] Fix scoping issue for `pandas_analyze_sample` setting by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11706](https://redirect.github.com/duckdb/duckdb/pull/11706) - Support REGEX matches expected error message by [@​hmeriann](https://redirect.github.com/hmeriann) in [https://github.com/duckdb/duckdb/pull/12327](https://redirect.github.com/duckdb/duckdb/pull/12327) - Allow run_fuzzer to reduce multi statements. by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/12278](https://redirect.github.com/duckdb/duckdb/pull/12278) - Fix [#​12328](https://redirect.github.com/duckdb/duckdb/issues/12328) - when flattening STRUCT vectors with NULL values, we need to flatten the children recursively as well by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12332](https://redirect.github.com/duckdb/duckdb/pull/12332) - Make `dbgen` generate data in parallel by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12337](https://redirect.github.com/duckdb/duckdb/pull/12337) - dbgen: skip parallel generation if DUCKDB_NO_THREADS is set by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12341](https://redirect.github.com/duckdb/duckdb/pull/12341) - Add prefix prefix_front_back. to get prefix_front\_ and prefix_back\_ by [@​liujiayi771](https://redirect.github.com/liujiayi771) in [https://github.com/duckdb/duckdb/pull/12344](https://redirect.github.com/duckdb/duckdb/pull/12344) - Issue [#​12171](https://redirect.github.com/duckdb/duckdb/issues/12171): Streaming Windowed DISTINCT by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12311](https://redirect.github.com/duckdb/duckdb/pull/12311) - Update README by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12357](https://redirect.github.com/duckdb/duckdb/pull/12357) - \[CSV Reader] \[Skip Option] Tests and fixes by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12213](https://redirect.github.com/duckdb/duckdb/pull/12213) - Adjust BM25 score in FTS extension to prevent negative scores by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12356](https://redirect.github.com/duckdb/duckdb/pull/12356) - Fix typos by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12360](https://redirect.github.com/duckdb/duckdb/pull/12360) - Fix [#​12293](https://redirect.github.com/duckdb/duckdb/issues/12293) - accept NULL values in generate_series with timestamp by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12367](https://redirect.github.com/duckdb/duckdb/pull/12367) - Fix [#​12335](https://redirect.github.com/duckdb/duckdb/issues/12335): avoid calling fsync when writing Parquet files, instead just close the file by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12371](https://redirect.github.com/duckdb/duckdb/pull/12371) - Fix parameters passed down to other workflows in OnTag.yml by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12369](https://redirect.github.com/duckdb/duckdb/pull/12369) - \[Python] Fixes for the SQLLogicTest runner implementation by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12372](https://redirect.github.com/duckdb/duckdb/pull/12372) - Bump julia to v1.0.0 by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12379](https://redirect.github.com/duckdb/duckdb/pull/12379) - Fix [#​11921](https://redirect.github.com/duckdb/duckdb/issues/11921) - varchar -> timestamp casts are not invertible by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12376](https://redirect.github.com/duckdb/duckdb/pull/12376) - Upgrade utf8proc - and move our custom extensions out of utf8proc itself by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12373](https://redirect.github.com/duckdb/duckdb/pull/12373) - change max_queries number back to 2000 by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/12375](https://redirect.github.com/duckdb/duckdb/pull/12375) - Remove sqlsmith extension by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/12300](https://redirect.github.com/duckdb/duckdb/pull/12300) - Reorder semi and anti joins. by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/11815](https://redirect.github.com/duckdb/duckdb/pull/11815) - Issue [#​12351](https://redirect.github.com/duckdb/duckdb/issues/12351): implicit cast to `TIMESTAMP_MS`, `TIMESTAMP_S`, `TIMESTAMP_NS` from `DATE` values by [@​akoshchiy](https://redirect.github.com/akoshchiy) in [https://github.com/duckdb/duckdb/pull/12352](https://redirect.github.com/duckdb/duckdb/pull/12352) - Issue [#​10023](https://redirect.github.com/duckdb/duckdb/issues/10023): Approx_Count_Distinct Memory Usage by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12355](https://redirect.github.com/duckdb/duckdb/pull/12355) - Fix a small typo in dev instructions for swift setup by [@​gjmwoods](https://redirect.github.com/gjmwoods) in [https://github.com/duckdb/duckdb/pull/12383](https://redirect.github.com/duckdb/duckdb/pull/12383) - Release lock before returning `BufferHandle` in `StandardBufferManager::Pin` by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12391](https://redirect.github.com/duckdb/duckdb/pull/12391) - Remote attach autoload by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12393](https://redirect.github.com/duckdb/duckdb/pull/12393) - Add JSON type to Parquet reader/writer by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12222](https://redirect.github.com/duckdb/duckdb/pull/12222) - Add `RETURN_FILES` parameter to `COPY TO` by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12220](https://redirect.github.com/duckdb/duckdb/pull/12220) - Updated JoinHashTable to use linear probing to resolve hash collisions by [@​gropaul](https://redirect.github.com/gropaul) in [https://github.com/duckdb/duckdb/pull/11472](https://redirect.github.com/duckdb/duckdb/pull/11472) - \[Benchmark Runner] Add `--disable-timeout` flag by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12387](https://redirect.github.com/duckdb/duckdb/pull/12387) - Don't replace unicode spaces within `$$` quotes in query strings by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12405](https://redirect.github.com/duckdb/duckdb/pull/12405) - \[Python] Fix fatal exception caused by empty Pandas Categorical objects. by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12370](https://redirect.github.com/duckdb/duckdb/pull/12370) - Release CSV Blocks when acquiring new blocks if single threaded by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12409](https://redirect.github.com/duckdb/duckdb/pull/12409) - Add support for prefetching multiple adjacent blocks in a single batched read when attaching to remote databases by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12413](https://redirect.github.com/duckdb/duckdb/pull/12413) - MatchRegex() fixed to do not return false positive result by [@​hmeriann](https://redirect.github.com/hmeriann) in [https://github.com/duckdb/duckdb/pull/12396](https://redirect.github.com/duckdb/duckdb/pull/12396) - Expected errors 2053 by [@​hmeriann](https://redirect.github.com/hmeriann) in [https://github.com/duckdb/duckdb/pull/12392](https://redirect.github.com/duckdb/duckdb/pull/12392) - \[C-API] Catch exception in `duckdb_execute_prepared` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12414](https://redirect.github.com/duckdb/duckdb/pull/12414) - Combining LIST_CONCAT and CONCAT binding by [@​maiadegraaf](https://redirect.github.com/maiadegraaf) in [https://github.com/duckdb/duckdb/pull/12317](https://redirect.github.com/duckdb/duckdb/pull/12317) - \[Appender] Add `AppendDefault` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11905](https://redirect.github.com/duckdb/duckdb/pull/11905) - \[Python Dev] Push CTE internally for every (python) replacement scan that occurred. by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12161](https://redirect.github.com/duckdb/duckdb/pull/12161) - Improve compiler compatibility by [@​krlmlr](https://redirect.github.com/krlmlr) in [https://github.com/duckdb/duckdb/pull/12401](https://redirect.github.com/duckdb/duckdb/pull/12401) - Write zero-length list offsets for NULL values when serializing vectors by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12423](https://redirect.github.com/duckdb/duckdb/pull/12423) - Get column statistics if Logical Get has a statistics function by [@​jeewonhh](https://redirect.github.com/jeewonhh) in [https://github.com/duckdb/duckdb/pull/12424](https://redirect.github.com/duckdb/duckdb/pull/12424) - jemalloc: Identify GNU source code properly by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12420](https://redirect.github.com/duckdb/duckdb/pull/12420) - Avoid parallelizing LIMIT clauses when the query plan is simple by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12433](https://redirect.github.com/duckdb/duckdb/pull/12433) - Prefetch metadata blocks for remote files by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12437](https://redirect.github.com/duckdb/duckdb/pull/12437) - \[Jupyter] Remove width limit on the BoxRenderer config by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12443](https://redirect.github.com/duckdb/duckdb/pull/12443) - Revert [#​10865](https://redirect.github.com/duckdb/duckdb/issues/10865) by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12426](https://redirect.github.com/duckdb/duckdb/pull/12426) - inline delta by [@​samansmink](https://redirect.github.com/samansmink) in [https://github.com/duckdb/duckdb/pull/12435](https://redirect.github.com/duckdb/duckdb/pull/12435) - Account for *tagged* dollar-quoted strings when stripping unicode spaces by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12421](https://redirect.github.com/duckdb/duckdb/pull/12421) - Work-around for broken github windows runner by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12447](https://redirect.github.com/duckdb/duckdb/pull/12447) - Prevents clearing of the types of the LogicalExecute operator by [@​NiclasHaderer](https://redirect.github.com/NiclasHaderer) in [https://github.com/duckdb/duckdb/pull/12436](https://redirect.github.com/duckdb/duckdb/pull/12436) - Add support for BEGIN TRANSACTION READ ONLY by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12202](https://redirect.github.com/duckdb/duckdb/pull/12202) - Make `range` and `generate_series` table in-out functions, and fix several issues with table in-out functions by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12431](https://redirect.github.com/duckdb/duckdb/pull/12431) - Issue [#​12412](https://redirect.github.com/duckdb/duckdb/issues/12412): AsOf Filter Push by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12448](https://redirect.github.com/duckdb/duckdb/pull/12448) - \[Fix] Block Size Nightly by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12427](https://redirect.github.com/duckdb/duckdb/pull/12427) - \[ART] Remove Flatten and template key generation by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12428](https://redirect.github.com/duckdb/duckdb/pull/12428) - \[Python] Clean up internals of `execute` / `executemany` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12434](https://redirect.github.com/duckdb/duckdb/pull/12434) - By default attach remote databases as READ_ONLY by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12461](https://redirect.github.com/duckdb/duckdb/pull/12461) - Fix [#​11837](https://redirect.github.com/duckdb/duckdb/issues/11837): use internal physical type for FIRST/LAST/ANY_VALUE instead of logical type by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12462](https://redirect.github.com/duckdb/duckdb/pull/12462) - Issue [#​12464](https://redirect.github.com/duckdb/duckdb/issues/12464): Windowed Order By All by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12470](https://redirect.github.com/duckdb/duckdb/pull/12470) - Specialize `list_value` for primitive types for significantly improved performance by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12468](https://redirect.github.com/duckdb/duckdb/pull/12468) - \[Dev] Remove dead code from `PhysicalBatchCopyToFile` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12459](https://redirect.github.com/duckdb/duckdb/pull/12459) - Disable Windows extensions CI until Github actions runners are fixed by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12479](https://redirect.github.com/duckdb/duckdb/pull/12479) - \[Fix] access_mode now lives in AttachOptions by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12482](https://redirect.github.com/duckdb/duckdb/pull/12482) - Internal [#​2186](https://redirect.github.com/duckdb/duckdb/issues/2186): Nanosecond Functionality by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12440](https://redirect.github.com/duckdb/duckdb/pull/12440) - \[C-API] Fix leak in `duckdb_create_config` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12465](https://redirect.github.com/duckdb/duckdb/pull/12465) - \[Python] No longer scan the entire frame lineage in a replacement scan, added option to disable (python) replacements entirely by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12425](https://redirect.github.com/duckdb/duckdb/pull/12425) - throw binder error for comment on system catalog by [@​samansmink](https://redirect.github.com/samansmink) in [https://github.com/duckdb/duckdb/pull/12486](https://redirect.github.com/duckdb/duckdb/pull/12486) - Parquet reader performance by [@​lnkuiper](https://redirect.github.com/lnkuiper) in [https://github.com/duckdb/duckdb/pull/12478](https://redirect.github.com/duckdb/duckdb/pull/12478) - Operators the Optimizer can skip by [@​Tmonster](https://redirect.github.com/Tmonster) in [https://github.com/duckdb/duckdb/pull/12489](https://redirect.github.com/duckdb/duckdb/pull/12489) - Fixes clang conversion warnings by [@​TinyTinni](https://redirect.github.com/TinyTinni) in [https://github.com/duckdb/duckdb/pull/12467](https://redirect.github.com/duckdb/duckdb/pull/12467) - Avoid creating internal schemas as non-internal when reading old database files by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12456](https://redirect.github.com/duckdb/duckdb/pull/12456) - Allow parquet encryption/decryption keys to be passed in as base64 encoded strings by [@​elefeint](https://redirect.github.com/elefeint) in [https://github.com/duckdb/duckdb/pull/12445](https://redirect.github.com/duckdb/duckdb/pull/12445) - \[Block Size] Introducing CompressionInfo by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12481](https://redirect.github.com/duckdb/duckdb/pull/12481) - add the number of filtered files to explain by [@​samansmink](https://redirect.github.com/samansmink) in [https://github.com/duckdb/duckdb/pull/12488](https://redirect.github.com/duckdb/duckdb/pull/12488) - Implement Map Type Detection for JSON Reader by [@​ZiyaZa](https://redirect.github.com/ZiyaZa) in [https://github.com/duckdb/duckdb/pull/11285](https://redirect.github.com/duckdb/duckdb/pull/11285) - \[Dev] Remove busy-spin from `ClientContext::ExecuteTaskInternal` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12483](https://redirect.github.com/duckdb/duckdb/pull/12483) - Pluggable collations by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12492](https://redirect.github.com/duckdb/duckdb/pull/12492) - \[Dev] Don't fail `make generate-files` if the python code generation fails by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12500](https://redirect.github.com/duckdb/duckdb/pull/12500) - Optimize `EXTRACT(year/month/day FROM date/timestamp)` by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12499](https://redirect.github.com/duckdb/duckdb/pull/12499) - \[Fix] Remove BLOCK_ALLOC_SIZE in the single file block manager by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12502](https://redirect.github.com/duckdb/duckdb/pull/12502) - Revert Windows CI fixes by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12510](https://redirect.github.com/duckdb/duckdb/pull/12510) - Fix [#​12467](https://redirect.github.com/duckdb/duckdb/issues/12467) changes to covariance calculation by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12515](https://redirect.github.com/duckdb/duckdb/pull/12515) - \[Python] Fix reading strided `datetime` and `timedelta` columns by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12519](https://redirect.github.com/duckdb/duckdb/pull/12519) - Add method for decoding sort keys, and use this in min/max for arbitrary types by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12520](https://redirect.github.com/duckdb/duckdb/pull/12520) - Reduce allocations & use predication in ColumnSegment::FilterSelection by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12521](https://redirect.github.com/duckdb/duckdb/pull/12521) - Skip only built-in optimizers by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12522](https://redirect.github.com/duckdb/duckdb/pull/12522) - Improve min/max performance for strings and fallback types by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12524](https://redirect.github.com/duckdb/duckdb/pull/12524) - Move arg_min/arg_max to use sort keys by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12525](https://redirect.github.com/duckdb/duckdb/pull/12525) - Move FIRST/LAST/ANY_VALUE to use sort keys by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12526](https://redirect.github.com/duckdb/duckdb/pull/12526) - CMake: use GNUInstallDirs as defaults for INSTALL\_{BIN,LIB,INCLUDE}\_DIR by [@​paparodeo](https://redirect.github.com/paparodeo) in [https://github.com/duckdb/duckdb/pull/12509](https://redirect.github.com/duckdb/duckdb/pull/12509) - More formatting and fix to stddev by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12516](https://redirect.github.com/duckdb/duckdb/pull/12516) - Linux Extensions CI: Attempt at fix missing dependencies by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12429](https://redirect.github.com/duckdb/duckdb/pull/12429) - Fix checkouts by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12366](https://redirect.github.com/duckdb/duckdb/pull/12366) - Etag if none match for extension install by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12333](https://redirect.github.com/duckdb/duckdb/pull/12333) - \[Block Size] FixedSizeAllocator, MetadataManager, PartialBlockManager by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12514](https://redirect.github.com/duckdb/duckdb/pull/12514) - \[Python] Skip the PandasAnalyzer if dtype is `'string'` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12511](https://redirect.github.com/duckdb/duckdb/pull/12511) - \[StreamQueryResult] Batched variant of the StreamQueryResult collector by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11494](https://redirect.github.com/duckdb/duckdb/pull/11494) - Move many tests to slow by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12534](https://redirect.github.com/duckdb/duckdb/pull/12534) - Add support for `arg_min(ANY, ANY)` by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12532](https://redirect.github.com/duckdb/duckdb/pull/12532) - Avoid overriding types in PrepareTypeForCast when not required by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12539](https://redirect.github.com/duckdb/duckdb/pull/12539) - Support all types in `histogram` function by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12538](https://redirect.github.com/duckdb/duckdb/pull/12538) - \[Python] Remove busy-spin during execution by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12512](https://redirect.github.com/duckdb/duckdb/pull/12512) - \[Block Size] String space constant by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12537](https://redirect.github.com/duckdb/duckdb/pull/12537) - Use string_t instead of std::string in histogram by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12545](https://redirect.github.com/duckdb/duckdb/pull/12545) - Add support for binned histograms by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12548](https://redirect.github.com/duckdb/duckdb/pull/12548) - \[Upsert] Fix RETURNING for `DO NOTHING` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12554](https://redirect.github.com/duckdb/duckdb/pull/12554) - Build Android Binaries by [@​hannes](https://redirect.github.com/hannes) in [https://github.com/duckdb/duckdb/pull/12550](https://redirect.github.com/duckdb/duckdb/pull/12550) - \[CI] Remove pyarrow version lock by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12566](https://redirect.github.com/duckdb/duckdb/pull/12566) - \[Dev] Change tests: np.NaN -> np.nan by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12565](https://redirect.github.com/duckdb/duckdb/pull/12565) - Internal [#​2017](https://redirect.github.com/duckdb/duckdb/issues/2017): DECIMAL Downcast Rounding by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12036](https://redirect.github.com/duckdb/duckdb/pull/12036) - Issue [#​12204](https://redirect.github.com/duckdb/duckdb/issues/12204): Summarize Temporal Quantiles by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12297](https://redirect.github.com/duckdb/duckdb/pull/12297) - Internal [#​2186](https://redirect.github.com/duckdb/duckdb/issues/2186): Nanosecond StrTimeFormat by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12551](https://redirect.github.com/duckdb/duckdb/pull/12551) - Add support for `equi_width_bins` function to compute histogram boundaries by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12574](https://redirect.github.com/duckdb/duckdb/pull/12574) - add support for casting 'yes'/'no' strings to boolean values by [@​chrisiou](https://redirect.github.com/chrisiou) in [https://github.com/duckdb/duckdb/pull/12501](https://redirect.github.com/duckdb/duckdb/pull/12501) - Julia: Add chunked results with Tables.partitions() by [@​frankier](https://redirect.github.com/frankier) in [https://github.com/duckdb/duckdb/pull/12395](https://redirect.github.com/duckdb/duckdb/pull/12395) - \[PySpark] - Allow spark session range by [@​mariotaddeucci](https://redirect.github.com/mariotaddeucci) in [https://github.com/duckdb/duckdb/pull/12346](https://redirect.github.com/duckdb/duckdb/pull/12346) - \[PySpark] Implement subset drop duplicates by [@​mariotaddeucci](https://redirect.github.com/mariotaddeucci) in [https://github.com/duckdb/duckdb/pull/12348](https://redirect.github.com/duckdb/duckdb/pull/12348) - ICU noaccent collation by [@​tiagokepe](https://redirect.github.com/tiagokepe) in [https://github.com/duckdb/duckdb/pull/12170](https://redirect.github.com/duckdb/duckdb/pull/12170) - Implement Brotli compression for Parquet reading & writing by [@​hannes](https://redirect.github.com/hannes) in [https://github.com/duckdb/duckdb/pull/12103](https://redirect.github.com/duckdb/duckdb/pull/12103) - \[FriendlySQL] Unpacked COLUMNS() Expression by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/11872](https://redirect.github.com/duckdb/duckdb/pull/11872) - \[PySpark] Implement UDFRegistration.register method on PySpark api by [@​mariotaddeucci](https://redirect.github.com/mariotaddeucci) in [https://github.com/duckdb/duckdb/pull/12179](https://redirect.github.com/duckdb/duckdb/pull/12179) - \[Python] Don't use `np.nan`, deprecated alias starting with NumPy 2.0 by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12583](https://redirect.github.com/duckdb/duckdb/pull/12583) - Add `bind_expression` callback to scalar function, and use it to turn `typeof` into a `BoundConstantExpression` by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12580](https://redirect.github.com/duckdb/duckdb/pull/12580) - Add `can_cast_implicitly` scalar function by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12581](https://redirect.github.com/duckdb/duckdb/pull/12581) - Add support for `histogram` and `histogram_values` table macro, and add support for default table macros (similar to how we support default macros) by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12590](https://redirect.github.com/duckdb/duckdb/pull/12590) - build: swap libclang for cxxheaderparser by [@​Mause](https://redirect.github.com/Mause) in [https://github.com/duckdb/duckdb/pull/12567](https://redirect.github.com/duckdb/duckdb/pull/12567) - \[C-API] Add `table_description` struct to query various information about the table. by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12460](https://redirect.github.com/duckdb/duckdb/pull/12460) - Change new micro benchmark script to only look for `.benchmark` files by [@​maiadegraaf](https://redirect.github.com/maiadegraaf) in [https://github.com/duckdb/duckdb/pull/12598](https://redirect.github.com/duckdb/duckdb/pull/12598) - Add HTTP error code to extension install failures by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12608](https://redirect.github.com/duckdb/duckdb/pull/12608) - Separate WAL write from commit, and allow writing to the WAL without holding the transaction lock by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12261](https://redirect.github.com/duckdb/duckdb/pull/12261) - Add `OwningStringMap` - and rework `histogram` and `mode` functions to use this by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12601](https://redirect.github.com/duckdb/duckdb/pull/12601) - Feature [#​1272](https://redirect.github.com/duckdb/duckdb/issues/1272): Window Executor State by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12573](https://redirect.github.com/duckdb/duckdb/pull/12573) - Add support for any type to `mode` aggregate by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12619](https://redirect.github.com/duckdb/duckdb/pull/12619) - WAL - when dropping a table, also delete any transaction local storage associated with that table by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12603](https://redirect.github.com/duckdb/duckdb/pull/12603) - \[Python] Allow Generators to be passed where List is expected by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12602](https://redirect.github.com/duckdb/duckdb/pull/12602) - VectorOperations::Copy - fast path when copying an aligned flat validity mask into a flat vector by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12618](https://redirect.github.com/duckdb/duckdb/pull/12618) - Move android CI to only run during nightly CI triggers by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12622](https://redirect.github.com/duckdb/duckdb/pull/12622) - Add initial support for GeoParquet + Bump spatial by [@​Maxxen](https://redirect.github.com/Maxxen) in [https://github.com/duckdb/duckdb/pull/12503](https://redirect.github.com/duckdb/duckdb/pull/12503) - Issue [#​12600](https://redirect.github.com/duckdb/duckdb/issues/12600): Streaming Positive LAG by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12609](https://redirect.github.com/duckdb/duckdb/pull/12609) - Feature [#​1272](https://redirect.github.com/duckdb/duckdb/issues/1272): Window Group Preparation by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12628](https://redirect.github.com/duckdb/duckdb/pull/12628) - Minor window improvements by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12617](https://redirect.github.com/duckdb/duckdb/pull/12617) - Merge feature into main by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12633](https://redirect.github.com/duckdb/duckdb/pull/12633) - Refactor `quantile` aggregate - clean up code & support `quantile_disc`/`median` for all types by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12630](https://redirect.github.com/duckdb/duckdb/pull/12630) - Feature 1272: Window Payload Preallocation by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12629](https://redirect.github.com/duckdb/duckdb/pull/12629) - \[ART] Configurable index scan threshold by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12635](https://redirect.github.com/duckdb/duckdb/pull/12635) - Subtract start offset for when fetching array child segment by [@​Maxxen](https://redirect.github.com/Maxxen) in [https://github.com/duckdb/duckdb/pull/12639](https://redirect.github.com/duckdb/duckdb/pull/12639) - Remove custom logic to detect main vs feature by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12643](https://redirect.github.com/duckdb/duckdb/pull/12643) - Do not quote fields with space in the CSV output mode by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12644](https://redirect.github.com/duckdb/duckdb/pull/12644) - Use lowercase in 'html' output mode by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12612](https://redirect.github.com/duckdb/duckdb/pull/12612) - Internal [#​2361](https://redirect.github.com/duckdb/duckdb/issues/2361): Window ROWS Overflow by [@​hawkfish](https://redirect.github.com/hawkfish) in [https://github.com/duckdb/duckdb/pull/12652](https://redirect.github.com/duckdb/duckdb/pull/12652) - Quantile: Fix variable used only in D_ASSERT by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12642](https://redirect.github.com/duckdb/duckdb/pull/12642) - Skip pytorch test, it fails spuriously in CI by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12645](https://redirect.github.com/duckdb/duckdb/pull/12645) - Add `histogram_exact` function that adds values to bins only if they match exactly, and add `other` column that contains values that do not fit in any bin by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12650](https://redirect.github.com/duckdb/duckdb/pull/12650) - Add operator hook for sink progress by [@​Maxxen](https://redirect.github.com/Maxxen) in [https://github.com/duckdb/duckdb/pull/12637](https://redirect.github.com/duckdb/duckdb/pull/12637) - Regression workflow on newly introduced benchmarks: remove for now by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12659](https://redirect.github.com/duckdb/duckdb/pull/12659) - Fix [#​12646](https://redirect.github.com/duckdb/duckdb/issues/12646) - allow SQL value functions in HAVING by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12654](https://redirect.github.com/duckdb/duckdb/pull/12654) - Add != operators on string_t and interval_t by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12658](https://redirect.github.com/duckdb/duckdb/pull/12658) - fix: improve C scalar functions API by [@​rustyconover](https://redirect.github.com/rustyconover) in [https://github.com/duckdb/duckdb/pull/12663](https://redirect.github.com/duckdb/duckdb/pull/12663) - Add `approx_top_k` aggregate based on the (Filtered) Space-Saving algorithm, and use it in histogram by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12653](https://redirect.github.com/duckdb/duckdb/pull/12653) - Fix std::sort requirements, from greater_equal to greater by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12669](https://redirect.github.com/duckdb/duckdb/pull/12669) - fix(parquet): two-complement zeroes check on FIXED_BYTE_ARRAY encoded DECIMAL ([#​12621](https://redirect.github.com/duckdb/duckdb/issues/12621)) by [@​fedefrancescon](https://redirect.github.com/fedefrancescon) in [https://github.com/duckdb/duckdb/pull/12655](https://redirect.github.com/duckdb/duckdb/pull/12655) - \[CSV Reader] Reorder of Columns for CSV Scans on multiple files. by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12288](https://redirect.github.com/duckdb/duckdb/pull/12288) - \[CSV] \[Bug-Fix] Fix for issue related with single-threaded execution and null padding. by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12679](https://redirect.github.com/duckdb/duckdb/pull/12679) - \[Block Size] String block limit and a few other places by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12671](https://redirect.github.com/duckdb/duckdb/pull/12671) - Rework arena allocator allocation policy - and increase pivot threshold by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12690](https://redirect.github.com/duckdb/duckdb/pull/12690) - Julia - Fix Base.isopen(db::DB) in [https://github.com/duckdb/duckdb/pull/12700](https://redirect.github.com/duckdb/duckdb/pull/12700) - \[CLI] Limit history size to 100MB, and avoid writing invalid UTF8 to the CLI history by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12677](https://redirect.github.com/duckdb/duckdb/pull/12677) - Add configurable thresholds for using nested loop join and merge join by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12689](https://redirect.github.com/duckdb/duckdb/pull/12689) - Prevent unnecessary usage of `std::string` in `list` aggregate - and use more efficient `memcpy` for batched copy by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12694](https://redirect.github.com/duckdb/duckdb/pull/12694) - Dont load spatial unless geoparquet metata is present by [@​Maxxen](https://redirect.github.com/Maxxen) in [https://github.com/duckdb/duckdb/pull/12692](https://redirect.github.com/duckdb/duckdb/pull/12692) - Serialization: add CustomData and better support for integrating with extensions by [@​jeewonhh](https://redirect.github.com/jeewonhh) in [https://github.com/duckdb/duckdb/pull/12681](https://redirect.github.com/duckdb/duckdb/pull/12681) - Removing ODBC driver by [@​hannes](https://redirect.github.com/hannes) in [https://github.com/duckdb/duckdb/pull/12706](https://redirect.github.com/duckdb/duckdb/pull/12706) - Support thousand separator for floating point numbers by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12717](https://redirect.github.com/duckdb/duckdb/pull/12717) - \[Python] Use non-owning references to hold created cursors by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12711](https://redirect.github.com/duckdb/duckdb/pull/12711) - LIST(VARCHAR) - reduce memory usage by avoiding allocation of nullmask for string data, and allocate larger initial batches by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12705](https://redirect.github.com/duckdb/duckdb/pull/12705) - \[CSV] Bug fix for race condition in single-threaded multifile reader + properly print paths on union_by_name errors. by [@​pdet](https://redirect.github.com/pdet) in [https://github.com/duckdb/duckdb/pull/12697](https://redirect.github.com/duckdb/duckdb/pull/12697) - Issue template: Add ODBC and Node (neo) clients by [@​szarnyasg](https://redirect.github.com/szarnyasg) in [https://github.com/duckdb/duckdb/pull/12714](https://redirect.github.com/duckdb/duckdb/pull/12714) - Shell: add .sql suffix to temporary file created with \e by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12723](https://redirect.github.com/duckdb/duckdb/pull/12723) - Partitioned write - keep only up until 100 files open, when this limit is exceeded close the file and create a new file if more data for this partition appears by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12708](https://redirect.github.com/duckdb/duckdb/pull/12708) - Change setting types to fix warnings by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12724](https://redirect.github.com/duckdb/duckdb/pull/12724) - Avoid unnecessarily copying child expression when binding COLLATE statements by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12725](https://redirect.github.com/duckdb/duckdb/pull/12725) - Support for variadic arguments in scalar UDFs in the C API by [@​taniabogatsch](https://redirect.github.com/taniabogatsch) in [https://github.com/duckdb/duckdb/pull/12678](https://redirect.github.com/duckdb/duckdb/pull/12678) - \[Relation API] Dont push DISTINCT modifier for EXCEPT/INTERSECT ALL by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12599](https://redirect.github.com/duckdb/duckdb/pull/12599) - Builds for Windows on ARM64 by [@​hannes](https://redirect.github.com/hannes) in [https://github.com/duckdb/duckdb/pull/12586](https://redirect.github.com/duckdb/duckdb/pull/12586) - Rework `union_by_name` so that files are no longer kept open by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12730](https://redirect.github.com/duckdb/duckdb/pull/12730) - Fix [#​12729](https://redirect.github.com/duckdb/duckdb/issues/12729): early-out when checking for perfect hash joins when running on empty tables by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12731](https://redirect.github.com/duckdb/duckdb/pull/12731) - CLI: Replace \n with \r\n again in history again by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12735](https://redirect.github.com/duckdb/duckdb/pull/12735) - Fix [#​11228](https://redirect.github.com/duckdb/duckdb/issues/11228) - add support for unsigned integers in printf/format by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12736](https://redirect.github.com/duckdb/duckdb/pull/12736) - Various CI fixes by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12737](https://redirect.github.com/duckdb/duckdb/pull/12737) - Add repeat(LIST\[], INT) that allows repetition of lists similar to how this is allowed in Python by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12738](https://redirect.github.com/duckdb/duckdb/pull/12738) - \[Python] Add missing options to `read_json` method by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12732](https://redirect.github.com/duckdb/duckdb/pull/12732) - Add support for fetching cardinality estimation and stats through a multifilelist by [@​samansmink](https://redirect.github.com/samansmink) in [https://github.com/duckdb/duckdb/pull/12740](https://redirect.github.com/duckdb/duckdb/pull/12740) - Fixes warnings detected by cppcheck by [@​carlopi](https://redirect.github.com/carlopi) in [https://github.com/duckdb/duckdb/pull/12745](https://redirect.github.com/duckdb/duckdb/pull/12745) - \[Arrow] Add `ArrowQueryResult` by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12496](https://redirect.github.com/duckdb/duckdb/pull/12496) - \[Dev] StreamQueryResult internals cleanup by [@​Tishj](https://redirect.github.com/Tishj) in [https://github.com/duckdb/duckdb/pull/12636](https://redirect.github.com/duckdb/duckdb/pull/12636) - ALP/ALPRD: correctly skip when we are skipping fewer values than in a vector by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12753](https://redirect.github.com/duckdb/duckdb/pull/12753) - Maintain prepared statement parameter types explicitly instead of converting into literals by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12759](https://redirect.github.com/duckdb/duckdb/pull/12759) - CLI .changes: use sqlite3\_changes64 and sqlite3\_totalchanges64 to prevent overflows by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12761](https://redirect.github.com/duckdb/duckdb/pull/12761) - Fix [#​12569](https://redirect.github.com/duckdb/duckdb/issues/12569): avoid truncating zeros that matter in format function by [@​Mytherin](https://redirect.github.com/Mytherin) in [https://github.com/duckdb/duckdb/pull/12762](https://redirect.github.com/duckdb/duckdb/pull/12762) - Fix [#​12418](https://redirect.github.com/duckdb/duckdb/issues/12418): Remove .lint command in SQLite shell by [@​Mytherin](https://redire </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/d-issy/dotfiles). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC41OS4yIiwidXBkYXRlZEluVmVyIjoiMzguNTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up from #12548
This PR adds support for the
equi_width_bins
helper function that can be used to compute histogram boundaries.The function signature is as follows:
The function computes
bin_count
bins betweenmin
andmax
. Ifnice_rounding
is set to true the bins generated will be "nice". That essentially means we will try to round the numbers in a way that is more appealing/easier to read. Below are some examples:The exact formula for what a nice number is depends on the data type:
2*10^p
or5*10^p
wherep=floor(log10(n))
. In other words, for numbers in the range of100..999
, nice numbers are divisible by either 20 or 50, i.e. 120 is a nice number, 115 is not (this was inspired by vega - https://github.com/vega/vega/blob/master/packages/vega-statistics/src/bin.js).Together with the previously introduced binned histogram function, this allows you to relatively easily compute histograms without having to manually specify boundaries, e.g.:
Stay tuned for some more friendly helper functions in a subsequent PR.