Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pg17 support #348

Merged
merged 12 commits into from
Nov 3, 2024
Merged

pg17 support #348

merged 12 commits into from
Nov 3, 2024

Conversation

var77
Copy link
Collaborator

@var77 var77 commented Oct 29, 2024

No description provided.

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Hi! Looks like you've reached your API usage limit. You can increase it from your account settings page here: app.greptile.com/settings/usage

7 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

…led from post_parse_analyze hook was causing an error `ERROR: ResourceOwnerEnlarge called after release started`.

This was coming from the function call `LookupOperName` which uses `SearchCatCacheList` (src/backend/utils/cache/catcache.c:1754) to search in catalog and ifa match is found it will call `ResourceOwnerEnlarge` to grow the hash if needed.
This error seems to happen when the transaction is reverted because of an error and the list needs to grow.
We will now keep the found operator oids in `CacheMemoryContext` and search them again only if it does not exist.
We will not free the list and let it go after the `CacheMemoryContext` will be reset
Copy link

Benchmarks

metric old new pct change
recall (after create) 0.950 0.954 +0.42%
recall (after insert) 0.000 0.000 -
select tps 23458.117 24793.388 +5.69%
select bulk(100) tps 34.821 35.857 +2.97%
select latency (ms) 0.795 ± 1.722𝜎 0.705 ± 1.470𝜎 -11.32%
select bulk(100) latency (ms) 905.136 ± 111.213𝜎 865.312 ± 107.804𝜎 -4.40%
create latency (ms) 410481.663 404780.443 -1.39%
insert tps 446.769 445.863 -0.20%
insert bulk(100) tps 4.611 4.785 +3.77%
insert latency (ms) 70.766 ± 16.113𝜎 71.158 ± 15.503𝜎 +0.55%
insert bulk(100) latency (ms) 6019.638 ± 917.612𝜎 6507.607 ± 238.602𝜎 +8.11%
disk usage (bytes) 8192008192.000 8192008192.000 -

…ion.

After release of Postgres 17, search_path is restricted to (pg_catalog, pg_temp)
for maintenance operations [ref: postgres/postgres@2af07e2]
So when pgvector was installed in public schema and we will try to get
the oid for vector type using `TypenameGetTypid("vector")` it would return `InvalidOid`
By this change the function `TypenameGetVectorTypid` will call SQL function `get_vector_type_oid`
which will query pg_type table and return the vector type oid
in `InitBuildState` we would try to close that file descriptor (which would be 0)
in `BuildIndexCleanup` function because of condition `if (buildstate->index_file_fd != -1)`
and trying to close fd 0 was crashing the server.
Now we will check if the fd is greater than 0 before closing it.
…xtension when invalid binary would be installed
@var77 var77 force-pushed the varik/pg17-support branch 6 times, most recently from b65017e to 702a254 Compare October 30, 2024 16:04
if enable_seqscan is set to false.
Modify failing tests because of operator rewrite hook to not use <?>
operator.
Modify test output files accordingly.
…rStart_hook` hooks and related codes.

Update tests to not use <?> operator
Copy link

codecov bot commented Nov 1, 2024

Codecov Report

Attention: Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
lantern_hnsw/src/hnsw/utils.c 71.42% 1 Missing and 1 partial ⚠️
lantern_hnsw/src/hnsw.c 50.00% 0 Missing and 1 partial ⚠️
lantern_hnsw/src/hnsw/build.c 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@@ -49,7 +49,8 @@ def __repr__(self):
return self.version

INCOMPATIBLE_VERSIONS = {
'16': [Version('0.0.4')]
'16': [Version('0.0.4')],
'17': [Version('0.3.0'), Version('0.3.1'), Version('0.3.2'), Version('0.3.3'), Version('0.3.4'), Version('0.4.0'), Version('0.4.1')],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we no longer have servers at an older 0.3.* version that we need to upgrade from, you can also move old update scripts to old_updates

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are still servers with 0.3.x versions, will move the update files to old_updates once all servers are upgraded

id
----
1
2
3
(3 rows)

DROP INDEX cos_idx;
CREATE INDEX ham_idx ON small_world_arr USING lantern_hnsw(v) WITH (m=3);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this supposed to be testing hamming distance? it seems the index creation uses the default l2sq operator in this case as well - there is a typo in the old test.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed to create an index with correct opclass

@var77 var77 merged commit 475d861 into main Nov 3, 2024
62 of 66 checks passed
@var77 var77 deleted the varik/pg17-support branch November 3, 2024 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants