Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid expensive Solr join when Guest users search (affects IP Groups) #10554

Closed
pdurbin opened this issue May 10, 2024 · 0 comments · Fixed by #10555
Closed

Avoid expensive Solr join when Guest users search (affects IP Groups) #10554

pdurbin opened this issue May 10, 2024 · 0 comments · Fixed by #10555
Milestone
6.3

Comments

@pdurbin
Copy link
Member

pdurbin commented May 10, 2024

This is a sub issue of:

We learned yesterday from a Solr expert that we should avoid joins in Solr whenever possible. I identified a place in the code where we should be able to remove a join for Guest users. However, it will make the IP Groups feature much less useful with objects not being searchable.
pdurbin added a commit that referenced this issue May 10, 2024
@pdurbin
avoid expensive Solr join when guest users search (affect IP Groups) #…
5a5c970 
…10554
@pdurbin pdurbin mentioned this issue May 10, 2024
Merged
@pdurbin pdurbin moved this to In Review 🔎 in IQSS Dataverse Project May 10, 2024
pdurbin added a commit that referenced this issue May 17, 2024
@pdurbin
fix copy/past error, target doc for file, not dataset #10554
6baef6a
pdurbin added a commit that referenced this issue May 17, 2024
@pdurbin
Merge branch 'develop' into 10554-avoid-solr-join-guest #10554
070b3c0
landreev added a commit that referenced this issue May 21, 2024
@landreev
Checking a few experimental changes into the branch:
bb33111 
Jim's soft commit fixes from 10547;
A quick experiment, replacing join on public objects with a boolean publicObject_b:true
for logged-in users as well (with a join added for just for their own personal documents;
groups are ignored for now). #10554
landreev added a commit that referenced this issue May 22, 2024
@landreev
Step 3, of the performance improvement effort relying on a boolean "p…
b34e1fe 
…ublicObject" flag for

published documents - now for logged-in users, AND with support for groups.
Group support experimental, but appears to be working. #10554
landreev added a commit that referenced this issue May 22, 2024
@landreev
Modified the implementation for the guest user, to support ip groups. #…
1a097cc 
…10554
landreev added a commit that referenced this issue May 24, 2024
@landreev
Removed the few autocommit-related changes previously borrowed from 1…
1f66420 
…0547, to keep things separate and clear, for testing etc. #10554
landreev added a commit that referenced this issue May 28, 2024
@landreev
Reorganized the optimized code in SearchServiceBean; combined the cod…
dbfa665 
…e block

for the guest and authenticated users. #10554
landreev added a commit that referenced this issue May 28, 2024
@landreev
updated the release note. #10554
0f7a4e5
landreev added a commit that referenced this issue May 28, 2024
@landreev
Removed the warning from the ip groups guide about the effect of the new
74ed174 
search optimization feture that was no longer true. #10554
landreev added a commit that referenced this issue May 28, 2024
@landreev
Updated the section of the guide describing the new Solr optimization
5731427 
feature flags. #10554
landreev added a commit that referenced this issue May 28, 2024
@landreev
Updated the performance section of the guide. #10554
aaec213
landreev added a commit that referenced this issue May 28, 2024
@landreev
Modified IndexServiceBean to use the new feature flag, that has been …
9b8c7d9 
…separated from the flag that

enables the search-side optimization;
Fixed the groups sub-query for the guest user. #10554
landreev added a commit that referenced this issue May 28, 2024
@landreev
cosmetic #10554
38b7d30
pdurbin added a commit that referenced this issue May 30, 2024
@pdurbin
doc tweaks #10554
15ace69
pdurbin added a commit that referenced this issue May 30, 2024
@pdurbin
no-op code cleanup, correct case of publicObject_b #10554
ae6cf4e
stevenwinship pushed a commit that referenced this issue Jun 4, 2024
@pdurbin @landreev
avoid expensive Solr join for public dvObjects in search (experimenta…
3c55c3f 
…l) (#10555)

* avoid expensive Solr join when guest users search (affect IP Groups) #10554

* fix copy/past error, target doc for file, not dataset #10554

* Checking a few experimental changes into the branch:
Jim's soft commit fixes from 10547;
A quick experiment, replacing join on public objects with a boolean publicObject_b:true
for logged-in users as well (with a join added for just for their own personal documents;
groups are ignored for now). #10554

* Step 3, of the performance improvement effort relying on a boolean "publicObject" flag for
published documents - now for logged-in users, AND with support for groups.
Group support experimental, but appears to be working. #10554

* Modified the implementation for the guest user, to support ip groups. #10554

* Removed the few autocommit-related changes previously borrowed from 10547, to keep things separate and clear, for testing etc. #10554

* Reorganized the optimized code in SearchServiceBean; combined the code block
for the guest and authenticated users. #10554

* updated the release note. #10554

* Removed the warning from the ip groups guide about the effect of the new
search optimization feture that was no longer true. #10554

* Updated the section of the guide describing the new Solr optimization
feature flags. #10554

* Updated the performance section of the guide. #10554

* Modified IndexServiceBean to use the new feature flag, that has been separated from the flag that
enables the search-side optimization;
Fixed the groups sub-query for the guest user. #10554

* cosmetic #10554

* doc tweaks #10554

* no-op code cleanup, correct case of publicObject_b #10554

---------

Co-authored-by: Leonid Andreev <leonid@hmdc.harvard.edu>
@pdurbin pdurbin added this to the 6.3 milestone Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
6.3
Development

Successfully merging a pull request may close this issue.

avoid expensive Solr join for public dvObjects in search (experimental) IQSS/dataverse
1 participant
@pdurbin