Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can IndexSearcher enable search concurrency by default? [LUCENE-10487] #11523

Open
asfimport opened this issue Mar 28, 2022 · 2 comments
Open

Comments

@asfimport
Copy link

IndexSearcher has the ability to search an index using multiple threads, but this behavior is an opt-in. Could we enable it by default so that:
 - Users experience better latency out-of-the-box.
 - Lucene performs better in comparative benchmarks against other engines that leverage multiple threads by default.

Note that this isn't totally straightforward to implement, as we'd need to share the threadpool across all index searchers and think of when to shut down this threadpool?


Migrated from LUCENE-10487 by Adrien Grand (@jpountz)

@jpountz
Copy link
Contributor

jpountz commented Apr 28, 2024

Optimistically adding this issue to the 10.0 milestone.

@javanna
Copy link
Contributor

javanna commented Jul 26, 2024

This should probably be taken into consideration again. With #13472, we can now use the same executor provided to the IndexSearcher constructor, to call IndexSearcher#search. That simplifies the requirements for concurrent searches quite a bit, in that you no longer as a user need to think about sizing and providing a separate workers thread pool. Can we leverage that to enable concurrency by default?

@javanna javanna modified the milestones: 10.0.0, 11.0.0 Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants