Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dynamic filters to text analyzer builder. #2110

Merged
merged 8 commits into from
Jul 3, 2023

Conversation

fulmicoton
Copy link
Collaborator

@fulmicoton fulmicoton commented Jun 30, 2023

Tokenizer bench on this branch

default-tokenize-alice  time:   [926.67 µs 941.91 µs 967.19 µs]
                        change: [-5.6580% -3.4840% -0.6031%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 200 measurements (2.00%)
  1 (0.50%) high mild
  3 (1.50%) high severe

dynamic-tokenize-alice  time:   [1.2427 ms 1.2498 ms 1.2576 ms]
                        change: [-6.5215% -4.5710% -3.0067%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 200 measurements (5.00%)
  5 (2.50%) high mild
  5 (2.50%) high severe

On the main branch

cargo bench --bench analyzer
    Finished bench [optimized + debuginfo] target(s) in 0.32s
warning: the following packages contain code that will be rejected by a future version of Rust: quick-xml v0.22.0
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
     Running benches/analyzer.rs (target/release/deps/analyzer-0abd8235407e8a29)
Gnuplot not found, using plotters backend
default-tokenize-alice  time:   [929.75 µs 955.32 µs 997.68 µs]
                        change: [-7.4124% -5.6996% -3.3711%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

@codecov-commenter
Copy link

codecov-commenter commented Jun 30, 2023

Codecov Report

Merging #2110 (b91d3f6) into main (910b0b0) will increase coverage by 0.02%.
The diff coverage is 98.59%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main    #2110      +/-   ##
==========================================
+ Coverage   94.36%   94.39%   +0.02%     
==========================================
  Files         321      321              
  Lines       60675    60735      +60     
==========================================
+ Hits        57258    57329      +71     
+ Misses       3417     3406      -11     
Impacted Files Coverage Δ
src/query/more_like_this/more_like_this.rs 65.91% <0.00%> (+0.24%) ⬆️
src/indexer/segment_writer.rs 97.87% <100.00%> (ø)
src/tokenizer/tokenizer.rs 96.77% <100.00%> (+4.09%) ⬆️
tokenizer-api/src/lib.rs 100.00% <100.00%> (+18.75%) ⬆️

... and 2 files with indirect coverage changes

@fmassot fmassot force-pushed the fulmicoton/dynamic-follow-up branch from d997027 to 61a3568 Compare June 30, 2023 13:26
@fmassot fmassot marked this pull request as ready for review June 30, 2023 13:28
@fmassot fmassot force-pushed the fulmicoton/dynamic-follow-up branch 2 times, most recently from 27e7b65 to bc5b412 Compare June 30, 2023 13:45
@fmassot fmassot changed the title Fulmicoton/dynamic follow up Add text analyzer builder with dynamic filters Jun 30, 2023
@fmassot fmassot changed the title Add text analyzer builder with dynamic filters Add dynamic filters to text analyzer builder. Jun 30, 2023
@fulmicoton fulmicoton force-pushed the fulmicoton/dynamic-follow-up branch from 99c6ee3 to a8e7651 Compare July 3, 2023 13:06
@fmassot fmassot merged commit 3c30066 into main Jul 3, 2023
@fmassot fmassot deleted the fulmicoton/dynamic-follow-up branch July 3, 2023 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants