Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add split with limit operator (to protect against long input without delimiters) #34

Merged
merged 3 commits into from
Apr 7, 2020

Conversation

davidmoten
Copy link
Owner

@davidmoten davidmoten commented Apr 6, 2020

Add Transformers.split(maxItemLength, pattern, maxPatternLength) operator.

This is a useful operator in the circumstance that the input is not neccessarily controlled and there exists the possibility of a really long stream of strings arriving without a delimiter. This may cause the normal split operator to throw an OutOfMemoryException as it accumulates a really long string waiting for the delimiter.

Unit test coverage of new operator is complete but for some isUnsubscribed() checks. These are painful to write tests for because buffering is occurring in the state-machine operator so I've left them out.

See javadoc of new operator for more detail.

In theory there is extra allocation overhead using this operator but benchmarks show no significant difference.

…h to avoid OutOfMemoryException when items very long
@codecov-io
Copy link

codecov-io commented Apr 7, 2020

Codecov Report

Merging #34 into master will decrease coverage by 0.06%.
The diff coverage is 81.96%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master      #34      +/-   ##
============================================
- Coverage     68.74%   68.68%   -0.07%     
- Complexity      417      427      +10     
============================================
  Files            69       69              
  Lines          3558     3602      +44     
  Branches        390      402      +12     
============================================
+ Hits           2446     2474      +28     
- Misses          988      996       +8     
- Partials        124      132       +8     
Impacted Files Coverage Δ Complexity Δ
.../rx/internal/operators/TransformerStringSplit.java 80.95% <81.35%> (+5.95%) 10.00 <10.00> (+9.00)
...in/java/com/github/davidmoten/rx/Transformers.java 75.72% <100.00%> (+0.28%) 45.00 <2.00> (+2.00)
...tors/QueueWithResourcesNonBlockingUnsubscribe.java 66.66% <0.00%> (-12.13%) 11.00% <0.00%> (ø%)
...va/com/github/davidmoten/rx/util/MapWithIndex.java 81.81% <0.00%> (-4.55%) 3.00% <0.00%> (ø%)
...en/rx/internal/operators/OperatorBufferToFile.java 82.17% <0.00%> (-2.33%) 4.00% <0.00%> (ø%)
...oten/rx/internal/operators/FileBasedSPSCQueue.java 72.99% <0.00%> (-2.19%) 12.00% <0.00%> (ø%)
...dmoten/rx/internal/operators/RollingSPSCQueue.java 50.00% <0.00%> (-1.82%) 18.00% <0.00%> (-1.00%)
...ub/davidmoten/rx/buffertofile/DataSerializers.java 53.84% <0.00%> (-1.06%) 3.00% <0.00%> (ø%)
...om/github/davidmoten/rx/testing/TestingHelper.java 77.77% <0.00%> (-0.08%) 27.00% <0.00%> (ø%)
src/main/java/com/github/davidmoten/rx/Jaxws.java 0.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a91d2ba...9912682. Read the comment docs.

@davidmoten davidmoten merged commit 5134628 into master Apr 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants