Add split with limit operator (to protect against long input without delimiters) #34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add
Transformers.split(maxItemLength, pattern, maxPatternLength)
operator.This is a useful operator in the circumstance that the input is not neccessarily controlled and there exists the possibility of a really long stream of strings arriving without a delimiter. This may cause the normal split operator to throw an OutOfMemoryException as it accumulates a really long string waiting for the delimiter.
Unit test coverage of new operator is complete but for some
isUnsubscribed()
checks. These are painful to write tests for because buffering is occurring in the state-machine operator so I've left them out.See javadoc of new operator for more detail.
In theory there is extra allocation overhead using this operator but benchmarks show no significant difference.