Channel: drain/drainUpTo include pending puts

[johnhungerford]

closes #978

[johnhungerford]

@fwbrasil @hearnadam this update produces a potential issue: when streaming from a channel, there is no guarantee that you will consume elements in order. In drainUpTo, between draining from the queue and taking from the pending puts, elements in pending puts can be flushed into the queue. This has resulted in flakiness in some older tests that assert that putting Seq(0 to n) will come out Seq(0 to n).

So I guess the question is do we care that order is not preserved?

[fwbrasil]

Maintaining ordering would be ideal. What if we had a cycle of queue.drainUpTo/flush/queue.drainUpTo/flush/.. until drainUpTo returns empty? It wouldn't handle the case where the channel/queue has capacity 0 but it might be ok. I've been thinking that we should have a separate impl for channels without capacity

[johnhungerford]

Good idea, that works nicely.

[johnhungerford]

If maintaining order is required, our current batched put implementation is also problematic. When a full batch does not fit in the queue, the remainder is sent to the back of puts. Is this something we want to rethink?

The only solution I see to this is to add a stack of Puts that has priority over the puts Queue, so that if an offer to the main queue fails during flush() it can be thrown on the stack and processed next instead of going to the back of the line. I think even this could lead to mis-ordering with concurrent consumers in some cases, however.

[fwbrasil]

More relaxed ordering when the queue becomes full and there are multiple concurrent puts seems more reasonable given that concurrent puts can arrive in any order. To ensure ordering even in this scenario, I think we'd need to have a single atomic operation instead of having to deal with both the main queue and the pending takes/puts queues, which I think wouldn't allow some of the fast paths we currently have in the implementation like put succeeding immediately if there's space in the queue. I'd say it's ok. The flushing logic can also change ordering under high contention (but it's very rare)