Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Stream Tracking #369

Closed
rzvoncek opened this issue Mar 19, 2018 · 1 comment
Closed

Add Stream Tracking #369

rzvoncek opened this issue Mar 19, 2018 · 1 comment
Assignees

Comments

@rzvoncek
Copy link
Contributor

rzvoncek commented Mar 19, 2018

This issue tracks the work related to adding the capability of tracking Cassandra streams to Reaper.

A draft of this functionality has been proposed in #392. Based on the review of @adejanovski and @michaelsembwever, here are the needed improvements:

  • The newly added POJOs must truly be POJOs:
    • FileProgress should be make private.
    • Any method that is not a getter/setter should go to a factory class:
      • Most notably Stream::newStream and Stream::streamId.
      • But also <add|update|finalize>Stream() in StreamSession.
      • Fields and classes should be made final wherever possible.
      • Guava precondition checks should be placed where appropriate, notably for StreamSession fields.
  • Streams from all C* versions should be tracked.
    • In the MVP, streams are tracked only in 2.2.9 and later.
  • Persistent stream notification listeners.
    • Subscription to stream messages should got to JmxFactory. The current placement in StreamManager doesn't re-establish the listener when reaper is restarted.
  • Make stream tracking configurable.
    • It should be possible to configure things like number of threads handling stream notifications and cache retention via Reaper's YAML.
  • StreamBuilder should be package protected.
  • The @Path in cluster resource may not need the /cluster/ part.
  • Use @Alpha for anything that's not tested
  • Handle lost notifications.
    • In addition to listening to notifications, Reaper should sometimes also pull stream info to ensure it has correct stream information.

Once the above is done, this should happen too:

  • The stream info should be stored in a distributed storage so that a multi-instance Reaper deployments offer consistent streaming info.
@rzvoncek rzvoncek self-assigned this Mar 19, 2018
rzvoncek pushed a commit that referenced this issue Mar 27, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
rzvoncek pushed a commit that referenced this issue Apr 3, 2018
@rzvoncek rzvoncek changed the title Add Stream Plan Visualisation Add Stream Tracking Apr 4, 2018
rzvoncek pushed a commit that referenced this issue Jul 5, 2018
rzvoncek pushed a commit that referenced this issue Jul 5, 2018
rzvoncek pushed a commit that referenced this issue Aug 3, 2018
rzvoncek pushed a commit that referenced this issue Aug 3, 2018
@rzvoncek
Copy link
Contributor Author

Done in #465.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant