Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial transactional producer work #41

Open
avelanarius opened this issue Dec 13, 2019 · 0 comments
Open

Initial transactional producer work #41

avelanarius opened this issue Dec 13, 2019 · 0 comments
Labels
kafka zpp ZPP: student project

Comments

@avelanarius
Copy link

avelanarius commented Dec 13, 2019

Some of my thoughts/observations (maybe useful?):

Looking at packet captures, it seems that transactional producer initializes itself in such fashion:

  • FindCoordinator packets: It gets the coordinator IP & port (and connects to it?)
  • InitProducerId packet: It sends producer_id (only to coordinator?)

Afterwards, when sending messages using Produce request:

  • AddPartitionsToTxn requests
  • Produce request - sets appropriate fields (transactional_id field) and inside records there is producer_epoch (is this also set?)

Commit or abort: EndTxn request

BTW: Best source for packets specification is: https://github.com/apache/kafka/tree/trunk/clients/src/main/resources/common/message (official Kafka protocol website can be outdated!)

Another area which seems important is "producer epoch", which I think is incremented after each EndTxn or InitProducerId. There is probably some logic to handle this epoch number.

Maybe an intial PR could only implement FindCoordinator + InitProducerId phase?

@avelanarius avelanarius added zpp ZPP: student project kafka and removed zpp ZPP: student project labels Dec 13, 2019
@StarostaGit StarostaGit removed their assignment Dec 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kafka zpp ZPP: student project
Projects
None yet
Development

No branches or pull requests

2 participants