Add Binance backend #170
Add Binance backend #170
Conversation
| } | ||
|
|
||
|
|
||
| class AutoReconWs: |
There was a problem hiding this comment.
Maybe put this in some sort of utils module or something like that, as it is shared between kraken & binance backends.
There was a problem hiding this comment.
Yah 💯.
We need to fix up the kraken ws respawning anyway so we should probably stick that generic subs mgmt code somewhere as well.
This gets the binance provider meeting the data feed schema requirements of both the OHLC sampling/charting machinery as well as proper formatting of historical OHLC history. Notably, - spec a minimal ohlc dtype based on the kline endpoint - use a dataclass to parse out OHLC bar datums and pack into np.ndarray/shm - add the ``aggTrade`` endpoint to get last clearing (traded) prices, validate with ``pydantic`` and then normalize these into our tick-quote format for delivery over the feed stream api. - a notable requirement is that the "first" quote from the feed must contain a 'last` field so the clearing system can start up correctly.
|
@guilledk lol so as mentioned in chat, the quote rates coming from binance easily fall in the "higher" frequency realm 😂 In which case we should be throttling graphics consumers to their local display / refresh rate to avoid unnecessary render cycles. Pretty happy with the look of things so far 🏄🏼 |
Get binance OHLC history and quote format correct
| row.append(bar.time / 1000.0) | ||
|
|
||
| else: | ||
| row.append(getattr(bar, name)) |
There was a problem hiding this comment.
This felt really really clunky.
What's a better way to one shot validate these?
I don't know if you can unpack *bar in a pydantic model..
There was a problem hiding this comment.
Yeah I mucked with this for a while and got nowhere too fancy 😂
Since our dtype is technically matching the dataclass we could actually probably just use asdict if we wanted to do it this way:
[nav] In [116]: @dataclass
...: class C:
...: doggy: int
...: num: int
[nav] In [117]: l = [C(doggy=9, num=10), C(doggy=11, num=12)]
[nav] In [118]: np.array(list(list(asdict(c).values()) for c in l), dtype={'names': ('doggy', 'num',), 'formats': (int, int)})
Out[118]:
array([[( 9, 9), (10, 10)],
[(11, 11), (12, 12)]], dtype=[('doggy', '<i8'), ('num', '<i8')])There was a problem hiding this comment.
Using a pydantic.BaseModel.dict() would require array unpacking each bar into dict form since I still don't think pydantic.BaseModel supports the OHLC(*bar) style creation.
Not sure it'd be worth it anyway for readability.
| # validate | ||
| msg = AggTrade(**msg) | ||
|
|
||
| # TODO: type out and require this quote format |
There was a problem hiding this comment.
This is definitely part of writing up the backend spec.
|
|
||
| # UI components allow this to be declared such that additional | ||
| # (historical) fields can be exposed. | ||
| ohlc_dtype = np.dtype(_ohlc_dtype) |
There was a problem hiding this comment.
Hmm it's interesting i think in this case we don't actually need to define this since it matches the default but, i guess for now documenting that we could have extras is handy?
| class Client: | ||
|
|
||
| def __init__(self) -> None: | ||
| self._sesh = asks.Session(connections=4) |
| # TODO: maybe we should go nanoseconds on all | ||
| # history time stamps? | ||
| if name == 'time': | ||
| # convert to epoch seconds: float |
There was a problem hiding this comment.
We should make an issue about possibly moving to nanoseconds as int for all time stamps.
I see no reason to not be thinking about UHFT down the road.
| # identify those messages by matching keys | ||
| # https://binance-docs.github.io/apidocs/spot/en/#individual-symbol-book-ticker-streams | ||
|
|
||
| if msg.get('u'): |
There was a problem hiding this comment.
Oh that was my other question: can we do similar BaseModel unpacking/validation here for l1 messages?
|
@guilledk oh one more thing would be to find out if binance supports historical tick data like kraken such that we can get long(er) term views of high frequency sampled data (1s OHLC) going. If not we may have to look to some other historical resources. Making at least issue for this task would be good. |
Binance seems to have a pretty acceptable API.
General Info
REST OHLC for historical backfilling
Websocket L1 stream
Realtime trade feed