Skip to content

Commit

Permalink
Improve iobuf -> numpy conversion latency
Browse files Browse the repository at this point in the history
Summary: roticv provided this change to bypass coalescing iobuf with `b"".join(iobuf)` and providing the iobuf itself as the buffer to numpy array

Reviewed By: roticv

Differential Revision: D69883091

fbshipit-source-id: 64f6268237f2e8de2b221ba3e1d0bf068c8e5f28
  • Loading branch information
kevinngo0407 authored and facebook-github-bot committed Feb 22, 2025
1 parent 0968586 commit 8dd8df7
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 0 deletions.
1 change: 1 addition & 0 deletions folly/python/iobuf.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ cdef extern from "folly/io/IOBuf.h" namespace "folly":
size_t countChainElements()
uint64_t computeChainDataLength()
unique_ptr[cIOBuf] clone()
unique_ptr[cIOBuf] cloneCoalesced()
cIOBuf* prev()
cIOBuf* next()
void insertAfterThisOne(unique_ptr[cIOBuf]&& ciobuf)
Expand Down
1 change: 1 addition & 0 deletions folly/python/iobuf.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ class IOBuf(Hashable):
def __init__(self, buffer: Union[IOBuf, bytes, bytearray, memoryview]) -> None: ...
def writable(self) -> bool: ...
def clone(self) -> IOBuf: ...
def cloneCoalesced(self) -> IOBuf: ...
@property
def next(self) -> Optional[IOBuf]: ...
@property
Expand Down
4 changes: 4 additions & 0 deletions folly/python/iobuf.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,10 @@ cdef class IOBuf:
""" Clone the iobuf chain """
return from_unique_ptr(self._this.clone())

def cloneCoalesced(self):
""" Clone the iobuf chain """
return from_unique_ptr(self._this.cloneCoalesced())

@property
def next(self):
_next = self._this.next()
Expand Down

0 comments on commit 8dd8df7

Please sign in to comment.