Some fixes (#11)

* v0.0.1 of DepthaiViewer, WIP (#1)

* node graph scaffolding and a bit of depthai integration

* remove import

* left, right  + depth

* fix fps sliders for mono cams

* partial refactor, partially added support for device selection.

* pc support + currently selected device fixes

* reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend

* Partially migrated to websockets for the api

* finish moving to websockets for the config api

* fix issues after merge, retry websocket connections

* fix ws thread not exiting

* sleep for 1 sec before trying to reconnect

* Ai support + imu support + rotate camera + bug fixes (#9)

* ai support + update pipeline on device select

* quick fix depth

* set subs when selecting device

* rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected

* added age gender detection

* added mobilenet support

* pointcloud support

* Toggle subs from visible (#4)

* very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff

* better implementation and actually seems to work

* fix windows not resizing properly when size gets smaller

* fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox

* Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho

* Toggle subs from visible (#5)

* very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff

* better implementation and actually seems to work

* fix windows not resizing properly when size gets smaller

* fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox

* Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho

* fix incorrect despawn of entity paths

* remove todo comment

* Fix ws not sending messages, fix stutter every 2s (#6)

* very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff

* better implementation and actually seems to work

* fix windows not resizing properly when size gets smaller

* fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox

* Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho

* fix incorrect despawn of entity paths

* remove todo comment

* error handling

* fix websocket not sending pipeline, hotfix more or less might keep

* change back to unbounded

* Imu support (#8)

* add imu logging, exclude it from regular space views

* fix imu logging

* POC imu plotting

* Imu accelerometer and gyroscope plotting

* add orientation to imu log, make device id a String

* fix imu charts layout

* imu log add magnetometer

* fix right panel ui as much as it makes sense to fix rn

* sadly idk how to create Option::None in pyarrow for the magnetometer field

* imu logging fixed, magnetometer value is now None when not logged

* fix plot scrolling

* Merge upstream (#10)

* `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742)

* arrow2_convert primitive benchmarks

* addressing PR comments

* Fix logged obb being displayed with half of the requested size (rerun-io#1749)

* benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747)

* benchmarks for common vector ops

* handle N=1

* Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751)

In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits".

* Improve dealing with raw buffers for texture read/write (rerun-io#1744)

* Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo

* comment & naming fixes

* `arrow2` erased refcounted clones benchmarks (rerun-io#1745)

* arrow2 erased refcounted clone benchmarks

* lint

* addressing PR comments

* dude

* `arrow2` estimated_bytes_size benchmarks (rerun-io#1743)

* arrow2 estimated_bytes_size benchmarks

* cleanup

* Fix crash when trying to do picking on depth clouds

* Readback depth from GPU picking (rerun-io#1752)

* gpu picking in the viewer picks up depth now
* WebGL workarounds

* Add new ARKitScenes example (rerun-io#1538)

Co-authored-by: Nikolaus West <nikolaus.west@me.com>
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Fix log_obb usage (rerun-io#1761)

* Make sure all log_obb uses uses half_size correctly

* Remove outdated link from README

* Fix docstring of save

* Force named arguments of log_scalar

* Add link to --memory-limit docs

* update ros example

* Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740)

* Python SDK: document that we also accept colors in 0-1 floats

* Assume float colors to be in gamma-space, and document that

* Update arkitscenes example

* Fix bug

* typo

* py-format

* Collapse space-view by default if there is only one child (rerun-io#1762)

* Always create the log_time timeline (rerun-io#1763)

* Columnar timepoints in data tables and during transport (rerun-io#1767)

* columnar timepoints

* self review

* Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765)

* Fix undo/redo selection shortcut/action changing selection history without changing selection
Fixes rerun-io#1172

* typo fix

* Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768)

* Allow torch tensors for log_rigid3 (rerun-io#1769)

* Option to show scene bounding box (rerun-io#1770)

* Include depth clouds in bounding box calculation

* Don't wrap text when showing bbox in ui

* Handle projective transforms

* Nicer selection view: don't wrap second column too early

* Add checkbox to show the scene bounding box

* Fix a whole lot of crashes, all at once (rerun-io#1780)

* Add typing_extensions to requirements-doc.txt (rerun-io#1786)

* auto_color class-ids if they are present (rerun-io#1783)

* auto_color class-ids if they are present

* Update log line in segmentation demo

* Avoid tuple structs

* Don't run 3rd party bench suites on CI (rerun-io#1787)

* dont run 3rd party bench suites on CI

* typo

* and other annoyances

* Use copilot markers in PR template (rerun-io#1784)

* Use copilot markers in PR template

* remove poem

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>

---------

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>

* re_format: barebone support for custom formatting (rerun-io#1776)

* implement barebone support for custom formatting and apply to Tuid

* unwrap

* <ERR> rather than []

* use re_tuid

* Always send recording_id as part of LogMsg (rerun-io#1778)

* Always send recording_id as part of LogMsg

* Rename build_chunk_from_components -> build_data_table_from_components

* Don't make RecordingInfo optional

* Always default the recording id

* Log an error if we hit the initialization issue

* Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772)

* CI: Check `rerun` with --no-default features and/or with --features sdk

* Create a new helper crate re_transport containing stream_rrd_from_http

* Fix warnings

* Move file sink to re_transport

* wasm compilation fix

* Move LogMsg encoding/decoding into re_transport

* Fix typo

* Fix web build

* Fix tests

* Remove a lot of unused dependencies with `cargo machete`

* Build fix

* Clarify

* Rename the crate to re_log_encoding

* better docstring

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

* better readme

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

---------

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>

* New example code for facebook research segment anything (rerun-io#1788)

* New example code for facebook research segment anything

* Add segmentation workaround for users still on 0.4.0

* Images should use class-id as label

* Add an alternative tensor-based view

* Implement `re_tuid::Tuid::random()` on web (rerun-io#1796)

* Implement `re_tuid::Tuid::random()` on web

* Fix bad error message

* ci: fix benchmarks (rerun-io#1799)

* workflow: just run --all

* datastore: skip bucket permutations etc on CI

* i give up, just replace re_log_types by re_log_encoding

* Add `minimal_options` example (`RerunArgs`) (rerun-io#1773)

* Allows connecting to remote server through rerun's RerunArgs.

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>

* Add `pacman` support to `setup_web.sh` (rerun-io#1797)

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>

* fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806)

* Compile with `panic = "abort"` (rerun-io#1813)

* Compile with `panic = "abort"`

This PR sets `panic = "abort"` for both debug and release builds.

This cuts down the `rerun` binary size in release builds from
29.9 MB to 22.7 MB - a 25% reduction!

## Details
The default panic behavior in Rust is to unwind the stack.
This leads to a lot of extra code bloat, and some missed
opportunities for optimization.

The benefit is that one can let a thread die without crashing the whole
application, and one can use `std::panic::catch_unwind` as a kind of
try-catch block.

We don't make use of these features at all (at least not intentionally),
and so are paying a cost for something we don't need.

I would also argue that a panic SHOULD lead to a hard crash unless
you are building an Erlang-like robust actor system where you use
defensive programming to protect against programmer errors
(all panics are programmer errors - user errors should use `Result`).

* Quiet clippy

* Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812)

* Add `rerun --strict`: crash if any warning or error is logged

Part of rerun-io#1483

* Can't doc-test private functions

* Refactor: Remove `TensorTrait` (rerun-io#1819)

* Refactor: Remove `TensorTrait`

We don't need it anymore

* End-to-end testing of python logging -> store ingestion (rerun-io#1817)

* Sort the arguments to `rerun`

* Pass on `LogMsg::Goodbye` just like any other message

* Add `rerun --test-receive`

* `just py-build --quiet` is now possible

* Add scripts/run_python_e2e_test.py

* replace `cargo r -p rerun` with `python3 -m rerun`

* lint and explain choice of examples

* Add to CI

* check returncode

* Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821)

* Use gpu picking for points, streamline/share picking code some more (rerun-io#1814)

* use gpu picking for picking points
* gpu based picking no longer works like a fallback but integrates with other picking sources
* fix incorrect cursor rounding for picking
* refactor picking context to be a pub struct with exposed state
* unify ui picking method for 2d & 3d space views
* less indentation for picking method
* picking rect size is dynamically chosen
* fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read

* CI: install pip requirements for Python e2e test

* Process 2d points always in batches (rerun-io#1820)

* Fix CI syntax error

* New option to disable persistent storage (rerun-io#1825)

* New option to disable persistent storage

* New API to reset_time (rerun-io#1826)

* Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727)

* Datastore revamp 2: serialization & formatting (rerun-io#1735)

* Datastore revamp 3: efficient incremental stats (rerun-io#1739)

* Datastore revamp 4: sunset `MsgId` (rerun-io#1785)

* Datastore revamp 5: (rerun-io#1791)

* Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795)

* Datastore revamp 7: garbage collection (rerun-io#1801)

* Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832)

* CI: Try installing the correct wheel on CI

* CI: try again with the CI

* re_query: up to date with latest data types and structures (rerun-io#1828)

* No more raw arrays for primary components

* Don't need to carry around component names no more

* Cluster keys are now raw-array-less and _not_ optional anymore

* that is done indeed

* helpers

* datastore: incremental metadata registry stats (rerun-io#1833)

* add profile scopes for stats

* implement incremental metadata registry stats

* lint

* future proofing comment

* RFC: datastore state of the union & end-to-end batching  (rerun-io#1610)

* add batching rfc

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* qa component instances vs. instance keys

* no sticky

* qa are there any special components & non-integer instance keys

* give some clue about cell incompatibility

* temporary constructs

* zstd is already setup

* more planning

* date and links

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Python CI: use bash as shell

* Fix too many points crash (rerun-io#1822)

* Simplify point cloud builder and fix crash on too many points
Fixes rerun-io#1779
* faster point cloud population by not chaining iterators with default values

* Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829)

* line strip builder no longer has user data, exposes picking id instead (not implemented yet)
* handle interactive object property when evaluating picking code
* take line strip builder directly when building up line draw data
* finish implementing picking for lines
* remove unused iter_strips_with_vertices
* Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021

* CI: try to fix mac wheel build

* Reduce memory used by staging belts on Web (rerun-io#1836)

In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version.

I took the opportunity to sharpens the definition of `HardwareTier` a bit.

* CI: only test the x86_64 wheel on macos

* Always flush when we remove a sink (rerun-io#1830)

Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously).

* GPU colormapping, first step (rerun-io#1835)

* Add TextureManager2D::get_or_create_with

* Small code cleanup

* Add code to upload a Tensor to a GPU texture

* Add helper method Tensor::image_height_width_depth

* Minor code cleanup (multiplicative_tint)

* Hook up color textures via the new path

* Refactor: introduce ColormappedTexture

* Start working on an uint sampler

* merge fix

* Dumb colormapping of depth textures!

* Use turbo for depth maps (and single-channel images :grimace:)

* Use grayscale for luminance

* ColorMap -> Colormap

* Apply annotation context colormaps

* Support sint textures too

* cleanup

* merge fix

* Fix RGB images

* More cleanup

* Better error-handlign and nicer error message

* Clean up the SAMPLE_TYPE with constants

* Nicer shader interface

* Better error handling

* Remove dead code

* Self-review cleanup

* Fix bug in shader when sampling sint textures

* Use textureSampleLevel

* Apply a gamma to the image (unused as of now)

* image_height_width_channels

* fix various review comments

* Optimize narrow_f64_to_f32s: avoid one allocation

* Test and handle all tensor dtypes as images (rerun-io#1840)

* Support i64 and u64 tensors

* api_demo: log all image types

* Don't even print out the contents of a tensor

* Handle unfilterable float textures

* fix typo

* py-format

* Simplify is_float_filterable

* Add a helper function pad_and_narrow_and_cast

* Reuse existing image

* Exclude image_tensors demo from default api_demo

* Still run all api demos in e2e test

* pyformat

* Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838)

* Install the rerun-sdk by the expected version

* Fix comment

* typo

* Use --no-index when installing the rerun wheel

* Use the cargo_version, not the new_version

* Split dependency install into its own step

* Don't use force-reinstall

* Refactor setting of expected_version variable.

* Use bash when setting env

* Always run the linux job first and use its rrds for the other wheels

* GPU tensor colormapping (rerun-io#1841)

* Refactor: introduce struct SliceSelection

* Refactor: use SliceSelection inside of ViewTensorState

* MVP of tensor colormapping on GPU

* Remove old ui code

* Support 64-bit tensors by narrowing to f32

* Allow more colormap options

* Clippy

* Report range errors instead of ignoring them

* Sort colormaps

* Shorten function name

* Create module gpu_bridge

* Move some code around

* Simnplify API

* Create ViewBuilder::new

* Fix missing colon in lint.py

* fix typos and formatting

* Disable texture filtering options for tensors for now

* Update docstrings

* Add profile scopes

* ViewBuilder cleanup

* Make ViewBuilder::setup non-Option

* Remove Result from thing that cannot fail

* Fix colormap numbering

* review cleanup

* pass in debug_name

* Unify the `range` function

* typo

* Show previews of colormaps when selecting them (rerun-io#1846)

* Make infallible version of get_or_create_texture

* Show colormap previews in UI

* Spelling

* Implement billinear filtering of textures (rerun-io#1850)

* Implement opt-in billinear filtering of textures

* bilinear

* MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844)

* Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage
* New API to output the current buffered messages as a cell in jupyter
* Example notebook with the cube demo
* Track that we need to send another recording msg after draining the backlog
* Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets
* Add some crude timeout logic in case the iframe fails to load
* Don't persist app state in notebooks
* Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834)
* Refactor the relationship between the assorted web / websocket servers (rerun-io#1844)
* Rename RemoteViewerServer to WebViewerSink
* CLI arguments for specifying ports
* Proper typing for the ports

* Disable wheel tests for x86_64-apple-darwin (rerun-io#1853)

* Fix typos in notebook readme (rerun-io#1852)

* Fix the python build when running without web_viewer enabled (rerun-io#1856)

* Error instead of expect inside msg_encode. (rerun-io#1857)

* Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854)

* Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)

This reverts commit d33dab6.

* Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802)

* Add Restart button to timeline UI. This sets the timeline back to zero.

* Adds Restart Command & Timeline Command

* Adds Ctrl-Shift-Space keyboard modifier

* Remove restart button from timeline UI.

* Use cmd(Key::LeftArrow) as key combo for restart timeline.

* Add TimeControl::restart to restart the current timeline.

* Use this from the kb_shortcut function

* fix some code nits

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861)

* Wait for encoder to shut down before shutting down the other threads

* Remove unused dependencies (rerun-io#1863)

* Gpu picking for depth clouds (rerun-io#1849)

* wip

* allow for hovering depth clouds via gpu picking

* Use `[x, y]: [u32; 2]` as argument

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* Remove manual depth projection from car and nyud examples (rerun-io#1869)

* Remove manual depth projection from car and nyud examples

* revert change to cube.ipynb

* revert changes to cube.ipynb

* third times a charm for cube.ipynb

* Improve end-to-end testing slightly (rerun-io#1862)

* CI: Run e2e tests with RUST_LOG=debug

* Move installing of pip packaged from CI to e2e script

* Re-enable bilinear interpolation again (rerun-io#1860)

* Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)"

This reverts commit 625d2bd.

* Split rectangle.wgsl into fragme/vertex parts to work around naga bug

* Use GPU colormapping when showing images in the GUI (rerun-io#1865)

* Cleanup: move Default close to the struct definition

* Simplify code: use if-let-else-return

* Simplify code: no need for Arc

* Add EntityDataUi so that the Tensor ui function knows entity path

* Better naming: selection -> item

* Simplify code: no optional tensor stats

* Less use of anyhow

* Use GPU colormapping when showing tensors in GUI

* Link to issue

* Optimize pad_to_four_elements for debug builds

* Refactor: simpler arguments to show_zoomed_image_region_area_outline

* Fix missing meter argument

* Refactor: break up long function

* Less use of Arc

* Pipe annotation context to the hover preview

* Simplify `AnnotationMap::find`

* Use new GPU colormapper for the hover-zoom-in tooltip

* Refactor

* Add helper function for turning a Tensor into an image::DynamicImage

* Fix warning on web builds

* Add helper function `Tensor::could_be_dynamic_image`

* Implement click-to-copy and click-to-save for tensors without egui

* Convert histogram to the new system

* Remove the TensorImageCache

* Fix TODO formatting

* bug fixes and cleanups

* Rename some stuff

* Build-fix

* Simplify some code

* Turn off benchmakrs comment on each PR (rerun-io#1872)

* Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866)

* Refactor TexturedRect

* Remove GpuTexture2DHandle::invalid

* `GpuTexture2DHandle` is always valid

* spacing

* Update enumflags2 to non-yanked version (rerun-io#1874)

* Update enumflags2 to non-yanked version
```
❯ cargo update -p enumflags2
    Updating crates.io index
    Updating enumflags2 v0.7.5 -> v0.7.7
    Updating enumflags2_derive v0.7.4 -> v0.7.7
    Updating proc-macro2 v1.0.47 -> v1.0.56
    Updating quote v1.0.21 -> v1.0.26
      Adding syn v2.0.15
```

Unfortunately this adds the syn v2 dependency for some platforms

* Updating dependencies is a valid label
* cargo deny: check more platforms

* fix stuff broken from merging upstream

---------

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
Co-authored-by: benjamin de charmoy <BenjaminDev@users.noreply.github.com>
Co-authored-by: Andreas Reich <andreas@rerun.io>
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
Co-authored-by: Pablo Vela <pablovela5620@gmail.com>
Co-authored-by: Nikolaus West <nikolaus.west@me.com>
Co-authored-by: Jeremy Leibs <jeremy@rerun.io>
Co-authored-by: h3mosphere <129932586+h3mosphere@users.noreply.github.com>
Co-authored-by: Urho Laukkarinen <urho.laukkarinen@gmail.com>

* expose depth config, get available sensor resolutions for the selected device

* move removing entities to a place where the removal will always be tried, not just when expanding space view header

* added depth alignment

* added stream enabled buttons

* Depth cloud textures are now cached frame-to-frame (rerun-io#1913)

* Depth cloud textures are now cached frame-to-frame
Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only)

* doc fix

* naming consistency, format check, remove unnecessary scaling

* improve depth cloud texture check

* fixes after merging

* Smooth out scroll wheel input for camera zooming (rerun-io#1920)

* Always spawn instead of fork in multiprocessing example (rerun-io#1922)

* Add `--num-frames` arg to canny (webcam) example (rerun-io#1923)

* fix formatting issues

* rerun format fix

* fix spelling

* lint check fixes

* mypy

* some more lint fixes

* some more fixes for python lint checks

* Collect extra egui features into the main Cargo.toml (rerun-io#1926)

* just rs-run-all

* `just py-run-all-{native|web|rrd}` (rerun-io#1927)

* make all python examples handle unknown arguments gracefully

* just py-run-all-{native|web|rrd}

* bump version

* comment out clang

* Join threads at end of multi-threading example (rerun-io#1934)

* Add argument parsing to the rerun_demo (rerun-io#1925)

* More robust wait for exit condition during .serve() (rerun-io#1939)

* More robust wait for exit condition during .serve()

* lint

* Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936)

* Use zipfile python library instead of `unzip` command in arkitscene
Windows doesn't have unzip!

* Nit: import sort order

---------

Co-authored-by: Nikolaus West <niko@rerun.io>

* Fix annotation images sometimes drawn in the background. (rerun-io#1933)

This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup

* Fix backslashes in arkitscene rigid transformation path (rerun-io#1938)

* Fix backslashes in arkitscene rigid transformation path
Should be fixed properly by rerun-io#1937

* Use PosixPath instead of .replace("\\", "/")

---------

Co-authored-by: Nikolaus West <niko@rerun.io>

* Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942)

Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to

* Fix hovering depth clouds (rerun-io#1943)

We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again.

* change python workflow for testing purposes, remove windows and macos wheels

* add back one macos so the matrix is valid

* 2.5GB before GC kick in on web (rerun-io#1944)

* change name to depthai-viewer

* change pip install/uninstalls from rerun-sdk to depthai-viewer

* change all occurances of rerun-sdk to depthai-viewer

* change windows runner to windows-latest for now when using my personal gh

* Release `0.5.0` (rerun-io#1919)

* changelog

* 0.5.0-alpha.0

* more changelog

* re_format: fix implicit recursive feature flags

* publish_crates.sh: fix crate ordering

* Join threads at end of multi-threading example (rerun-io#1934)

* Add argument parsing to the rerun_demo (rerun-io#1925)

* More robust wait for exit condition during .serve() (rerun-io#1939)

* More robust wait for exit condition during .serve()

* lint

* Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936)

* Use zipfile python library instead of `unzip` command in arkitscene
Windows doesn't have unzip!

* Nit: import sort order

---------

Co-authored-by: Nikolaus West <niko@rerun.io>

* Fix annotation images sometimes drawn in the background. (rerun-io#1933)

This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup

* Fix backslashes in arkitscene rigid transformation path (rerun-io#1938)

* Fix backslashes in arkitscene rigid transformation path
Should be fixed properly by rerun-io#1937

* Use PosixPath instead of .replace("\\", "/")

---------

Co-authored-by: Nikolaus West <niko@rerun.io>

* changelog

* Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942)

Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to

* changelog

* Fix hovering depth clouds (rerun-io#1943)

We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again.

* changelog

* 2.5GB before GC kick in on web (rerun-io#1944)

* changelog

* 0.5.0

---------

Co-authored-by: Jeremy Leibs <jeremy@rerun.io>
Co-authored-by: Andreas Reich <andreas@rerun.io>
Co-authored-by: Nikolaus West <niko@rerun.io>

* Fix imu plots scrolling past their container

* fix bottom/top panel sizing after showing the spinner on config setting

* Lint error names in `map_err` (rerun-io#1948)

* Lint: Properly name errors in `map_err`

* Use correct names for errors

* fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU

* New dispatch-only workflow for running the lint-job (rerun-io#1950)

* Fix secret in dispatch_lint.yml

* Only maintain a single manual-dispatch job for testing workflows

* apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel

* Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951)

* Add other build parameterizations to manual_dispatch.yml

* Use proper if gates on the manual_dispatch.yml jobs

* Add ability to save cache to manual_disaptch.yml

* Standard case of inputs

* Add manual step for packaging to 'manual_dispatch.yml'

* add back panels when the underlying subscription reappears

* Fix crash for missing class ids causing zero sized texture (rerun-io#1947)

* Fix crash for missing class ids causing zero sized texture
Two things fixed actually:
* texture manager now checks for zero sized texture, this ripples out in a lot more error handling
* class id texture texture handles not having any classes gracefully

* Use Display for all errors

* typo

* Better naming of error

* Better docs and names

* Fix off-by-one error

* some use of `context`, change which error is implicitly converted on texture manager2d

---------

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15)

* make config ui scrollable, fix padding

* Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949)

* Move clippy_wasm/clippy.toml to under scripts

This is just to clean up the root a bit, and to move it closer
to where it is actually used.

* Fix comment typo

---------

Co-authored-by: Andreas Reich <andreas@rerun.io>

* change crate version to 0.6.0-alpha.0 (rerun-io#1952)

* New workflow_dispatch for building wheels for a PR

* initial light mode, luxonis depthai viewer

* Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml

* Fix run-wasm crash on trying to wait for server (rerun-io#1959)

This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server

* Update egui to latest and wgpu to 0.16 (rerun-io#1958)

* update to wgpu 0.16 and egui using this version

* shader fixup for type aliases and rectangle shader

* shader signed/unsigned shenanigans

* more signed/unsigned issues

* fix texture component count

* fix picking layer depth readback crash on web

* patch wgpu

* better texture size estimate

* fix patches

* Handle leaking of prerelease into alpha version (rerun-io#1953)

* Make device config panel remember it's height throughout loading

* hide time panel, make ai model dropdown wider

* Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963)

We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt)

* New manual workflow for running benches

* Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954)

There are 8 reusable workflow "components" that we can use to build different scenarios:

reusable_checks.yml - These are all the checks that run to ensure the code is formatted,
reusable_bench.yml - This job runs the benchmarks to check for performance regressions.
reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io
reusable_build_and_test_wheels.yml - This job builds the wheels, runs the
end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise
uploaded anywhere.
reusable_upload_wheels.yml- This job uploads the wheels to google cloud
reusable_build_web.yml - This job builds the wasm artifacts for the web.
reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this
only uploads to: app.rerun.io/commit/<commit>/
reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run.
Example summary can be found at:
https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html
This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations.

* New manual workflow for adhoc web builds

* Use new CI workflows for pull-request and merge to main (rerun-io#1955)

on_pull_request.yml includes the following pieces:
 - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc.
 - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd.
 - reusable_build_web.yml -- Verifies we can build the wasm
 - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing.
 - reusable_pr_summary.yml -- Create a manifest page with a link to the

on_push_main.yml includes the following pieces:
 - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc.
 - reusable_bench.yml -- Run the benchmarks
 - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms
 - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud
 - reusable_build_web.yml -- Builds the wasm bundle
 - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io
 - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g.

* Fix name of on_push_main.yml

* Fix usage of long commit in generate_prerelease_pip_index.py

* Try making pull-request workflows non-concurrent (rerun-io#1970)

* Try making pull-request workflows non-concurrent

* Concurrency groups for push_main as well

* Each sub-workflow needs its own name or they fight

* Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974)

* Another attempt to make jobs non-concurrent on a per-PR basis

* Move concurrency into the reusable job

* Jobs with duplicated instances still need separate concurrency keys based on platform

* Round to nearest color_index when doing color mapping (rerun-io#1969)

* Full (experimental) WebGPU support (rerun-io#1965)

* always build with unstable web sys apis

* Make shader Tint friendly

* expose webgl feature flag on re_renderer & re_viewer

* fix bug link on negative hexadecimal

* hardware tier is now created from wgpu adapter

* sort out build flags for webgpu & document building webviewer

* introduce shader text replacement workarounds to workaround current chrome issue

* latest egui master

* typo fix

* doc fixes, use if cfg! instead of attribute cfg

* move backend to rerun

* If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971)

* If there's a `{{ pr-build-summary }}`  in the PR description, update it.

* Add comment to the PR template

* Add pull-requests permission to pr_summary job

* Run the cube notebook on PR (rerun-io#1972)

* Run the cube notebook on PR

* Add notebooks to the build summary

* Use the new concurrency model

* reformat py files

* reformat

* fix pylint errors

* Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966)

* Add ability to manually run a web build to upload to an adhoc name

* Pass through ADHOC_NAME

* Add a concurrency criteria for the new adhoc job

* Make input description more explicit

* change entity paths

* merged wip albedo colormap into latest depth_cloud

* remove pointclouds generated in sdk

* fix compiler error, trying to compile frame.close for wasm

* Default to albedo texture for depth cloud, added support for mono albedo textures

* restart backend on failure, added oak_cam.device.close seems to really close the cam

* py lint fix

* don't run notebooks

* remove run-notebook dependency

---------

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
Co-authored-by: benjamin de charmoy <BenjaminDev@users.noreply.github.com>
Co-authored-by: Andreas Reich <andreas@rerun.io>
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
Co-authored-by: Pablo Vela <pablovela5620@gmail.com>
Co-authored-by: Nikolaus West <nikolaus.west@me.com>
Co-authored-by: Jeremy Leibs <jeremy@rerun.io>
Co-authored-by: h3mosphere <129932586+h3mosphere@users.noreply.github.com>
Co-authored-by: Urho Laukkarinen <urho.laukkarinen@gmail.com>
Co-authored-by: Nikolaus West <niko@rerun.io>

* Fix rerun lints

* rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions

* remove compiler warnings

* fix memory leak when app is in background

* Custom viewport UI (#4)

* custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible

* small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui

* custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly.

* improve auto layout to not split when only 3D or 2D view is available

* MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB

* Styling (#6)

* initial styling impl

* make buttons that should be small, small

* Runtime depth config and fix device selection ui

* comment

* Proper runtime depth config updates

* switch to Yolo v8

* add comma to label for non open zoo models

* split 2d + 3d cam view vertically instead of horizontally

* Tabify all panels (except for blueprint) (#7)

* initial styling impl

* make buttons that should be small, small

* Runtime depth config and fix device selection ui

* comment

* Proper runtime depth config updates

* switch to Yolo v8

* add comma to label for non open zoo models

* split 2d + 3d cam view vertically instead of horizontally

* Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive

* remove bottom panel, switch to RAW imu sensors

* XLink statistics initial implementation

* initial xlink throughput statistics impllementation, have to glow it up  a bit and maybe clean up the code

* Xlink and rerun rename (#9)

* Rename rerun py library to depthai_viewer

* bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts

* auto layout

* Fix maximize not working and add maximize for Stats tab

* delete profiling stuff that shouldn't have been commited

* mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split

* add docstring for  update_tree

* WIP auto layout can_create_mono_quad checker, not at all finished yet

* pass lint checks and bugfixes

* forgot to sort imports

* Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid

* pylints and fix Queue typehinting

* fix doc build

* try to pass pylints with py.typed

* ignore misc mypy errors

* forgot to run black formatter

* switch back to old ci

* sync mono camera settings

* fix web build

* Update scripts/version_util.py

---------

Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
Co-authored-by: benjamin de charmoy <BenjaminDev@users.noreply.github.com>
Co-authored-by: Andreas Reich <andreas@rerun.io>
Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
Co-authored-by: Pablo Vela <pablovela5620@gmail.com>
Co-authored-by: Nikolaus West <nikolaus.west@me.com>
Co-authored-by: Jeremy Leibs <jeremy@rerun.io>
Co-authored-by: h3mosphere <129932586+h3mosphere@users.noreply.github.com>
Co-authored-by: Urho Laukkarinen <urho.laukkarinen@gmail.com>
Co-authored-by: Nikolaus West <niko@rerun.io>

Author: 10 people

crates/re_ui/src/lib.rs

@@ -195,16 +195,19 @@ impl ReUi {
         label: &str,
         value: &mut Num,
         range: RangeInclusive<Num>,
-    ) where
+    ) -> egui::Response
+    where
         Num: egui::emath::Numeric,
     {
         ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
-            ui.add_sized(
+            let response = ui.add_sized(
                 [Self::box_width(), Self::box_height()],
                 egui::DragValue::new(value).clamp_range(range),
             );
             ui.label(egui::RichText::new(label).color(self.design_tokens.gray_900));
-        });
+            response
+        })
+        .inner
     }
 
     pub fn labeled_toggle_switch(&self, ui: &mut egui::Ui, label: &str, value: &mut bool) {

crates/re_viewer/src/app.rs

@@ -493,9 +493,12 @@ impl eframe::App for App {
             };
         }
 
-        if self.backend_handle.is_none() {
-            self.backend_handle = App::spawn_backend();
-        };
+        #[cfg(not(target_arch = "wasm32"))]
+        {
+            if self.backend_handle.is_none() {
+                self.backend_handle = App::spawn_backend();
+            };
+        }
 
         if self.startup_options.memory_limit.limit.is_none() {
             // we only warn about high memory usage if the user hasn't specified a limit

crates/re_viewer/src/ui/device_settings_panel.rs

@@ -152,20 +152,35 @@ impl DeviceSettingsPanel {
                                                     .selected_device
                                                     .supported_left_mono_resolutions
                                                 {
-                                                    ui.selectable_value(
-                                                        &mut device_config.left_camera.resolution,
-                                                        *res,
-                                                        format!("{res}"),
-                                                    );
+                                                    if ui
+                                                        .selectable_value(
+                                                            &mut device_config
+                                                                .left_camera
+                                                                .resolution,
+                                                            *res,
+                                                            format!("{res}"),
+                                                        )
+                                                        .changed()
+                                                    {
+                                                        device_config.right_camera.resolution =
+                                                            *res;
+                                                    }
                                                 }
                                             },
                                         );
-                                        ctx.re_ui.labeled_dragvalue(
-                                            ui,
-                                            "FPS",
-                                            &mut device_config.left_camera.fps,
-                                            0..=120,
-                                        );
+                                        if ctx
+                                            .re_ui
+                                            .labeled_dragvalue(
+                                                ui,
+                                                "FPS",
+                                                &mut device_config.left_camera.fps,
+                                                0..=120,
+                                            )
+                                            .changed()
+                                        {
+                                            device_config.right_camera.fps =
+                                                device_config.left_camera.fps;
+                                        }
                                         ctx.re_ui.labeled_checkbox(
                                             ui,
                                             "Stream",
@@ -191,20 +206,34 @@ impl DeviceSettingsPanel {
                                                     .selected_device
                                                     .supported_right_mono_resolutions
                                                 {
-                                                    ui.selectable_value(
-                                                        &mut device_config.right_camera.resolution,
-                                                        *res,
-                                                        format!("{res}"),
-                                                    );
+                                                    if ui
+                                                        .selectable_value(
+                                                            &mut device_config
+                                                                .right_camera
+                                                                .resolution,
+                                                            *res,
+                                                            format!("{res}"),
+                                                        )
+                                                        .changed()
+                                                    {
+                                                        device_config.left_camera.resolution = *res;
+                                                    }
                                                 }
                                             },
                                         );
-                                        ctx.re_ui.labeled_dragvalue(
-                                            ui,
-                                            "FPS",
-                                            &mut device_config.right_camera.fps,
-                                            0..=120,
-                                        );
+                                        if ctx
+                                            .re_ui
+                                            .labeled_dragvalue(
+                                                ui,
+                                                "FPS",
+                                                &mut device_config.right_camera.fps,
+                                                0..=120,
+                                            )
+                                            .changed()
+                                        {
+                                            device_config.left_camera.fps =
+                                                device_config.right_camera.fps;
+                                        }
                                         ctx.re_ui.labeled_checkbox(
                                             ui,
                                             "Stream",