Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

re_datastore: puffin probes! #942

Merged
merged 7 commits into from
Jan 27, 2023
Merged

re_datastore: puffin probes! #942

merged 7 commits into from
Jan 27, 2023

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Jan 26, 2023

This is pretty much the only thing left before we can close #443 (despite the issue telling otherwise 😬).

Copy link
Member

@emilk emilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice with scopes!

What does the puffin flamegraph of nyud look like now?

In the other crates we only include puffin on native, because it adds quite a bit of wasm for very little gain (timers only have 1ms accuracy on web).

So all other crates do this:

/// Profiling macro for puffin
#[doc(hidden)]
#[macro_export]
macro_rules! profile_function {
    ($($arg: tt)*) => {
        #[cfg(not(target_arch = "wasm32"))]
        puffin::profile_function!($($arg)*);
    };
}

/// Profiling macro for puffin
#[doc(hidden)]
#[macro_export]
macro_rules! profile_scope {
    ($($arg: tt)*) => {
        #[cfg(not(target_arch = "wasm32"))]
        puffin::profile_scope!($($arg)*);
    };
}

and then use crate::profile_function etc.

@teh-cmc
Copy link
Member Author

teh-cmc commented Jan 27, 2023

nyud (release, no history):
image

nyud (release, 0.6s history for point cloud (any more than that crashes!!)):
image

I'll open an issue for the weird behaviors I'm seeing with the history.

@teh-cmc teh-cmc requested a review from emilk January 27, 2023 08:18
@teh-cmc teh-cmc merged commit f9ba39c into main Jan 27, 2023
@teh-cmc teh-cmc deleted the cmc/datastore/probes branch January 27, 2023 09:47
@emilk
Copy link
Member

emilk commented Jan 29, 2023

I think might have gotten a bit overboard with the number of profiling scopes. This is the "plot" demo:

image

Each puffin call has an overhead of around 100ns-200ns, so that's a per-frame overhead here of ~3-6ms, which is too much. We should try to keep the number of total profile scopes below 10k.

…or, perhaps SceneTimeSeries::load_Scalars is calling ::get way too often, and we should fix that. I'm not looking any deeper right now (I'm busy with other stuff!)

There is also some levels in the profiling missing - I don't immediately understand what part of SceneTimeSeries::load_scalars is actually calling the underlying get methods.

(NOTE: this overhead is only when a profiler is actually connected. When profiling is off, the per-call overhead drops from ~100ns to ~1ns)

EDIT:

@emilk emilk mentioned this pull request Jan 29, 2023
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

re_datastore: all around performance pass tracking issue
2 participants