Skip to content
This repository was archived by the owner on Aug 30, 2019. It is now read-only.

cmd/trace-agent: reduce histogram calls in concentrator #522

Merged
merged 2 commits into from
Nov 8, 2018

Conversation

gbbr
Copy link
Contributor

@gbbr gbbr commented Nov 7, 2018

Attemtp to reduce UDP hits and flooding.

We had reports of excessive UDP packets (>3000 per 10 seconds) for metric datadog.trace_agent.err_distribution.len, all with values of zero. Seems uncalled for. This change should hopefully alleviate.

@gbbr gbbr requested a review from LotharSee November 7, 2018 15:24
@gbbr gbbr force-pushed the gbbr/stats-reduce branch 2 times, most recently from 4875242 to a782fe6 Compare November 7, 2018 16:10
@gbbr
Copy link
Contributor Author

gbbr commented Nov 7, 2018

I reduced the rate to 0.2, but not sure if this will alleviate the problem enough.

@gbbr gbbr added this to the 6.7.0 milestone Nov 7, 2018
@LotharSee
Copy link

Good catch, contrary to most metrics which we batch internally/report periodically, this one was done once per distribution.

In practice, we never look at it. I'd suggest to simply remove it.

gbbr added 2 commits November 8, 2018 09:53
These metrics were causing excessive UDP hits, errors and flooding.
This change removes them.
This change moves the `bucket.Export` call to after the precondition
where it is needed in an attempt to save some memory and CPU.
@gbbr gbbr force-pushed the gbbr/stats-reduce branch from a782fe6 to 47278f4 Compare November 8, 2018 08:56
@gbbr
Copy link
Contributor Author

gbbr commented Nov 8, 2018

Great! That's probably the best thing to do. Note that I've added another commit with a small improvement. PTAL.

@gbbr gbbr merged commit 4a56f12 into master Nov 8, 2018
@gbbr gbbr deleted the gbbr/stats-reduce branch November 8, 2018 09:41
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants