[Security Solution][Detections] Ignore interim results in ML Rule anomalies query #90316
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We were incorrectly including records with is_interim: true in our query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then dipped below after it was finalized.
rylnd
added
bug
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
FrankHassanabad
approved these changes
Feb 4, 2021
spong
approved these changes
Feb 5, 2021
There was a problem hiding this comment.
LGTM as well! Thanks for seeing this one through @rylnd! And all the debugging, and even squeezing some type fixes in here as well too! 😉 🌶️ ⏳
Gonna sneak this blog post about bucket spans in here for anyone passing by -- learned a ton from this post about the interworkings of ML, so definitely worth giving a gander!
https://www.elastic.co/blog/explaining-the-bucket-span-in-machine-learning-for-elasticsearch
benwtrent
reviewed
Feb 5, 2021
Comment on lines
+114
to
+121
| expect(filters).toEqual( | ||
| expect.arrayContaining([ | ||
| { | ||
| term: { | ||
| is_interim: false, | ||
| }, | ||
| }, | ||
| ]) |
There was a problem hiding this comment.
FYI, this means that alerts won't fire for anomalies until after the bucket has been completed.
For short buckets, this is cool (like, 15m). But if there are jobs with 1hr+ length buckets, it means an anomaly won't trigger this alert until after a whole hour.
There was a problem hiding this comment.
We're going to meet with @randomuserid and the @elastic/protections folks to review the existing ML Job bucket times, and corresponding Rule's interval + lookback to ensure we don't have any gaps here. I believe @randomuserid took this all into account when initially developing the jobs/rules, but we'll ensure to do an audit here to make sure everything is 👍.
Thanks again for all your help and feedback around debugging this issue, really grateful for your feedback 🙂
There was a problem hiding this comment.
Most of the jobs have 15m buckets. I set the lookbacks are with the assumption that anomaly may take 15-30 minutes to be picked up by the rules depending on where the bucket is when the rule fires. Some, like CloudTrail, have longer lookbacks, because those pipelines have much more variable latency.
|
@elasticmachine merge upstream |
💛 Build succeeded, but was flaky
Test FailuresKibana Pipeline / general / Firefox XPack UI Functional Tests.x-pack/test/functional/apps/status_page/status_page·ts.Status page Status Page allows user to navigate without authenticationStandard OutStack TraceMetrics [docs]
History
To update your PR or re-run it, just comment with: |
rylnd
added a commit
to rylnd/kibana
that referenced
this pull request
Feb 9, 2021
We were incorrectly including records with is_interim: true in our query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then dipped below after it was finalized. Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd
added a commit
that referenced
this pull request
Feb 10, 2021
We were incorrectly including records with is_interim: true in our query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then dipped below after it was finalized. Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd
added a commit
to rylnd/kibana
that referenced
this pull request
Feb 10, 2021
We were incorrectly including records with is_interim: true in our query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then dipped below after it was finalized. Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd
added a commit
that referenced
this pull request
Feb 10, 2021
We were incorrectly including records with is_interim: true in our query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then dipped below after it was finalized. Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
We were incorrectly including records with
is_interim: truein our anomalies query, which lead to false positive signals if the rule executed while an anomaly's score was (temporarily) above the specified threshold, but then drops below after re-evaluation. Whilerecord_scoremay continue to be re-evaluated afteris_interim: falseis set, any changes at that point would be in reaction to new anomalous data, which would have its own alert.Checklist
Delete any items that are not applicable to this PR.
For maintainers