Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][CTI] Event enrichment search strategy #101553

Merged
merged 7 commits into from
Jun 15, 2021

Conversation

rylnd
Copy link
Contributor

@rylnd rylnd commented Jun 8, 2021

Summary

This adds a new search strategy implementation to perform simplified "indicator match" logic in service of a new CTI feature. As design for the UI component of this is still being finalized, I'm opening this backend-only work as a self-contained PR to get the ball rolling.

At a high level, this strategy:

  1. Converts an allowlist of field parameters into named queries to be sent to elasticsearch
  2. Transforms the response into an array of enrichments, as specified in the enrichments RFC

Checklist

For maintainers

@rylnd rylnd added v8.0.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0 release_note:feature Makes this part of the condensed release notes Team: CTI labels Jun 8, 2021
@rylnd rylnd self-assigned this Jun 8, 2021
rylnd added 5 commits June 10, 2021 14:14
This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)
A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.
* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout
@rylnd rylnd force-pushed the ad_hoc_enrichment branch from ada56e4 to d5f420c Compare June 10, 2021 19:14
@rylnd rylnd changed the title [Security Solution][CTI] Ad hoc event enrichment [Security Solution][CTI] Event enrichment search strategy Jun 10, 2021
@rylnd rylnd requested a review from ecezalp June 10, 2021 19:33
@rylnd rylnd marked this pull request as ready for review June 10, 2021 19:33
@rylnd rylnd requested a review from a team as a code owner June 10, 2021 19:33
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@rylnd
Copy link
Contributor Author

rylnd commented Jun 14, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @rylnd

Copy link
Contributor

@ecezalp ecezalp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -44,3 +44,16 @@ export const SORTED_THREAT_SUMMARY_FIELDS = [
INDICATOR_FIRSTSEEN,
INDICATOR_LASTSEEN,
];

export const EVENT_ENRICHMENT_INDICATOR_FIELD_MAP = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about

const specialFields = ["foo", "bar"]

const desiredFieldmap = specialFields.reduce((acc, item) => {
  acc[item] = `${DEFAULT_INDICATOR_SOURCE_PATH}.${item}`; 
  return acc;
}, {})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source.ip and destination.ip break that pattern, unfortunately.

import { IEsSearchResponse } from 'src/plugins/data/public';

import {
CtiEventEnrichmentRequestOptions,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we consider treating CTI as an acronym in general?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, the old "acronyms in camel case" debate! I know it well 😄 .

My personal preference is to use pascal case for acronyms longer than two characters: CtiEventEnrichment or HtmlButton, but I agree that consistency is tantamount. I don't think we have any existing constants that break this pattern, but if you've been taking another approach in some parallel work let's discuss.

@rylnd rylnd merged commit 4d921ff into elastic:master Jun 15, 2021
@rylnd rylnd deleted the ad_hoc_enrichment branch June 15, 2021 22:00
cuff-links pushed a commit to cuff-links/kibana that referenced this pull request Jun 15, 2021
…1553)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd added a commit to rylnd/kibana that referenced this pull request Jun 15, 2021
…1553)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd added a commit that referenced this pull request Jun 16, 2021
…102275)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:feature Makes this part of the condensed release notes Team: CTI Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants