Taxii download problems caused by wrong time resolution for added-after query

[ukuellmarks]

Description

We provide a Taxi 2.1 server instance for our customers, and we encountered a problem while openCPTI consumed data from the server.

The server limits the returned objects during a query to 100 objects and sets the more to true and the next flag in the response. Opencti will page through the responses until we respond with more false. The following query triggered by the workers uses the X-TAXII-Date-Added-Last Value returned with the last response but uses a wrong time resolution.

The server returns X-TAXII-Date-Added-Last: 2025-02-13T19:41:56.302472Z but the timestamp then used 2025-02-13T19:41:56.302Z during the query.

This behavior can lead the fetching process to consume data again and again.
On our server, we are doing batch exports for several hundred objects, and the timestamps are very close to each other, so it happens from time to time that if this difference is just some microseconds, an amount of data gets requested again, and again because of the millisecond resolution.

The taxii 2.1 spec specifies that the timestamp needs a microsecond resolution to avoid this kind of problem.
https://docs.oasis-open.org/cti/taxii/v2.1/os/taxii-v2.1-os.html#_Toc31107512 exactly

Environment

1. OS (where OpenCTI server runs): { e.g. Mac OS 10, Windows 10, Ubuntu 16.4, etc. }
2. OpenCTI version: { e.g. OpenCTI 1.0.2 }
3. OpenCTI client: { e.g. frontend or python }
4. Other environment details:

Reproducible Steps

descript above

Expected Output

Store and use the X-TAXII-Date-Added-Last in microsecond resolution

[romain-filigran]

@ukuellmarks : Thank you for reporting this problem. We will investigate.

[romain-filigran]

@aHenryJard: can we look together