OSS 2.0 GA

[stuartcarnie]

Step 1: Forward port 1.x tsdb, Flux

Step 1a: Initial port 1.x tsdb, Flux

• Est: 3-4 weeks
• Current 1.x Flux (0.66.1)
• Includes replacement abstractions to support storage read / write (e.g. MetaClient)
• V2 reads and writes should work

Success

• V2 Reads work
• V2 Writes work
• Create and delete buckets work
• Onboarding / setup works

Step 2: Parallel tracks

These can run in parallel

2a: 1.x Compatibility

Compute Team

• Est: 2-3 weeks?
• Port 1.x InfluxQL + Cloud 2.x changes and v1 /query via Add InfluxQL and V1 query API #19304
• v1 /write via Add V1 write API #19303

Success

• V1 /query (InfluxQL) works
• V2 /write works

2b: Flux Upgrade

Flux Team

• Port latest Flux release via Upgrade Flux to latest version #19302

Success

• Running latest Flux
• Build steps socialized / updated

2c: Migration Tooling / Retention Policy / Backup Restore

Storage Team, Compute Team

• In-place upgrade of metadata, file moving, etc via In-place OSS 1.x to 2.x upgrade #19308
• Retention policy service Port 1.x Retention Policy service #19309
  • (port from 1.x such that influxd correctly drops shards based on retention policy)
• Backup / Restore
• Port influx_inspect tools via Port influx_inspect tools to influxd inspect subcommand #19315

Other work:

• Unknown unknowns that won't reveal themselves until we're in the trenches

---

Design Discussions

On Disk Structure

Every bucket will have its own database and retention policy. To state it another way, every database will have exactly one retention policy, which is the bucket.

Creating two buckets, bucket-a and bucket-b would result in the following:

data/
  bucket-a/
    autogen/
  bucket-b/
    autogen/

NOTE: The bucket metadata will be separate from the TSM 1 metadata (database name, retention policy name, shards, etc). As is with 1.x, the database name and retention policy names are immutable. Other metadata, such as shard duration may be exposed via subcommands of the influx CLI tool.

Migration

Given the above property, if the user has the following existing structure:

data/
    my-db-1/
        default/
        1year
    my-db-2/
        autogen/
        1year

The resulting layout on disk will be:

data/
    my-db-1-default/
        autogen/
    my-db-1-1year/
        autogen/
    my-db-2-autogen/
        autogen/
    my-db-2-1year/
        autogen/

[stuartcarnie]

OSS 2.0 is generally available. See the official release announcement.