Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port 1.x influxd backup and restore commands #19316

Closed
2 tasks
stuartcarnie opened this issue Aug 12, 2020 · 3 comments
Closed
2 tasks

Port 1.x influxd backup and restore commands #19316

stuartcarnie opened this issue Aug 12, 2020 · 3 comments
Assignees
Labels
area/tooling area/2.x OSS 2.0 related issues and PRs team/storage

Comments

@stuartcarnie
Copy link
Contributor

stuartcarnie commented Aug 12, 2020

What

Users will require backup and restore capabilities. Port the TSM backup and restore commands from 1.x and integrate with 2.x.

  • influx backup
  • influx restore

2.0 Backup (prior to TSM 1.x transition)

The 2.0 backup / restore command can be summarized as the following:

  • Online backup and offline restore
  • Backup all meta data
  • Backup all buckets

This approach is limited and largely governed by the storage architecture, best described as a single database and single shard. All buckets were co-mingled in a single set of TSM files.

2.0 Backup

The OSS 1.8 backup system is considerably more powerful and flexible per the documentation.

With the transition to the TSM 1.x storage engine, the on-disk structure lends itself to the same backup / restore features. This layout is discussed in #19114, however, it is included below for reference:

data/
  bucket-a/
    autogen/
      <shardid-1>/
        *.tsm
      <shardid-2>/
        *.tsm
  bucket-b/
    autogen/
      <shardid-1>/
        *.tsm

In summary, each bucket is now represented as a single database and retention policy and is made up of one or more shards.

Given that, many of the features of 1.8 can be ported, including

  • Online backup and restore
  • Backup of individual buckets (database / retention policy in 1.x parlance)
  • Backup of individual shards (single shard from a single bucket)
@stephanie-engel
Copy link
Contributor

Once this fix is implemented, the litmus backup and restore tests will need to be un-skipped.

@benbjohnson
Copy link
Contributor

Update: Backup/restore PR is up: #19864

@8none1 8none1 modified the milestones: Sprint 20-Q4-2, Sprint 20-Q4-3 Nov 2, 2020
@benbjohnson
Copy link
Contributor

Completed & merged into master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tooling area/2.x OSS 2.0 related issues and PRs team/storage
Projects
None yet
Development

No branches or pull requests

5 participants