Proposal on DB Storage, tune rocksdb options for store multiple SSD/HDD.

[trustfarm-dev]

Before filing a new issue, please provide the following information.

I'm running:

• Which Parity version?: 1.xx.xxx ~ latest
• Which operating system?: All Platform
• How installed?: Release binary and self-compiled all.
• Are you fully synchronized?: yes
• Which network are you connected to?: ethereum / ropsten / kovan / ...
• Did you try to restart the node?: yes

Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.

actual
Blockheights 58986xxx , FULL Archive mode sync : Storage is 1.24TB in 2TB SSD.
It is increasing very stiff. when 2TB SSD will be full soon.

suggestion
For solve this kind of storage scalability , I suggest parity use multiple [dbpaths storage options provided by rocksdb.](https://github.com/facebook/rocksdb/blob/17339dc2f39a87a9cf097e056a59560b0795ff6e/include/rocksdb/options.h#L468 ,"db_paths options")

  // For example, you have a flash device with 10GB allocated for the DB,
  // as well as a hard drive of 2TB, you should config it to be:
  //   [{"/flash_path", 10GB}, {"/hard_drive", 2TB}]

Because latest fast block importing and synchronizing is done in SSD , old and infrequent referencing block data will be stored in HDD.

It will helpful, shorterm storage scalability.

[Tbaut]

@andresilva @debris @folsen what's your opinion on this?

[folsen]

It's something we've talked about, but almost definitely won't be that easy. Worth investigating though.

[andresilva]

IMO this should be solved at a lower level, i.e. pooling disks together using RAID/LVM or a filesystem with multi-disk capabilities (ZFS/btrfs). RocksDB, which is the database we're currently using, doesn't have any support for multiple disks. We'd have to shard the database to handle this use case and I don't see it providing any benefit over just creating a large virtual disk (both from a performance perspective and code complexity).