Need to support more efficient cacheing

[fisherab]

Currently with two level storage writing to the archive is performed as quickly as possible. In the case of archiving by dataset if datafiles are added to a dataset over a period of time then many different copies of the dataset will be written out which is not good for tape storage. I propose to add another option to delay writing to archive until such time as the disk space is needed. The benefit for archiving by datafile rather than be dataset will be much less and only means that if a datafile is created and then deleted before it is written to archive then no time/space is wasted doing the write to archive. With writing as late as possible the disk cache must be trusted not to lose data (RAID or frequent backups or ...)

[RKrahl]

With writing as late as possible the disk cache must be trusted not to lose data (RAID or frequent backups or ...)

That is the drawback that I see with this approach. At the moment, one does not need a backup for main storage at all, the archive storage is the backup. If one needs to introduce backup for the main storage, all possible benefit of this approach is gone: the cost for the backup will be at least as much as the current quick write to archive.