Handle expiration of storage groups

[cthulhu-rider]

• based on and blocked by Use well-known expiration attribute for SGs neofs-api#205

Overview

Storage groups have limited lifetime in NeoFS. All system parts should handle SG expiration: storage nodes shouldn't store expired STORAGE_GROUP objects, inner ring nodes should not perform data audit of them.

Inner ring

Data audit uses SGSource primitive to get list of storage groups to audit data for. The implementation selects objects from the container. We need to complicate some of the components with the logic which excludes expired storage groups.

I propose to have a component that allows you to select storage groups in a container:

type StorageGroup struct {
	Descriptor storagegroup.StorageGroup

	ExpiresAfter uint64 // epoch
}

type StorageGroupSource interface {
	Iterate(container cid.ID, handle func(StorageGroup)) error
}

// implements StorageGroupSource via NeoFS API protocol.
type neofsContainerClient struct {
	// wraps NeoFS SDK client
}

// StorageGroupSource mediator which excludes expired storage groups.
type aliveStorageGroups struct {
	// wraps base StorageGroupSource with network state
}

The data auditing component will use this interface to sequentially process storage groups.

Storage node

For these network members, storage group lifetimes are only relevant for garbage collection. Based on the fact that the SG descriptor is embedded in the binary structure of the object, and the expiration mechanism is the same as other types of objects, STORAGE_GROUP objects do not require additional logic to be processed by GC implementation. Within the framework of this task, it is only possible to double-check this fact.

NeoFS CLI

User interface should print expiration date in epochs. Seems like it is already implemented

neofs-node/cmd/neofs-cli/modules/storagegroup.go

Line 260 in 14ea7b2

cmd.Printf("Expiration epoch: %d\n", sg.ExpirationEpoch())

I'd pay attention on the approach of storage group descriptor decoding

neofs-node/cmd/neofs-cli/modules/storagegroup.go

Line 257 in 14ea7b2

err = sg.Unmarshal(buf.Bytes())

Since expiration is written in object attribute it's not enough to decode the object payload. I propose to create a function which links Object and StorageGroup types:

// StorageGroupFromObject reads StorageGroup from the Object instance.
func ReadStorageGroupFromObject(sg *storagegroup.StorageGroup, obj object.Object)

May be we also need a way to select alive storage groups only in storagegroup command.

[alexvanin]

PoC is not blocked, but it should be merged after API update.