share: EDSStore definition and constructor

[Wondertan]

Context

FNs/BNs keep an EDSStore to manage every EDS on the disk. The EDSStore type is introduced in the share pkg. Each EDS, together with its Merkle Proofs, serializes into a CARv1 file. All the serialized CARv1 file blobs are mounted on DAGStore via Local FS Mounts and registered as Shards.

The introduced EDSStore also maintains (via DAGStore) a top-level index enabling granular and efficient random access to every share and/or Merkle proof over every registered CARv1 file. The EDSStore provides a custom Blockstore interface implementation to achieve access. The main use-case is randomized sampling over the whole chain of EDS block data and getting data by namespace.

Implementation Details

NOTES:

• EDSStore should have lifecycle methods(Start/Stop).
• The approximate definition is given below and may change in the actual implementation.

type EDSStore struct {
 basepath string 
 dgstr dagstore.DAGStore
 topIdx index.Inverted 
 carIdx index.FullIndexRepo
 mounts  *mount.Registry
 ...
}

// NewEDSStore constructs EDStore over OS directory to store CARv1 files of EDSes and indices for them.
// Datastore is used to keep the inverted/top-level index.
func NewEDSStore(basepath string, ds datastore.Batching) *EDSStore {
 topIdx := index.NewInverted(datastore)
 carIdx := index.NewFSRepo(basepath + "/index")
 mounts := mount.NewRegistry()
 r := mount.NewRegistry()
 err = r.Register("fs", &mount.FSMount{FS: os.DirFS(basePath + "/eds/")}) // registration is a must 
 if err != nil {
  panic(err)
 }
 
 return &EDSStore{
  basepath: basepath,
  dgst: dagstore.New(dagstore.Config{...}),
  topIdx: index.NewInverted(datastore),
  carIdx: index.NewFSRepo(basepath + "/index")
  mounts: mounts,
 }   
}

Refs

Depends on #878