Description
Several Rook users have expressed interest in per-bucket quotas. Specifically max objects and max size quotas.
In COSI's upstream design planning, we've come to the agreement that these 2 quotas don't fit into the main COSI API. The major cloud providers don't provide these types of quotas, and so the interface wouldn't be portable. Instead, vendors like Ceph should use the opaque parameters on Bucket(Access)Classes to choose these quotas.
Because some Rook users also use Noobaa, it would be nice if COSI objects (BucketClass, for this design) can be portable between Ceph-COSI and Noobaa-COSI. I've done some pre-planning with Noobaa to reach an agreed-upon spec.
kind: BucketClass
# ...
parameters:
maxObjectsPerBucket: <int>
maxSizePerBucket: <string, parsable to Kubernetes resource.Quantity>Kubernetes resource.Quantity spec: https://pkg.go.dev/k8s.io/apimachinery@v0.28.2/pkg/api/resource#Quantity
The Noobaa team prefers to use the maxSizePerBucket as defined here rather than the config Ceph uses underneath which is max_size_kb. Kilobytes are very granular when many of these quotas will be Gigabytes for large users, so the resource.Quantity representation is preferred.
We should be sure to have clear error messaging for users when this quantity doesn't parse correctly.