Skip to content

feat(provisioning): extra format options (mkfs) added #335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

mhkarimi1383
Copy link
Member

@mhkarimi1383 mhkarimi1383 commented Sep 19, 2024

Pull Request template

Please, go through these steps before you submit a PR.

Why is this PR required? What issue does it fix?:

adding extra format (mkfs) options to newly created Volumes

also switched from deprecated utils/monut to mount-utils to be able to add this feature and moving from an old package to a maintained one

What this PR does?:

Refactor to new mount utils package and change format and mount function

Does this PR require any upgrade changes?:

No, But requires document updates for StorageClasses

If the changes in this PR are manually verified, list down the scenarios covered::

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

  • Fixes Extra mkfs options #296
  • PR Title follows the convention of <type>(<scope>): <subject>
  • Has the change log section been updated?
  • Commit has unit tests
  • Commit has integration tests
  • (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
  • (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them:

@dsharma-dc
Copy link
Contributor

How is the new formatOptions going to be set by user? via storageclass? There is no mkfsOptions unlike mountOptions as provided by storageclass.
Please write a test as well to show how this is used.

@tiagolobocastro
Copy link
Contributor

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

@mhkarimi1383
Copy link
Member Author

mhkarimi1383 commented Dec 5, 2024

Hi

formatOptions is added to MountInfo just like mountOptions

So, It will work if you add it to storageclass

@mhkarimi1383
Copy link
Member Author

Also there is no test for the mounter itself

@dsharma-dc
Copy link
Contributor

dsharma-dc commented Dec 6, 2024

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

@dsharma-dc
Copy link
Contributor

Also there is no test for the mounter itself

You can run this manually and document here the steps to set formatOptions.

@mhkarimi1383
Copy link
Member Author

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

It's my first time that I'm contributing to this project so I just looked for where mountOptions is handled to handle formatOptions too

@mhkarimi1383
Copy link
Member Author

Hello again

I'm using NixOS, and I can't run make, waiting on #351

@mhkarimi1383
Copy link
Member Author

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

I checked out more and I saw that we have to put this on where we are mounting volume for the first time, So there is no need to add this into GetVolAndMountInfo, we have to handle that just like mountOptions

@mhkarimi1383
Copy link
Member Author

Not able to test due to #353 and #352

@dsharma-dc
Copy link
Contributor

@mhkarimi1383
Also, please refer to the change https://github.com/openebs/zfs-localpv/pull/542/files . Something similar is likely required here also for these formatOptions to be usable.
cc: @abhilashshetty04 @Abhinandan-Purkait

@mhkarimi1383
Copy link
Member Author

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

@dsharma-dc
Copy link
Contributor

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

Would you like to do that as part of this PR? or instead try using a mount-utils that might work with go 1.19.

@mhkarimi1383
Copy link
Member Author

I can do both, whatever is Ok :)

@dsharma-dc
Copy link
Contributor

I can do both, whatever is Ok :)

Thanks, I'd say then try using a compatible mount-utils version(I hope no other issues pop up), and Go upgrade can be taken up as a new PR.

@mhkarimi1383
Copy link
Member Author

@dsharma-dc

Ok, I will revert my changes to go.mod, etc. And will change the package again

@mhkarimi1383
Copy link
Member Author

mhkarimi1383 commented Dec 22, 2024

I have fixed package versions and switched to a correct version of mount utils

And I was able to build project successfully and tests passed

@abhilashshetty04
Copy link
Member

But I don't see the start of my tests in the logs, Also I have not set any -F option

@mhkarimi1383 , Let me have a look at it.

@abhilashshetty04
Copy link
Member

@mhkarimi1383 I did try locally also. I got stuck myself because of PVC not getting Bound. FInally figured out that it was due to GO int limitation which was causing parsing issue. All test in develop passes after this fix.

#387

For failure in current PR. It seems to be failing during the mount. Pod is not going into Running state. This PR involves changes to mount logic also. Can you please test it manually also.

E0420 10:03:54.534370 1 grpc.go:79] GRPC error: rpc error: code = Internal desc = failed to format and mount the volume error: format of disk "/dev/lvmvg/pvc-bb191605-f11e-4e6d-94b1-a6ee8a2f80cf" failed: type:("ext4") target:("/var/lib/kubelet/pods/7cbb193c-b511-4ffb-b38f-1018d4f1c7da/volumes/kubernetes.io~csi/pvc-bb191605-f11e-4e6d-94b1-a6ee8a2f80cf/mount") options:("defaults") errcode:(exit status 1) output:(mke2fs 1.47.0 (5-Feb-2023)
mkfs.ext4: invalid blocks '-F' on device ''

@mhkarimi1383
Copy link
Member Author

@abhilashshetty04
Copy link
Member

abhilashshetty04 commented Apr 23, 2025

@abhilashshetty04

Will apply https://patch-diff.githubusercontent.com/raw/openebs/lvm-localpv/pull/387.patch locally and will do build and test

This patch is unrelated to the errors that we have on this PR. You might have to look into mount code changes that was done.

@mhkarimi1383
Copy link
Member Author

mhkarimi1383 commented Apr 24, 2025

Found the problem with this,

package main

import (
	"fmt"
	"strings"
)

func main() {
	splitted := strings.Split("", " ")
	fmt.Println(len(splitted))
	fmt.Println(splitted[0])
}

will print

1

Notice empty line

This will cause the format command to be something like

mkfs.ext4 "" -F -m0 ./a.img

@mhkarimi1383
Copy link
Member Author

Now every thing is correct, I just have to work on the test case

I0424 20:51:44.835644       1 grpc.go:72] GRPC call: /csi.v1.Node/NodePublishVolume requests {"target_path":"/var/lib/kubelet/pods/e081a5a3-346c-45fd-bebd-770fdb3f78de/volumes/kubernetes.io~csi/pvc-eb7f5060-9c8e-46ff-88bd-a7fd04aaf571/mount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/ephemeral":"false","csi.storage.k8s.io/pod.name":"format-options-verifier-7d5d49cf75-nw4lp","csi.storage.k8s.io/pod.namespace":"openebs","csi.storage.k8s.io/pod.uid":"e081a5a3-346c-45fd-bebd-770fdb3f78de","csi.storage.k8s.io/serviceAccount.name":"default","openebs.io/cas-type":"localpv-lvm","openebs.io/format-options":"-b 4096 -N 5000000","openebs.io/volgroup":"lvmvg","storage.kubernetes.io/csiProvisionerIdentity":"1745527750375-7479-local.csi.openebs.io"},"volume_id":"pvc-eb7f5060-9c8e-46ff-88bd-a7fd04aaf571"}
I0424 20:51:44.844322       1 mount_linux.go:516] Disk "/dev/lvmvg/pvc-eb7f5060-9c8e-46ff-88bd-a7fd04aaf571" appears to be unformatted, attempting to format as type: "ext4" with options: [-b 4096 -N 5000000 -F -m0 /dev/lvmvg/pvc-eb7f5060-9c8e-46ff-88bd-a7fd04aaf571]

@mhkarimi1383
Copy link
Member Author

Also I found test problem

Containers are not able to do tune2fs since they only get a mapped device I think

@mhkarimi1383
Copy link
Member Author

Found the issue, We are using deployments to test things but here I only need the container to success fully exit

@mhkarimi1383 mhkarimi1383 force-pushed the feat-mkfs-options branch 2 times, most recently from 40d40c3 to eb67ccf Compare April 26, 2025 10:45
@mhkarimi1383
Copy link
Member Author

@abhilashshetty04, @tiagolobocastro

I have done with tests ;)

@mhkarimi1383
Copy link
Member Author

Also it's good to make tests to run in parallel in CI I think

@abhilashshetty04
Copy link
Member

Also it's good to make tests to run in parallel in CI I think

Lets do a separate PR for that. We dont have bors or similar utility inplace for that right now in localpv repo.

@mhkarimi1383
Copy link
Member Author

Also it's good to make tests to run in parallel in CI I think

Lets do a separate PR for that. We dont have bors or similar utility inplace for that right now in localpv repo.

Yeah we should check if we don't have any naming conflicts, etc in resources that are creating in K8s but ginkgo has support for parallel tests

@abhilashshetty04
Copy link
Member

Also it's good to make tests to run in parallel in CI I think

Lets do a separate PR for that. We dont have bors or similar utility inplace for that right now in localpv repo.

Yeah we should check if we don't have any naming conflicts, etc in resources that are creating in K8s but ginkgo has support for parallel tests

Got it. We definately check it out.

Copy link
Member

@abhilashshetty04 abhilashshetty04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Please squash commits before merging @mhkarimi1383

@mhkarimi1383
Copy link
Member Author

Thanks, Please squash commits before merging @mhkarimi1383

I will

@mhkarimi1383 mhkarimi1383 requested a review from a team as a code owner April 28, 2025 08:10
@mhkarimi1383 mhkarimi1383 force-pushed the feat-mkfs-options branch 3 times, most recently from 1421cdb to 3931703 Compare April 28, 2025 09:23
Signed-off-by: Muhammed Hussein Karimi <info@karimi.dev>
@mhkarimi1383
Copy link
Member Author

mhkarimi1383 commented Apr 28, 2025

@abhilashshetty04
Squshed.

I had to recreate branch and apply diff of the PR, Because of some merge commits, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extra mkfs options
5 participants