-
Notifications
You must be signed in to change notification settings - Fork 473
feat: implement OCI per-mount propagation options #3699
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
|
|
@@ -21,6 +21,10 @@ pub struct MountOptionConfig { | |||
|
|
||||
| /// RecAttr represents mount properties to be applied recursively. | ||||
| pub rec_attr: Option<linux::MountAttr>, | ||||
|
|
||||
| /// Mount propagation flags, kept separate from regular mount flags because | ||||
| /// they are applied after the mount is attached. | ||||
| pub propagation_flags: Vec<MsFlags>, | ||||
| } | ||||
|
|
||||
| pub fn default_devices() -> Vec<LinuxDevice> { | ||||
|
|
@@ -87,6 +91,7 @@ pub fn to_sflag(dev_type: LinuxDeviceType) -> SFlag { | |||
|
|
||||
| pub fn parse_mount(m: &Mount) -> std::result::Result<MountOptionConfig, MountError> { | ||||
| let mut flags = MsFlags::empty(); | ||||
| let mut propagation_flags = Vec::new(); | ||||
| let mut data = Vec::new(); | ||||
| let mut mount_attr: Option<linux::MountAttr> = None; | ||||
|
|
||||
|
|
@@ -167,14 +172,17 @@ pub fn parse_mount(m: &Mount) -> std::result::Result<MountOptionConfig, MountErr | |||
| MountOption::Nodiratime(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Bind(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Rbind(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Unbindable(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Runbindable(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Private(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Rprivate(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Shared(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Rshared(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Slave(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Rslave(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Unbindable(_, flag) | ||||
| | MountOption::Runbindable(_, flag) | ||||
| | MountOption::Private(_, flag) | ||||
| | MountOption::Rprivate(_, flag) | ||||
| | MountOption::Shared(_, flag) | ||||
| | MountOption::Rshared(_, flag) | ||||
| | MountOption::Slave(_, flag) | ||||
| | MountOption::Rslave(_, flag) => { | ||||
| propagation_flags.push(flag); | ||||
| continue; | ||||
| } | ||||
| MountOption::Relatime(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Norelatime(is_clear, flag) => Some((is_clear, flag)), | ||||
| MountOption::Strictatime(is_clear, flag) => Some((is_clear, flag)), | ||||
|
|
@@ -197,6 +205,7 @@ pub fn parse_mount(m: &Mount) -> std::result::Result<MountOptionConfig, MountErr | |||
| flags, | ||||
| data: data.into_iter().map(|s| s.to_string()).collect(), | ||||
| rec_attr: mount_attr, | ||||
| propagation_flags, | ||||
| }) | ||||
| } | ||||
|
|
||||
|
|
@@ -245,6 +254,7 @@ mod tests { | |||
| flags: MsFlags::empty(), | ||||
| data: vec![], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -267,6 +277,7 @@ mod tests { | |||
| flags: MsFlags::MS_NOSUID | MsFlags::MS_STRICTATIME, | ||||
| data: vec!["mode=755".to_string(), "size=65536k".to_string()], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -297,6 +308,7 @@ mod tests { | |||
| "gid=5".to_string() | ||||
| ], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -320,6 +332,7 @@ mod tests { | |||
| flags: MsFlags::MS_NOSUID | MsFlags::MS_NOEXEC | MsFlags::MS_NODEV, | ||||
| data: vec!["mode=1777".to_string(), "size=65536k".to_string()], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -342,6 +355,7 @@ mod tests { | |||
| flags: MsFlags::MS_NOSUID | MsFlags::MS_NOEXEC | MsFlags::MS_NODEV, | ||||
| data: vec![], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -367,6 +381,7 @@ mod tests { | |||
| | MsFlags::MS_RDONLY, | ||||
| data: vec![], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -394,6 +409,7 @@ mod tests { | |||
| | MsFlags::MS_RELATIME, | ||||
| data: vec![], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config, | ||||
| ); | ||||
|
|
@@ -448,9 +464,19 @@ mod tests { | |||
| | MsFlags::MS_NOATIME | ||||
| | MsFlags::MS_NODIRATIME | ||||
| | MsFlags::MS_BIND | ||||
| | MsFlags::MS_UNBINDABLE, | ||||
| | MsFlags::MS_REC, | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does changing this mean that the existing behavior will also change? Is this change acceptable?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
youki/crates/libcontainer/src/rootfs/mount.rs Line 640 in a46e6a1
The https://github.com/youki-dev/youki/tree/main/tests/contest/contest/src/tests/mounts_recursive |
||||
| data: vec![], | ||||
| rec_attr: None, | ||||
| propagation_flags: vec![ | ||||
| MsFlags::MS_UNBINDABLE, | ||||
| MsFlags::MS_UNBINDABLE | MsFlags::MS_REC, | ||||
| MsFlags::MS_PRIVATE, | ||||
| MsFlags::MS_PRIVATE | MsFlags::MS_REC, | ||||
| MsFlags::MS_SHARED, | ||||
| MsFlags::MS_SHARED | MsFlags::MS_REC, | ||||
| MsFlags::MS_SLAVE, | ||||
| MsFlags::MS_SLAVE | MsFlags::MS_REC, | ||||
| ], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
@@ -485,6 +511,7 @@ mod tests { | |||
| flags: MsFlags::empty(), | ||||
| data: vec![], | ||||
| rec_attr: Some(MountAttr::all()), | ||||
| propagation_flags: vec![], | ||||
| }, | ||||
| mount_option_config | ||||
| ); | ||||
|
|
||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| mod mount_propagation_test; | ||
| pub use mount_propagation_test::get_mount_propagation_test; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| use std::fs::{create_dir, remove_dir_all}; | ||
| use std::path::{Path, PathBuf}; | ||
| use std::str::FromStr; | ||
|
|
||
| use anyhow::{Context, Ok, Result, anyhow}; | ||
| use nix::mount::{MsFlags, mount, umount}; | ||
| use oci_spec::runtime::{Mount, ProcessBuilder, Spec, SpecBuilder, get_default_mounts}; | ||
| use tempfile::TempDir; | ||
| use test_framework::{Test, TestGroup, TestResult, test_result}; | ||
|
|
||
| use crate::utils::test_inside_container; | ||
| use crate::utils::test_utils::CreateOptions; | ||
|
|
||
| const MOUNT_DEST: &str = "/mnt/mount_propagation"; | ||
| const SUB_DIR_NAME: &str = "sub"; | ||
|
|
||
| fn create_spec(source: &Path, propagation: &str) -> Result<Spec> { | ||
| let mut mounts = get_default_mounts(); | ||
|
|
||
| let mut mount_spec = Mount::default(); | ||
| mount_spec | ||
| .set_destination(PathBuf::from_str(MOUNT_DEST).unwrap()) | ||
| .set_typ(None) | ||
| .set_source(Some(source.to_path_buf())) | ||
| .set_options(Some(vec!["rbind".to_string(), propagation.to_string()])); | ||
| mounts.push(mount_spec); | ||
|
|
||
| let process = ProcessBuilder::default() | ||
| .args(vec![ | ||
| "runtimetest".to_string(), | ||
| "mount_propagation".to_string(), | ||
| ]) | ||
| .build() | ||
| .context("failed to build process")?; | ||
|
|
||
| let spec = SpecBuilder::default() | ||
| .mounts(mounts) | ||
| .process(process) | ||
| .build() | ||
| .context("failed to build spec")?; | ||
|
|
||
| Ok(spec) | ||
| } | ||
|
|
||
| // Make mount_dir shared and add a submount so propagation changes are | ||
| // observable, including the difference between recursive and non-recursive options. | ||
| fn setup_mount(mount_dir: &Path, sub_mount_dir: &Path) -> Result<()> { | ||
| create_dir(mount_dir)?; | ||
| mount::<Path, Path, str, str>(Some(mount_dir), mount_dir, None, MsFlags::MS_BIND, None)?; | ||
|
|
||
| mount::<Path, Path, str, str>(None, mount_dir, None, MsFlags::MS_SHARED, None)?; | ||
| create_dir(sub_mount_dir)?; | ||
| mount::<Path, Path, str, str>(None, sub_mount_dir, Some("tmpfs"), MsFlags::empty(), None)?; | ||
| Ok(()) | ||
| } | ||
|
|
||
| fn clean_mount(mount_dir: &Path, sub_mount_dir: &Path) -> Result<()> { | ||
| umount(sub_mount_dir)?; | ||
| umount(mount_dir)?; | ||
| remove_dir_all(mount_dir)?; | ||
| Ok(()) | ||
| } | ||
|
|
||
| fn check_propagation(propagation: &str) -> TestResult { | ||
| let base_dir = TempDir::new().unwrap(); | ||
| let dir_path = base_dir.path().join("mount_dir"); | ||
| let sub_path = dir_path.join(SUB_DIR_NAME); | ||
|
|
||
| let spec = test_result!(create_spec(&dir_path, propagation)); | ||
|
|
||
| let result = test_inside_container(&spec, &CreateOptions::default(), &|_rootfs| { | ||
| setup_mount(&dir_path, &sub_path).map_err(|e| anyhow!("setup_mount failed: {e:?}"))?; | ||
| Ok(()) | ||
| }); | ||
|
|
||
| if let Err(e) = clean_mount(&dir_path, &sub_path) { | ||
| eprintln!( | ||
| "clean_mount failed (mount_dir={}, sub_dir={}): {e:?}", | ||
| dir_path.display(), | ||
| sub_path.display(), | ||
| ); | ||
| } | ||
|
|
||
| result | ||
| } | ||
|
|
||
| fn shared_test() -> TestResult { | ||
| check_propagation("shared") | ||
| } | ||
|
|
||
| fn rshared_test() -> TestResult { | ||
| check_propagation("rshared") | ||
| } | ||
|
|
||
| fn slave_test() -> TestResult { | ||
| check_propagation("slave") | ||
| } | ||
|
|
||
| fn rslave_test() -> TestResult { | ||
| check_propagation("rslave") | ||
| } | ||
|
|
||
| fn private_test() -> TestResult { | ||
| check_propagation("private") | ||
| } | ||
|
|
||
| fn rprivate_test() -> TestResult { | ||
| check_propagation("rprivate") | ||
| } | ||
|
|
||
| fn unbindable_test() -> TestResult { | ||
| check_propagation("unbindable") | ||
| } | ||
|
|
||
| fn runbindable_test() -> TestResult { | ||
| check_propagation("runbindable") | ||
| } | ||
|
|
||
| pub fn get_mount_propagation_test() -> TestGroup { | ||
| let mut test_group = TestGroup::new("mount_propagation"); | ||
|
|
||
| let shared_test = Test::new("mount_propagation_shared_test", Box::new(shared_test)); | ||
| let rshared_test = Test::new("mount_propagation_rshared_test", Box::new(rshared_test)); | ||
| let slave_test = Test::new("mount_propagation_slave_test", Box::new(slave_test)); | ||
| let rslave_test = Test::new("mount_propagation_rslave_test", Box::new(rslave_test)); | ||
| let private_test = Test::new("mount_propagation_private_test", Box::new(private_test)); | ||
| let rprivate_test = Test::new("mount_propagation_rprivate_test", Box::new(rprivate_test)); | ||
| let unbindable_test = Test::new( | ||
| "mount_propagation_unbindable_test", | ||
| Box::new(unbindable_test), | ||
| ); | ||
| let runbindable_test = Test::new( | ||
| "mount_propagation_runbindable_test", | ||
| Box::new(runbindable_test), | ||
| ); | ||
|
|
||
| test_group.add(vec![ | ||
| Box::new(shared_test), | ||
| Box::new(rshared_test), | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please add the test case for slave.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added. |
||
| Box::new(slave_test), | ||
| Box::new(rslave_test), | ||
| Box::new(private_test), | ||
| Box::new(rprivate_test), | ||
| Box::new(unbindable_test), | ||
| Box::new(runbindable_test), | ||
| ]); | ||
|
|
||
| test_group | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apply_mount_propagationusesmount_setattr(2)for propagation flags (MS_SHARED, MS_SLAVE, etc.). However,mount_setattr(2)was added in Linux 5.12 (https://man7.org/linux/man-pages/man2/mount_setattr.2.html), and is not available on kernels still widely in use — notably Ubuntu 20.04 LTS (5.4), Debian 11 (5.10), Amazon Linux 2 (5.10), and RHEL 8 (4.18).Neither runc nor crun uses
mount_setattr(2)for propagation:runc uses
mount_setattr(2)exclusively forRecAttr(setRecAttr, rootfs_linux.go:1475) — attributes that cannot be applied recursively and atomically viamount(2). All other mount attributes (MS_RDONLY, MS_NOSUID, etc.) are set viamount(2)flags. Propagation flags are also applied viamount(2)inmountPropagate()(rootfs_linux.go:1458):crun similarly retains a
mount(2)path for propagation insidedo_mount()(linux.c:1317-1322):Note that youki already uses
mount_setattr(2)in other parts of the mount flow. For mount attribute setting (L671, L760), youki usesmount_setattras part of its new mount API (open_tree → mount_setattr → move_mount), which has no equivalent in runc (runc usesmount(2)flags for attributes). For RecAttr (L681, L770), theAT_RECURSIVEusage is consistent with runc'ssetRecAttr. The issue is specifically that PR #3699 extendsmount_setattr(2)to propagation flags, whereas both runc and crun usemount(2)for that purpose.Since
MS_RECcan be passed directly tomount(2), usingmount(NULL, dest, NULL, propagation_flag, NULL)would be sufficient and avoids the Linux 5.12 requirement for this step.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of avoiding the Linux 5.12 requirement, doesn’t that have little practical meaning for youki as a whole, since
mount_setattris already used in other parts of the runtime?If the only reason is simply to align the implementation with runc and crun, I’m not particularly convinced that this is worth doing.
If we switch this back to mount(2), how do we preserve the fd-based path resolution guarantees we get from mount_setattr? As far as I can tell, youki does not currently have a helper equivalent to runc's handling for safely applying a legacy mount operation to an already-resolved fd-backed target.
Even if we decide to implement this using
mount, I think it should be treated as a fallback path and addressed in a separate issue.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this is a good point. Sorry, I realize that my comment may not have made my intention clear.
My concern was not simply to align with
runcorcrun. I think we should separately discuss which kernel versionsyoukiitself should support. Sinceyoukialready usesmount_setattr(2)in other parts of the runtime, switching this path tomount(2)would not meaningfully change the minimum kernel version foryoukias a whole.For now, I think it is reasonable to support kernel 5.12 and later. My main goal was to make sure we were aligned on the support policy.
For this PR, I’m fine with keeping
mount_setattr(2)and considering the legacymount(2)fallback, including safe fd-based path resolution, out of scope. We can discuss that separately in another issue.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I created an issue here.
#3726