Inability to run valgrind if rustix touches anything involving vdso, like clock_gettime

[kchibisov]

When trying to profile https://github.com/YaLTeR/niri I get the following error due to valgrind not being able to understand vdso access, the only way to kind of fix it is to patch things to use libc::clock_gettime, but it sometimes present in dependencies and takes time to patch out, so it would be nice to somehow not being able to manually patch all the crates in the dependencies tree, since it's time consuming.

==267786== Warning: unimplemented fcntl command: 1027
==267786== Invalid read of size 4
==267786==    at 0x78401DA: spec_eq<u8, u8, 4> (equality.rs:157)
==267786==    by 0x78401DA: <u8 as core::array::equality::SpecArrayEq<u8, 4>>::spec_ne (equality.rs:160)
==267786==    by 0x78400D2: ne<u8, u8, 4> (equality.rs:15)
==267786==    by 0x78400D2: <[u8] as core::cmp::PartialEq<[u8; 4]>>::ne (equality.rs:57)
==267786==    by 0x783F2F8: rustix::backend::param::auxv::check_elf_base (auxv.rs:490)
==267786==    by 0x783CDA2: rustix::backend::param::auxv::init_from_aux_iter::<rustix::backend::param::auxv::AuxPointer> (auxv.rs:407)
==267786==    by 0x783F594: rustix::backend::param::auxv::init_auxv_impl (auxv.rs:301)
==267786==    by 0x783F645: rustix::backend::param::auxv::maybe_init_auxv (auxv.rs:276)
==267786==    by 0x783F21C: rustix::backend::param::auxv::sysinfo_ehdr (auxv.rs:150)
==267786==    by 0x783E095: rustix::backend::vdso::init_from_sysinfo_ehdr (vdso.rs:84)
==267786==    by 0x783D46C: <rustix::backend::vdso::Vdso>::new (vdso.rs:247)
==267786==    by 0x783A50D: rustix::backend::vdso_wrappers::init (vdso_wrappers.rs:499)
==267786==    by 0x783A3B5: rustix::backend::vdso_wrappers::init_clock_gettime (vdso_wrappers.rs:294)
==267786==    by 0x533F3AE: rustix::backend::vdso_wrappers::clock_gettime (vdso_wrappers.rs:70)
==267786==  Address 0x7ff21b666000 is not stack'd, malloc'd or (recently) free'd
==267786==
==267786==
==267786== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==267786==  Access not within mapped region at address 0x7FF21B666000
==267786==    at 0x78401DA: spec_eq<u8, u8, 4> (equality.rs:157)
==267786==    by 0x78401DA: <u8 as core::array::equality::SpecArrayEq<u8, 4>>::spec_ne (equality.rs:160)
==267786==    by 0x78400D2: ne<u8, u8, 4> (equality.rs:15)
==267786==    by 0x78400D2: <[u8] as core::cmp::PartialEq<[u8; 4]>>::ne (equality.rs:57)
==267786==    by 0x783F2F8: rustix::backend::param::auxv::check_elf_base (auxv.rs:490)
==267786==    by 0x783CDA2: rustix::backend::param::auxv::init_from_aux_iter::<rustix::backend::param::auxv::AuxPointer> (auxv.rs:407)
==267786==    by 0x783F594: rustix::backend::param::auxv::init_auxv_impl (auxv.rs:301)
==267786==    by 0x783F645: rustix::backend::param::auxv::maybe_init_auxv (auxv.rs:276)
==267786==    by 0x783F21C: rustix::backend::param::auxv::sysinfo_ehdr (auxv.rs:150)
==267786==    by 0x783E095: rustix::backend::vdso::init_from_sysinfo_ehdr (vdso.rs:84)
==267786==    by 0x783D46C: <rustix::backend::vdso::Vdso>::new (vdso.rs:247)
==267786==    by 0x783A50D: rustix::backend::vdso_wrappers::init (vdso_wrappers.rs:499)
==267786==    by 0x783A3B5: rustix::backend::vdso_wrappers::init_clock_gettime (vdso_wrappers.rs:294)
==267786==    by 0x533F3AE: rustix::backend::vdso_wrappers::clock_gettime (vdso_wrappers.rs:70)

[notgull]

Does valgrind work if you build with RUSTFLAGS="--cfg=rustix_use_libc"? It could just be an issue with how the linux_raw backend communicates with vDSO, or Valgrind trying to instrument calls we don't make.

[kchibisov]

Yes with RUSTFLAGS="--cfg=rustix_use_libc it does work.

[sunfishcode]

There are two ways to use the libc backend; that RUSTFLAGS way is one, and another is by adding the "use_libc" feature to the rustix depenency in Cargo.toml.

[notgull]

@sunfishcode I think the problem here is that Valgrind doesn't work with the linux_raw backend. I'm not familiar enough with the underlying mechanics of Valgrind to know why.

[sunfishcode]

The valgrind log in the original report here points to the code where rustix is accessing the header of the ELF image pointed to by AT_SYSINFO_EHDR. That led me to this code in valgrind:

https://sourceware.org/git/?p=valgrind.git;a=blob;f=coregrind/m_initimg/initimg-linux.c;h=5477075555f14a3abb6329a98d4d1ba01755427e;hb=HEAD#l915

Since valgrind seems to be intentionally withholding this, perhaps the solution is to have rustix detect that it's running in valgrind somehow and disable the vDSO code in that case.

Or re-open #1494.

[kchibisov]

Could be brought to valgrind instead?

[kchibisov]

I just brought here, because it looked like no one run rustix under valgrind with vdso before, so I mostly wanted to check who is at fault.

Generally adjusting to profiling tools is not ideal, because then you just hide execution path from them and instead profiling tools should get better at handling regular path, in my opinion.

Given that there's a way to force libc and users(like me) in case of wanting to run under valgrind.

Maybe rustix should just document that running under certain profilers may require use_libc in its docs(the one that end up rendering on docs.rs), etc, since I tried to search in docs, but it wasn't mentioned that I can switch to libc all together.

[notgull]

Agreed with @kchibisov, my concern was "we're doing something unexpected to cause Valgrind to bail out", but if it's Valgrind that's doing something unexpected, then we're not doing anything wrong.