Random build failure due to `std::time::Instant::duration_since`

[pmnoxx]

@matklad Can you take a look?
It looks like another issue with Instant::duration_since. It's becoming as popular as null pointer exception.

Compiling wasmtime v0.25.0
[2021-11-16T23:35:02Z] thread 'main' panicked at 'supplied instant is later than self', library/std/src/time.rs:292:48
--
  | [2021-11-16T23:35:02Z] stack backtrace:
  | [2021-11-16T23:35:02Z]    0: rust_begin_unwind
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:517:5
  | [2021-11-16T23:35:02Z]    1: core::panicking::panic_fmt
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/panicking.rs:101:14
  | [2021-11-16T23:35:02Z]    2: core::option::expect_failed
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/option.rs:1615:5
  | [2021-11-16T23:35:02Z]    3: core::option::Option<T>::expect
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/option.rs:698:21
  | [2021-11-16T23:35:02Z]    4: std::time::Instant::duration_since
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/time.rs:292:9
  | [2021-11-16T23:35:02Z]    5: <std::time::Instant as core::ops::arith::Sub>::sub
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/time.rs:417:9
  | [2021-11-16T23:35:02Z]    6: std::time::Instant::elapsed
  | [2021-11-16T23:35:02Z]              at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/time.rs:356:9
  | [2021-11-16T23:35:02Z]    7: std::sync::condvar::Condvar::wait_timeout_while
  | [2021-11-16T23:35:02Z]    8: cargo::util::queue::Queue<T>::pop
  | [2021-11-16T23:35:02Z]    9: cargo::core::compiler::job_queue::DrainState::drain_the_queue
  | [2021-11-16T23:35:02Z]   10: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  | [2021-11-16T23:35:02Z]   11: crossbeam_utils::thread::scope
  | [2021-11-16T23:35:02Z]   12: cargo::core::compiler::job_queue::JobQueue::execute
  | [2021-11-16T23:35:02Z]   13: cargo::core::compiler::context::Context::compile
  | [2021-11-16T23:35:02Z]   14: cargo::ops::cargo_compile::compile_ws
  | [2021-11-16T23:35:02Z]   15: cargo::ops::cargo_compile::compile
  | [2021-11-16T23:35:02Z]   16: cargo::ops::cargo_test::run_tests
  | [2021-11-16T23:35:02Z]   17: cargo::commands::test::exec
  | [2021-11-16T23:35:02Z]   18: cargo::cli::main
  | [2021-11-16T23:35:02Z]   19: cargo::main
  | [2021-11-16T23:35:02Z] note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
  | [2021-11-16T23:35:02Z] thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: PoisonError { .. }', src/tools/cargo/src/cargo/util/queue.rs:37:27
  | [2021-11-16T23:35:02Z] stack backtrace:

https://buildkite.com/nearprotocol/nearcore/builds/7208#18658c9e-78f3-4a17-8a97-b22cbdacdf14

[matklad]

Oh wow, so this actually panics in Cargo during the build? I guess on our side the best person to look into this is @chefsale -- this error means that the machine we run the build on is buggy. That is, this is usually a bug in OS (which promises to return a monotonic time, but fails), or in hypervisor (which emulates a clock device which promises monotonic time, and then fails).

[matklad]

I guess it would be useful (mainly for folks at rust-lang/rust) to know various low-level things here like kernel version, CPU version, hypervisor version.

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[matklad]

Will be fixed once we upgrade to the version of Rust which includes rust-lang/rust#89926 (currently a nightly)