Intermittent CI failures in `cache::list_files_cache::tests::test_cache_with_ttl_and_lru` ( seems flaky)

[alamb]

Describe the bug

@adriangb reports on discord:

cache::list_files_cache::tests::test_cache_with_ttl_and_lru is intermittently failing on CI

To Reproduce

Seems like it's failing on main: https://github.com/apache/datafusion/actions/runs/19962057560/job/57244600157#step:4:2618

failures:

---- cache::list_files_cache::tests::test_cache_with_ttl_and_lru stdout ----

thread 'cache::list_files_cache::tests::test_cache_with_ttl_and_lru' (146096) panicked at datafusion/execution/src/cache/list_files_cache.rs:618:9:
assertion failed: cache.contains_key(&path3)
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/f8297e351a40c1439a467bbbb6879088047f50b3/library/std/src/panicking.rs:698:5
   1: core::panicking::panic_fmt
             at /rustc/f8297e351a40c1439a467bbbb6879088047f50b3/library/core/src/panicking.rs:75:14
   2: core::panicking::panic
             at /rustc/f8297e351a40c1439a467bbbb6879088047f50b3/library/core/src/panicking.rs:145:5
   3: test_cache_with_ttl_and_lru
             at ./src/cache/list_files_cache.rs:618:9
   4: {closure#0}
             at ./src/cache/list_files_cache.rs:596:37
   5: call_once<datafusion_execution::cache::list_files_cache::tests::test_cache_with_ttl_and_lru::{closure_env#0}, ()>
             at /rustc/f8297e351a40c1439a467bbbb6879088047f50b3/library/core/src/ops/function.rs:250:5
   6: core::ops::function::FnOnce::call_once
             at /rustc/f8297e351a40c1439a467bbbb6879088047f50b3/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Also failed on a PR with unrelated changes
https://github.com/apache/datafusion/actions/runs/19966253699/job/57258712946?pr=19094#step:4:2618

Expected behavior

test should always pass (or fail) not intermittently pass

Additional context

The test is here
https://github.com/apache/datafusion/blame/a5fc3c704b261f3a3ccf35089b93d8424901ba96/datafusion/execution/src/cache/list_files_cache.rs#L596

It was added in d150af7 / #18855 from @BlakeOrth

I suspect the runner is so slow that the ttl expires prematurely

[alamb]

Suggested course of action:

1. Disable the test temporarily to unblock CI on main
2. Fix the test (probably by mocking out where the current time comes from, rather than using the real system clock)

[alamb]

Suggestion for implementation: #18855 (comment)

One classic way is to pass a template / object to the cache to provide the current time, rather than calling std::instant::now() directly

Then the tests can use a mock version of the time provider (that is explicitly controlled) and the normal code uses std::Instant::now

The downside of this approach is that it does mean what is tested has some differences than what is in production

[BlakeOrth]

I will plan on allocating time to address this early next week unless someone beats me to it over the weekend.