-
Notifications
You must be signed in to change notification settings - Fork 2.2k
feat(host_metrics): expose number of oom_kill events
#25802
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| The `host_metrics` source now exposes an `oom_kill` counter metric on Linux, | ||
| reporting the number of Out-Of-Memory kill events recorded by the kernel. | ||
|
|
||
| authors: simonhammes |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,9 @@ use vector_lib::event::MetricTags; | |
| use super::HostMetrics; | ||
| use crate::internal_events::HostMetricsScrapeDetailError; | ||
|
|
||
| #[cfg(target_os = "linux")] | ||
| const OOM_KILL: &str = "oom_kill"; | ||
|
|
||
| impl HostMetrics { | ||
| pub async fn memory_metrics(&self, output: &mut super::MetricsBuffer) { | ||
| output.name = "memory"; | ||
|
|
@@ -122,4 +125,62 @@ impl HostMetrics { | |
| } | ||
| } | ||
| } | ||
|
|
||
| #[cfg(target_os = "linux")] | ||
| pub async fn vmstat_metrics(&self, output: &mut super::MetricsBuffer) { | ||
| output.name = "memory"; | ||
|
|
||
| // Spawn blocking task to avoid blocking the async runtime with synchronous I/O | ||
| let result = tokio::task::spawn_blocking(procfs::vmstat) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As far as I'm aware, In addition, the |
||
| .await | ||
| .unwrap_or_else(|join_error| { | ||
| Err(procfs::ProcError::Other(format!( | ||
| "Failed to join blocking task: {}", | ||
| join_error | ||
| ))) | ||
| }); | ||
|
|
||
| match result { | ||
| Ok(stats) => { | ||
| if let Some(&oom_kill) = stats.get("oom_kill") { | ||
| output.counter(OOM_KILL, oom_kill as f64, MetricTags::default()); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This emits a public counter named Useful? React with 👍 / 👎. |
||
| } | ||
| } | ||
| Err(error) => { | ||
| emit!(HostMetricsScrapeDetailError { | ||
| message: "Failed to load vmstat info.", | ||
| error, | ||
| }); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[cfg(all(test, target_os = "linux"))] | ||
| mod tests { | ||
| use crate::event::metric::MetricValue; | ||
| use crate::sources::host_metrics::{HostMetrics, HostMetricsConfig, MetricsBuffer}; | ||
|
|
||
| use super::OOM_KILL; | ||
|
|
||
| #[tokio::test] | ||
| async fn generates_vmstat_oom_kill_metric() { | ||
| let mut buffer = MetricsBuffer::new(None); | ||
| HostMetrics::new(HostMetricsConfig::default()) | ||
| .vmstat_metrics(&mut buffer) | ||
| .await; | ||
| let metrics = buffer.metrics; | ||
|
|
||
| assert_eq!(metrics.len(), 1); | ||
|
|
||
| let metric = &metrics[0]; | ||
| assert_eq!(metric.name(), OOM_KILL); | ||
| assert!( | ||
| matches!(metric.value(), MetricValue::Counter { .. }), | ||
| "oom_kill metric should be a counter" | ||
| ); | ||
|
|
||
| let tags = metric.tags().expect("metric must have tags"); | ||
| assert_eq!(tags.get("collector"), Some("memory")); | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was a bit unsure regarding the naming of this metric.
Should this be called
memory_oom_kill_events? Oroom_kill_events?