You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: index.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,19 +21,23 @@ An **LSTM (Long Short-Term Memory)** is a **recurrent neural network (RNN)** arc
21
21
In a **reconstruction-based anomaly detection** setup, an LSTM Autoencoder is trained to reconstruct normal behaviour. When model is tested, windows that deviate from learned dynamics give higher **reconstruction errors** and are flagged as anomalies if the errors exceed the set threshold. The training methods studied are:
22
22
-**Standard MSE loss**: model **minimizes the pointwise reconstruction error between the input and it's reconstruction**. Though being simple, it's generally stable and widely applicable. However, it presents limitations in understanding the dynamics of the system, so every reconstruction minimizing MSE is considered acceptable, even if not physically normal.
23
23
-**Physics-informed loss**: add MSE loss term to a residual penalty that embeds the damped oscillator equation $x'' + 2\zeta\omega_0 x' + \omega_0^2 x = 0$. To achieve this, I approximate $x'$ and $x''$ with **finite differences** on the reconstructed sequence and penalize the squared residual, thus obtaining the total loss:
24
+
24
25
$$
25
26
L = \mathrm{MSE}(x, \hat{x}) + \lambda_{\mathrm{phys}} \cdot \left\| x'' + 2\zeta\omega_0 x' + \omega_0^2 x \right\|^2
26
27
$$
28
+
27
29
where $\lambda_{\mathrm{phys}}$ controls how much the physics loss contributes to total loss estimation, thus enforcing the autoencoder to understand reconstruction that are both close to the input but also physically consistent with system's dynamics, **improving sensitivity** to physically implausable deviations.
28
30
29
31
Comparing these objectives on the same dataset, I can assess wether pure MSE loss or embedded physical knowledge into the training can improve anomaly detection without lossing performance.
30
32
31
33
## Dataset
32
34
I simulate the **dynamics of a 1-dimensional noisy damped harmonic oscillator**. Dynamics of the system are governed by the equation:
where $\zeta$ is the damping ratio, $\omega_0$ the natural angular frequency and $\varepsilon(t)$ the Gaussian noise. I generate $T = 60,000$ samples spanning $t = 0$ to $t = 599.99$ with $dt = 0.01$ step. I then inject four types of anomalies:
38
42
-**Spikes**: isolated points with abnormal impulses $x_i + a \rightarrow x_i$, with $a \in [2.5, 5.5]$
39
43
-**Level shifts**: 400-width segments are shifted by constant offset
@@ -44,9 +48,11 @@ The feature analyzed is the displacement $x_t$. To preprocess the data I create
44
48
- length $W = 128$ and stride $s = 32$
45
49
- for each start index $k$, I extract $X_k = (x_k, x_{k+1},...,x_{k+W-1}) \in \mathbb{R}^{128 \times 1}$
46
50
- window label $y_k$ is set to 1 if at least 5% of it's points are anomalous
otherwise $y_k = 0$. After windowing process, I get 1,872 windows with approximately anomaly rate ≈ 31.7%.
51
57
52
58
I split normal windows in:
@@ -59,6 +65,7 @@ I split anomaly windows in:
59
65
- 50% validation anomalies
60
66
61
67
Finally, before model training, (normal) data is **scaled** using ```StandardScaler``` on the flattened windows and **normalized** so that training distribution has zero mean and unit variance
0 commit comments