Add sigma_cauchy and scp_norm to solver statistics in all solvers

[MohamedLaghdafHABIBOULLAH]

Fix this 252

[Copilot]

Pull Request Overview

This PR adds tracking of sigma_cauchy and scp_norm statistics to solver output across all trust-region and regularization-based solvers in the package. These statistics provide important diagnostic information about the solver's behavior at each iteration.

Key changes:

• Addition of sigma_cauchy (reciprocal of step size parameter) tracking
• Addition of scp_norm (norm of the proximal gradient step) tracking
• Consistent placement of these statistics after they are computed in each solver

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/TR_alg.jl	Added sigma_cauchy and scp_norm statistics tracking in the TR solver at initialization and main iteration loop
src/TRDH_alg.jl	Added sigma_cauchy and scp_norm statistics tracking in the TRDH solver for both reduce_TR branches
src/R2_alg.jl	Added scp_norm statistics tracking in the R2 solver at initialization and main iteration loop
src/R2N.jl	Added scp_norm statistics tracking in the R2N solver at initialization and main iteration loop
src/R2DH.jl	Added scp_norm statistics tracking and moved sigma_cauchy to after ν₁ is computed in the R2DH solver
src/LM_alg.jl	Added sigma_cauchy and scp_norm statistics tracking in the Levenberg-Marquardt solver at initialization and main iteration loop
src/LMTR_alg.jl	Added sigma_cauchy and scp_norm statistics tracking in the LMTR solver at initialization and main iteration loop

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[codecov]

Codecov Report

❌ Patch coverage is 89.18919% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.75%. Comparing base (e0f214d) to head (b8f941b).
⚠️ Report is 259 commits behind head on master.

Files with missing lines	Patch %	Lines
src/TRDH_alg.jl	33.33%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #253       +/-   ##
===========================================
+ Coverage   61.53%   84.75%   +23.22%     
===========================================
  Files          11       13        +2     
  Lines        1292     1660      +368     
===========================================
+ Hits          795     1407      +612     
+ Misses        497      253      -244     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dpo]

Would it also be useful to report $\Vert s_k \Vert$ in addition to $\Vert s_{k,cp} \Vert$? PANOC doesn't compute an $s_{k,cp}$, and it could be that $\Vert s_k \Vert < \Vert s_{k,cp} \Vert$.

[MohamedLaghdafHABIBOULLAH]

Would it also be useful to report ‖ s k ‖ in addition to ‖ s k , c p ‖ ? PANOC doesn't compute an s k , c p , and it could be that ‖ s k ‖ < ‖ s k , c p ‖ .

I am not sure if the use of ‖ s k ‖ would be suitable for a stationary measure as in R2N we do not necessarily have ‖ s k ‖ < ‖ s k , c p ‖!

[MaxenceGollier]

Excellent,

I wonder if scp_norm is a good name though. Since we have a sigma_cauchy, wouldn't it be more appropriate to rename scp_norm as step_cauchy_norm or cauchy_point_norm ? Just an idea we might as well just keep scp_norm.

[MohamedLaghdafHABIBOULLAH]

@MaxenceGollier I think the name is correct, what do you think @dpo?

[MaxenceGollier]

Maybe add the .gitignore in a separate PR ?

The rest looks good to me.

[MaxenceGollier]

Can you make the corrections please ? @MohamedLaghdafHABIBOULLAH, we are waiting for this to be merged before releasing the package :)

[MohamedLaghdafHABIBOULLAH]

@MaxenceGollier @dpo I think we can keep this as a draft for now and revisit it after the release. The goal here is to store the norm of $$s$$ intelligently, but $$s$$ itself changes because $$s_{cp}$$ is overwritten by $$s$$. This is not the case for R2DH and R2, where $$s_{cp}$$ is already considered as $$s$$, and for TRDH it is more delicate since it depends on the reduce_tr argument.

[MohamedLaghdafHABIBOULLAH]

I think this PR is ready to be merged, @MaxenceGollier @dpo.
I avoided computing the norm of scp twice when it can occur, which is the case for R2DH, R2, and LM.

[dpo]

@MohamedLaghdafHABIBOULLAH I agree with @MaxenceGollier that step_norm and cauchy_step_norm would be more explicit names. I suggest that all solvers have both attributes. For R2 and R2DH, the values will simply be equal.

[dpo]

Would it also be useful to report ‖ s k ‖ in addition to ‖ s k , c p ‖ ? PANOC doesn't compute an s k , c p , and it could be that ‖ s k ‖ < ‖ s k , c p ‖ .

I am not sure if the use of ‖ s k ‖ would be suitable for a stationary measure as in R2N we do not necessarily have ‖ s k ‖ < ‖ s k , c p ‖!

We have a theorem that says that if $s_k = 0$, then $x_k$ is stationary.

[dpo]

@MohamedLaghdafHABIBOULLAH

[MaxenceGollier]

Could you make the update @MohamedLaghdafHABIBOULLAH ? I am trying to reduce the number of issues in this package.