feat: add summary generation and export functionality for results

[JessUWE]

This PR introduces a session-level summary of outputs to help output checkers get an overview of all results generated during a session.

• Implemented generate_summary() method to create session output summary DataFrame
• Added write_summary() method to export summary to CSV file
• Enhanced finalise_excel() to include a 'summary' sheet in Excel results
• Integrated summary generation into finalise() workflow

[JessUWE]

Needs further updating

[codecov]

Codecov Report

❌ Patch coverage is 82.40000% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.96%. Comparing base (fe1f204) to head (9002c7f).

Files with missing lines	Patch %	Lines
acro/record.py	78.57%	30 Missing ⚠️
acro/acro_regression.py	80.28%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #341      +/-   ##
==========================================
- Coverage   99.70%   96.96%   -2.74%     
==========================================
  Files           9        9              
  Lines        1354     1584     +230     
==========================================
+ Hits         1350     1536     +186     
- Misses          4       48      +44     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jim-smith]

Hi @JessUWE three things:

1: Just run this on a same demo which has the same tables with and without suppression. Should there fore be reporting diff risk as both have identical variables, but that is showing as False.

• will look into the logic there.

2: think we re nearly ready to go out to the community saying this is what we can produce, is this helpful / how should we tweak it?

3: Based on the original meetings with PHS, I think we ma y want to add another table which has one column for each variable (and an extra for the output type) and one row for each output. Entryies big 'output type are 'table', 'regression', 'plot' etc pulled from summary) then the rest of the table is a binary -'is the variable (column) in the list of variables for that output.

[JessUWE]

@jim-smith

[jim-smith]

@JessUWE @rpreen hope to have time to look at this tomorrow.
Would like to be able to make screenshot and put it on uk-tre and sec-reboot email lists asking for input/feedback from output checkers about how we present this info and whether it is problematic if this got accidentally released.

[jim-smith]

is this deleted test not needed any more?

[jim-smith]

Some changes needed to what variables are extracted for regression analyses.

Don;t understand what the diff is telling me about insertions and deletions within the test file. think those are easy yes/no answers

[jim-smith]

one option to consider is to extract the variables involved at the time that the query is first dealt with e.g.

for a crosstab: called as

acro.crosstab((df.rowvar1,df.rowvar2), 
                               (df.colvar1,df.colvar2),
                                values=valvar,
                                 aggfunc="mean")

inside our acro.crosstab method we do:

....
vars_used = []
#get names of variables used in rows
if isinstance (index, pd.series):
     vars_used .append(index.name)
else: #index must be a list of pandas series to define row hierarchy)
     for var in index: 
             vars_used .append(var.name)

#get names of variables used in columns
if isinstance (columns, pd.series):
     vars_used .append(columns)
else: #columns must be a list of pandas series to define columns hierarchy)
     for var in columns: 
             vars_used .append(var.name)
#get name of series reported on by aggregation function if present
if values is not None:
    vars_used.append(values.name)

# now save your array of values in the json

[jim-smith]

for regression commands specified with the trailing 'r' , the variables are all held in the lists endog and exog,

For regression commands specified with a formula, it has the form of a string and can be parsed

cheatsheet to help with that parsing

[JessUWE]

Closing this PR for now as this approach isn't giving us the desired result instead of leaving the PR longer. I'll revisit this after doing some further research and testing. Thanks for the reviews so far, I'll incorporate this feedback when I come back to it