-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Issue Summary
The uplift plots currently saved to the plots folder show only the preliminary 'forward' test-reference analysis results and do not reflect the final reported uplift values.
Current Behaviour
Uplift plots are generated from the initial 'forward' test-reference analysis
These plots do not incorporate the full analysis pipeline including:
- Bootstrapping calculations
- Test-reference reversal analysis
The plots therefore show different (higher) uplift values than the final reported results
Expected Behaviour
The uplift plots in the plots folder should be based on the final uplift results that have been processed through the complete analysis pipeline, including bootstrapping and test-reference reversal calculations.
Additional Context
Final uplift plots will typically show the same or lower uplift values compared to the current 'forward'-only plots due to the uncertainty methodology
This change will ensure consistency between the visual outputs and the reported final results.
A couple of example uplift plots that are currently based on the 'forward' analysis only
