Round robin validation tests #94
Conversation
Introduces a new OpticalProperties member for the back surface of Heliostat and methods to set mirror and back optics independently. Updates usage in tests and simulation logic to use set_mirror_optics instead of set_optics, and ensures back optics are applied correctly in element construction.
initially adding, test is passing with low tolerances. Will revisit.
Updated the z-rotation calculation in heliostat.cpp. Added a comprehensive unit test for a single facet flat heliostat oriented north of receiver, verifying geometry, ray tracing, and power absorption with sunshape and surface errors.
Replaced manual azimuth calculation in heliostat.cpp with atan2 for improved accuracy. Updated tracing_errors.cpp to consistently convert errors from mrad to rad and removed redundant conversions. Moved the SingleFacetFlat_North test from heliostat_test.cpp to a new single_heliostat_test.cpp file, expanding it with additional focused and southeast position tests. Updated CMakeLists.txt to include the new test file. Added a flux binning method for the test environment. This should be standardized.
Added a bounds check before accessing key.at(index) to prevent out-of-range exceptions. The method now returns 'this' if the index is out of bounds.
cmake copies them to the binary build directory.
Introduces heliostat_field_test.cpp to test simulation of a heliostat field, including ray tracing, receiver flux mapping, and validation of power and flux results. Most tests are commented out currently. Updates CMakeLists.txt to include the new test.
Expanded the heliostat field simulation test suite to cover multiple configurations, including slant-focused and band-focused single and multi-facet heliostats, center and scatter aim strategies, and different solar positions. Added methods for reading expected results, checking simulation outputs, and saving flux maps for validation. Also added a get_area() method to the Heliostat class for area calculations.
commented out because native runner is too slow to run as tests.
Added initialization of System->SunRayCount to 0 to ensure correct ray counting at the beginning of each run.
Consolidated and refactored heliostat field test cases to reduce code duplication and improve maintainability. Introduced helper methods for common simulation steps, parameterized test logic for different configurations, and replaced repetitive test cases with more modular and reusable code blocks. Also added TODOs for further refactoring in single heliostat tests.
Eliminated the print_info and print_sun_info parameters from several helper methods in heliostat_field_test.cpp, as they were no longer needed. Updated all method calls accordingly to improve code clarity.
Refactored the test fixture to modularize heliostat setup, simulation, and result checking. Added helper methods for setting up different heliostat configurations, reading expected results, and comparing simulation outputs. Consolidated and parameterized test cases for various heliostat positions and configurations, improving code reuse and maintainability.
Removed per-heliostat ray count vectors in favor of total counters, updated ray counting logic, and adjusted test tolerances and efficiency calculations. Added gap settings for heliostats, included heat shield elements in simulation stages, and improved output information for test diagnostics. Commented out sun size checks and cleaned up test structure.
Added a call to set the number of threads to 10 in the heliostat_field_test.cpp setup.
There was a problem hiding this comment.
Pull request overview
This PR adds validation tests for the native runner based on the round robin study published in Solar Energy journal. The tests compare native runner results against SolTrace reference data for various heliostat aiming strategies and time periods.
Key Changes:
- Added 16 CSV files containing SolTrace reference results for different task/aiming strategy combinations
- Added 1 flux data summary CSV comparing results across different solvers
- Updated CMakeLists.txt to copy the round_robin_study directory to the build location
Reviewed changes
Copilot reviewed 94 out of 173 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| google-tests/round_robin_study/results/phase_II/*.csv | Reference SolTrace flux map results for validation testing across multiple task and aiming strategy configurations |
| google-tests/round_robin_study/results/phase_II/fluxDataSummary_Task_3b_AimStrat_2_Hour_8.csv | Summary comparing flux data across SolTrace, Solstice, and TieSOL solvers |
| google-tests/CMakeLists.txt | Build configuration update to include round_robin_study directory in test binary output |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Are we concerned about the test duration? Current develop takes ~6 minutes to run, while the windows debug with round robin tests is 46 minutes. Should we reduce the number of rays for the heliostat field tests, or wait to merge until the native runner multi threading is merged in? |
The run times will be reduced once we merge the multi-threading native runner. We could skip the full field tests in debug configuration. |
added multi-threading to the single heliostat tests
Added a flag to include these.
taylorbrown75
left a comment
taylorbrown75
left a comment
There was a problem hiding this comment.
Looks good, and will be very helpful for future validation. I added a few comments on places that might speed up the tests (with help from copilot).
There was a problem hiding this comment.
Should we ignore the .vscode folder for git?
| EXPECT_NEAR(PeakFlux / 1.e3, expected_peak_flux, peak_tol * expected_peak_flux); | ||
|
|
||
| // RMS of flux values | ||
| if (!high_accuracy) calculate_receiver_flux_map(result, 100, 150, false); // Re-calculate |
There was a problem hiding this comment.
Is this making the high accuracy flux map that is skipped on line 594? Can we call calculate_receiver_flux_map(result, 100, 150, false) once for either case?
| } | ||
| } | ||
|
|
||
| void calculate_ray_counts(SimulationResult result) { |
There was a problem hiding this comment.
Recommend pass by reference since SimulationResult can be very large.
| NumberOfRays = 0; | ||
| } | ||
|
|
||
| bool calculate_receiver_flux_map(SimulationResult result, int nbinsx, int nbinsy, bool is_cylinder) { |
There was a problem hiding this comment.
Recommend pass by reference.
|
|
||
| } | ||
|
|
||
| void check_outputs(SimulationResult result) { |
There was a problem hiding this comment.
Recommend pass by reference.
Created validation tests for the native runner using the Heliostat CST template. These results are based on the round robin study published here: https://doi.org/10.1016/j.solener.2025.113785
This closes #78 for the native runner. Other runners can use this framework to perform the same validations cases.