Summary of open topics related to the tests #10129
Description
With the relase of 2.7.23 we moved to the new AppVeyor image and test against SQL Server 2022 and 2017. We also use the old image and test some commands against SQL Server 2008R2 Express.
I use a lab on Azure with multiple computers and multiple instances, including a failover cluster instance, to test more "real world" scenarios with only remote instances (SQL Server host is different from dbatools host).
Here is a list of problems that still need to be addressed.
On AppVeyor:
- Test stability: Some tests randomly fail. I see messages about databases or database transaktionlogfiles not beeing accessable. If I rerun, other tests fail.
- Testing both against new and old versions takes much more time. How much test time do we want to invest?
- We filter tests to changed commands, but only on the PR run. Every new branch and every merge to development triggers a complete run. In case we merge 5 open PRs at once, I think only one run at the end is needed.
On my lab:
- Some tests only work on local instances or the special setup on AppVeyor.
To be updated in the enxt days...