Testing

Testing

How QueryForge is tested, and how to run those tests yourself.

---

Running everything

dotnet test QueryForge.slnx -c Release

That runs everything needing no setup — the core, in-memory, SQLite and EF Core-over-SQLite suites. The four server-backed engines are opt-in, so an ordinary build skips them instantly rather than waiting out a connection timeout per engine.

The projects

Project	Covers
PepperX.QueryForge.Tests	Core models, builder, validation, the shared query engine, HierarchyBuilder — in isolation
PepperX.QueryForge.Conformance	Not a test project. The shared suites every provider must satisfy
PepperX.QueryForge.Dapper.Tests	Compiled SQL per dialect, plus both shared suites executed against SQLite, PostgreSQL, MySQL, SQL Server and Oracle
PepperX.QueryForge.EFCore.Tests	Both shared suites executed through EF Core against the same five engines
PepperX.QueryForge.InMemory.Tests	Both shared suites executed against a plain collection

---

The two shared suites

Everything meaningful lives in PepperX.QueryForge.Conformance and is inherited by each provider, so a behaviour change shows up on every provider at once instead of drifting apart quietly.

QueryForgeConformanceTests

90 tests taking each input apart: Criteria (all eleven operators, nulls, unusable conditions, all four logic modes, multiple groups), Paging (windows, page counts, past-the-end, non-positive values), SelectColumns, SortColumns (multi-level, null placement, numeric-not-lexical) and GroupByColumns (one to three levels, null keys, group paging) — for flat and grouped results.

SalesScenarioTests

35 tests using the library the way an application does. A seeded order book of 36 orders across two years, nine countries, four statuses and three sales reps, then whole requests: a sales dashboard, a drill-down grid, an export, a search box, a client JSON payload posted verbatim.

Every expected value was derived from the seed data independently of the engine, so a wrong answer fails rather than being re-asserted.

Adding a provider to them

One subclass of each, supplying a RunAsync:

using PepperX.QueryForge.Conformance;

public sealed class InMemoryConformanceTests : QueryForgeConformanceTests
{
    private readonly List<Widget> _widgets = WidgetData.Fresh();

    protected override Task<QueryResult<Widget>> RunAsync(Query query)
        => _widgets.ToQueryResultAsync(query);
}

xUnit then runs the whole suite against it. That is the entire contract — see Extending QueryForge.

---

Running against real database servers

An engine runs when you either point it at a server explicitly, or ask for the built-in local defaults.

# Use the repository's local defaults for anything not configured explicitly
QUERYFORGE_DB_TESTS=1 dotnet test QueryForge.slnx -c Release

Both providers read the same variables, so one set of connection strings configures the Dapper and EF Core suites together:

export QUERYFORGE_POSTGRES="Host=localhost;Port=5432;Username=postgres;Password=postgres;Database=queryforge"
export QUERYFORGE_MYSQL="Server=localhost;Port=3306;Uid=root;Pwd=root;Database=queryforge"
export QUERYFORGE_MSSQL="Server=localhost,1433;User Id=sa;Password=Your_password123;Database=master;TrustServerCertificate=true"
export QUERYFORGE_ORACLE="User Id=system;Password=queryforge;Data Source=localhost:1521/FREEPDB1"

dotnet test QueryForge.slnx -c Release

Variable	Engine
QUERYFORGE_POSTGRES	PostgreSQL
QUERYFORGE_MYSQL	MySQL / MariaDB
QUERYFORGE_MSSQL	SQL Server
QUERYFORGE_ORACLE	Oracle
QUERYFORGE_DB_TESTS	1 to use built-in local defaults for anything not set explicitly
QUERYFORGE_REQUIRE_DB	1 to make an unreachable configured engine a failure rather than a skip

Oracle has no built-in local default — there is no ubiquitous throwaway instance the way there is for the others — so it runs only when pointed at one explicitly.

Throwaway servers with Docker

docker run -d --name qf-pg     -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:16
docker run -d --name qf-mysql  -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=queryforge -p 3306:3306 mysql:8
docker run -d --name qf-mssql  -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD='Your_password123' -p 1433:1433 \
  mcr.microsoft.com/mssql/server:2022-latest
docker run -d --name qf-oracle -e ORACLE_PASSWORD=queryforge -p 1521:1521 gvenzl/oracle-free:23-slim

SQL Server also installs natively on Debian and Ubuntu from Microsoft's repo if you would rather not use a container:

curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor \
  | sudo tee /usr/share/keyrings/microsoft-prod.gpg > /dev/null
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/microsoft-prod.gpg] \
  https://packages.microsoft.com/ubuntu/22.04/mssql-server-2022 jammy main" \
  | sudo tee /etc/apt/sources.list.d/mssql-server-2022.list
sudo apt-get update && sudo apt-get install -y mssql-server
sudo /opt/mssql/bin/mssql-conf setup

The tests create and drop their own tables (qf_widget, qf_order, qf_ef_widget, qf_ef_order), so an empty database is all they need.

To confirm a server was actually used rather than skipped, watch the skip count fall — each engine contributes 126 tests per provider.

Making a skip a failure

An unreachable engine skips its suites, which is right on a developer's machine and wrong in CI: a database that never started would leave a green run that tested nothing.

QUERYFORGE_REQUIRE_DB=1 dotnet test QueryForge.slnx -c Release

This turns "configured but unreachable" into a failure, so a run cannot claim to have covered an engine it never touched. It judges only engines you configured — an engine with no connection string is still simply absent.

---

Tests that need no database at all

The compiler and the dialects are pure functions, so most of the interesting assertions cost nothing.

Asserting generated SQL

using PepperX.QueryForge.Dapper.Compiler;
using PepperX.QueryForge.Dapper.Dialects;

var compiler = new SqlQueryCompiler(new PostgreSqlDialect());

var columns = new ColumnWhitelist(new Dictionary<string, Type?>
{
    ["UserId"]  = typeof(int),
    ["Country"] = typeof(string),
    ["Score"]   = typeof(decimal)
});

var sql = compiler.CompileRows(dapperQuery, columns);

Assert.Contains("\"Score\" DESC NULLS LAST", sql.Text);
Assert.Equal("Germany", sql.Parameters["p0"]);
Assert.DoesNotContain("Germany", sql.Text);      // values are never inlined

The dialect portability suite

The repository carries a suite that runs the same query through every dialect and asserts the invariants that must hold on all of them:

• no caller value appears in the SQL text;
• every identifier is quoted;
• a paging clause is present;
• null ordering is pinned;
• the group-count alias has no AS.

Adding a dialect means adding it to that suite, and the suite then tells you what you got wrong.

Validation

Pure, so no infrastructure:

[Fact]
public void Denied_column_is_stripped()
{
    var query = QueryBuilder.Select("UserId", "PasswordHash").Build();

    query.Validate(r => r.Select(c => c.Deny("PasswordHash")), QueryValidationMode.SilentStrip);

    Assert.Equal(["UserId"], query.SelectColumns);
}

Your application's query logic

Point the in-memory provider at seed data and you have a fast, real test of the same semantics the database will apply:

var result = SeedData.Users.ToQueryResult(query);

Keeping in mind the known divergences — string case in particular.

---

Continuous integration

.github/workflows/publish-nuget.yml has two test jobs.

Build and test runs on every push and pull request. It sets none of the QUERYFORGE_* variables, so the server-backed suites skip themselves and the job needs no services.

Test against real databases spins up PostgreSQL, MySQL, SQL Server and Oracle as service containers and runs the full matrix with QUERYFORGE_REQUIRE_DB=1. It is triggered manually (workflow_dispatch) and automatically on a v*.*.* release tag, so a version cannot be published without having been exercised against every engine the Dapper provider claims to support.

Because a release tag is a bad place to discover a broken engine, run the job by hand once on the commit you intend to tag — Actions → Build, Test, and Publish NuGet → Run workflow. It is the same job on the same containers, so a green dispatch means the tag will get the same answer.

What the gate has caught

Three times so far, and every catch was in the test fixtures rather than the library — which is the point: the suites cannot vouch for an engine they were never able to set up on.

• ORA-00955, 125 tests. The EF Core fixture dropped its tables with an undelimited name. Oracle folds that to upper case while EF Core creates a quoted lower-case table, so the drop matched nothing and the following CreateTables collided with what the previous fixture had left behind. The names now come from the model and are delimited by the provider's own ISqlGenerationHelper.
• No SQL Server health check. A slow first-run upgrade would have left the engine unreachable, skipping its suites and handing the gate a green run that never touched SQL Server. The container has a health check now, and QUERYFORGE_REQUIRE_DB makes that whole class of problem impossible to miss rather than merely less likely.
• ORA-01745, 35 tests. The Dapper seeder named each bind variable after its column, and the order table has a Number column — NUMBER is an Oracle datatype keyword, so :Number is rejected at parse time. Bind names are prefixed now, which sidesteps the entire reserved-word collision class on every engine at once. The widget table has no reserved-word column, which is exactly why this hid until an order-shaped table met Oracle.

---

Writing tests against QueryForge in your own project

Three levels, cheapest first:

1. Validation policies — pure, no infrastructure. Assert that a denied column is stripped or throws.
2. Query construction — build the Query your endpoint would build and assert its shape. Also pure.
3. End-to-end semantics — run the query through the in-memory provider against seed data and assert the rows. Fast, and exercises the real filtering rules.

Reserve database-backed tests for what genuinely needs a database: your schema, your collation, your indexes, and anything using a view, a table-valued function, or a stored procedure.

The single most valuable test to write is a parity test — the same query through your database provider and through the in-memory provider, asserting the same answer. See Cross-Provider Parity. Remember the unique tie-breaker in the sort, or the test is legitimately flaky.