[6.x] Match entries with an unset field when querying with a not equals operator

[lazerg]

When a blueprint field has never been set on an entry, its key is simply missing from the data column, so json_extract returns NULL and where('archived', '!=', true) skips the entry. The Stache matches it. A template like {{ collection:things archived:isnt="true" }} therefore lists everything on the file driver and nothing here.

I've deliberately kept this much narrower than the change suggested in the issue. It only applies to != and <>, and only to a top level field key in the data column. Real columns are left alone, so where('date', '!=', ...) still follows normal SQL null rules. Nested paths are untouched too, so where('content->value', '<>', 1) still ignores entries whose content isn't shaped that way, as entries_are_found_using_where_with_json_value expects. No existing test needed changing.

Fixes #603.

[ryanmitchell]

As I said in the issue, I'm still not convinced this is necessary. With eloquent you can make a migration to backfill the data, and most developers using eloquent driver would understand and expect to do that.

This PR basically forces a whereNull check for any field you query, which is not necessaroly what you would want if you are doing direct queries.

Happy to be overruled @duncanmcclean

[lazerg]

One clarification on the scope, since "any field you query" is broader than what this actually does.

The rewrite only applies to != and <>, and only when the column resolves to a top level key in the data array. Real database columns never reach it, so direct queries against those keep normal SQL null behaviour. Nested paths don't reach it either, so your content->value test from 2022 is unchanged and still passes. The test file is additions only.

The rest is your call, I'm happy either way.

[ryanmitchell]

Yes I understand the scope.

However it would mean if I run $query->where('my_field', '<>', 1) it will now actually run $query->where('my_field', '<>', 1)->orWhereNull('my_field') which I don't think its correct, even if it matches the stache.

[ryanmitchell]

Chatted with Duncan off-thread on this and as its a breaking change we'll target the next major version with this and document it in the upgrade guide.