Types: Emulate PostgreSQL's JSON(B) types using CrateDB's OBJECT

[amotl]

About

The test suite of meltano-tap-cratedb, derived from the corresponding PostgreSQL adapter, is using PostgreSQL's JSON and JSONB types. By emulating them using CrateDB's OBJECT type, the corresponding test cases succeed without further ado. 💯

References

This is coming from a monkeypatch to meltano-tap-cratedb.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5a0d6be0-4ad1-4e91-b4c6-815dc22f2492

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch amo/type-json-jsonb

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kneth]

What if the column policy is strict? Do we need a test of that case?

[amotl]

Good point.

column_policy = 'strict' is a table option. I don't think the dialect currently implements it, and I don't know if it has any impact on columns that are defined as OBJECT without qualifier.

The dialect currently provides the options crate_number_of_shards, crate_clustered_by, crate_number_of_replicas, crate_partitioned_by, crate_index, crate_columnstore, no column policy option in sight.

Other than this, the object type does not seem to implement any specials wrt. DDL generation.

sqlalchemy-cratedb/src/sqlalchemy_cratedb/type/object.py

Lines 64 to 79 in 889c767

	class ObjectTypeImpl(sqltypes.UserDefinedType, sqltypes.JSON):
	__visit_name__ = "OBJECT"
	cache_ok = False
	none_as_null = False
	# Designated name to refer to. `Object` is too ambiguous.
	ObjectType = MutableDict.as_mutable(ObjectTypeImpl)
	# Backward-compatibility aliases.
	_deprecated_Craty = ObjectType
	_deprecated_Object = ObjectType
	# https://www.lesinskis.com/deprecating-module-scope-variables.html
	deprecated_names = ["Craty", "Object"]

[amotl]

@coderabbitai: Please converge this conversation into a dedicated issue.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
gh issue create \
  --repo crate/sqlalchemy-cratedb \
  --title "feat: Support \`column_policy\` table option for CrateDB OBJECT type" \
  --assignee amotl \
  --body "## Overview

During the review of PR `#27` (adding \`test_ddl_with_json_columns\`), the question was raised whether the dialect should also support and test CrateDB's \`column_policy\` table option (e.g. \`STRICT\`, \`DYNAMIC\`, \`IGNORED\`) in DDL generation.

## Background

\`column_policy\` is a CrateDB table-level option that controls how columns are handled when inserting data that does not match the defined schema. It is not currently implemented in the dialect.

Currently implemented CrateDB-specific dialect options are:
- \`crate_number_of_shards\`
- \`crate_clustered_by\`
- \`crate_number_of_replicas\`
- \`crate_partitioned_by\`
- \`crate_index\`
- \`crate_columnstore\`

The \`ObjectType\` implementation does not currently handle any DDL-specific extras beyond what is defined in [\`src/sqlalchemy_cratedb/type/object.py\` (lines 64–79)](https://github.com/crate/sqlalchemy-cratedb/blob/889c7670960d28124746ee20a7eacbb2e538af45/src/sqlalchemy_cratedb/type/object.py#L64-L79).

## Tasks

- [ ] Investigate whether \`column_policy\` should be added as a supported table option in the CrateDB dialect
- [ ] If implemented, add corresponding DDL compiler tests (e.g. \`column_policy = 'strict'\`, \`column_policy = 'dynamic'\`, \`column_policy = 'ignored'\`)

## References

- PR: `#27`
- Discussion comment: https://github.com/crate/sqlalchemy-cratedb/pull/27#discussion_r3355672545"

Length of output: 1675

---

---

@amotl, I've created a dedicated issue to track this: #269 – feat: Support column_policy table option for CrateDB OBJECT type.

It summarises the conversation, lists the currently supported dialect options, and includes tasks to investigate and potentially implement column_policy support along with corresponding DDL compiler tests.