Skip to content

[PROTOTYPE] Add unnamed types support#115

Draft
Strech wants to merge 2 commits intomasterfrom
fix-unnamed-type-schemas
Draft

[PROTOTYPE] Add unnamed types support#115
Strech wants to merge 2 commits intomasterfrom
fix-unnamed-type-schemas

Conversation

@Strech
Copy link
Owner

@Strech Strech commented Apr 24, 2024

Resolves #95

This PR will add support for unnamed types. All codecs are going to support it. The only new limitation comes from Registry storage.

To "name" an unnamed type, we need its subject name and to get it we send a request to endpoint /schemas/ids/{id}/versions which exists only starting Confluent Schema Registry version 5.5.0.


TODO

  • Complete all todo's in the code
  • Replace io.confluent namespace with io.acme
  • Move confluent naming update into main
  • Resolver should use new information about schema (name & version)
  • Replace primitive schema with union and add tests on encoding/decoding
  • Confirm that Registrar also works
  • Update readme

@Strech Strech added feature Something new to build internal API Anything related to the internal API and implementations labels Apr 24, 2024
@Strech Strech added this to the Pre-major release 0.99 milestone Apr 24, 2024
@Strech Strech force-pushed the fix-unnamed-type-schemas branch 3 times, most recently from e569fd4 to fe58fab Compare April 25, 2024 11:54
assert encoded == "59B02128"
end

test "when payload is matching the Union schema and schema is resolvable" do
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Add decoding test!

@Strech Strech force-pushed the fix-unnamed-type-schemas branch from 9eba871 to a07dd77 Compare June 20, 2024 20:02
@Strech Strech force-pushed the fix-unnamed-type-schemas branch from a07dd77 to ca7842c Compare July 16, 2024 19:37
"""
@spec reference_lookup(String.t()) :: {:ok, String.t()} | {:error, term()}
def reference_lookup(_), do: {:ok, ~s({})}
@spec reference_lookup(String.t()) :: {:error, term()}
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec is not complete!

@Strech Strech force-pushed the fix-unnamed-type-schemas branch from ca7842c to 5c884cb Compare July 16, 2024 20:17
@Strech Strech changed the title Add unnamed types support [PROTO] Add unnamed types support Jul 16, 2024
@Strech Strech changed the title [PROTO] Add unnamed types support [PROTOTYPE] Add unnamed types support Jul 17, 2024
@Strech Strech force-pushed the fix-unnamed-type-schemas branch from 5c884cb to b7e2619 Compare July 17, 2024 21:39
@Strech Strech force-pushed the fix-unnamed-type-schemas branch from b7e2619 to 7340ef8 Compare July 17, 2024 21:43
alias Avrora.Schema.ReferenceCollector

@type reference_lookup_fun :: (String.t() -> {:ok, String.t()} | {:error, term()})
@undefined_name :undefined
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not much value

do: SchemaEncoder.from_json(body, &read_schema_file_by_name/1)
with {:ok, schema_name} <- Name.parse(key),
{:ok, body} <- read_schema_file_by_name(key) do
SchemaEncoder.from_json(body, name: schema_name.name, reference_lookup_fun: &read_schema_file_by_name/1)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a refactoring here, nested calls generate too much lookup table. Considering unnamed types, we need to overhaul how we store lookup tables and how we work with them.

@Strech Strech mentioned this pull request Aug 3, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Something new to build internal API Anything related to the internal API and implementations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unnamed_type error with primary type schema

1 participant