Fix #237 - Support multi-dimensional arrays in DataItemJsonConverter (supersedes #238)

[philnach]

Summary

Fixes #237 by adding multi-dimensional array support to DataItemJsonConverter. Supersedes #238 — original fix authored by @mlankamp, refactored to address review feedback and harden against pathological input.

What changed

• WriteFieldValue no longer recurses into itself with a sentinel empty-string field name. Instead, two dedicated helpers handle in-array writes:
  • WriteArrayItems(writer, items, includeNullFields, depth) — iterates items inside an already-opened JSON array.
  • WriteArrayItemValue(writer, item, includeNullFields, depth) — single per-item dispatcher; the new nested-IEnumerable branch opens an unnamed nested array via WriteStartArray() / WriteEndArray().
• Removes the implicit IsNullOrWhiteSpace(propertyName) sentinel from Fixes #237 - support multi-dimensional arrays #238 — legitimate empty-string and whitespace property names are now preserved verbatim.
• Adds MaxJsonDepth = 64 and an EnsureDepth guard threaded through WriteDataItem, WriteFieldValue, WriteArrayItems, and WriteArrayItemValue. Pathological / recursively nested input now throws InvalidOperationException with a clear message instead of stack-overflowing the host process.

Why

The original PR #238 worked, but during review I had two design concerns:

1. Sentinel-string overload. Passing "" as fieldName and branching on IsNullOrWhiteSpace overloads the method contract and makes legitimate empty-string keys ambiguous with the sentinel.
2. Recursion with no depth guard. Newtonsoft.Json was historically bitten by stack-overflow on adversarially nested input. Utf8JsonWriter's built-in MaxDepth is bypassed when callers use SkipValidation = true (the existing tests do), so we cannot rely on it. A deterministic exception is the right failure mode for a migration tool.

Tests

17 new tests in DataItemJsonConverterTests.cs:

GeoJSON / issue #237 scenarios

• GeoJsonPoint_Coordinates_OneDimensional (regression guard)
• GeoJsonLineString_Coordinates_TwoDimensional
• GeoJsonPolygon_Coordinates_ThreeDimensional
• AsJsonString_FullGeoJsonDocument — the exact failing document from Support multi-dimensional arrays when writing json (example geojson polygon) #237; asserts no more System.Collections.Generic.List leakage.

Nested-array edge cases

• Empty nested array [[]]
• Nested array with nulls [[null,1],[null]]
• Mixed scalars [[1,"a",true],[2.5,null]]
• Nested array of IDataItem
• Nested array of Dictionary<string,object?>
• Wide nested array smoke (200×10)

Regression / contract preservation

• Empty-string property name still emits "":...
• Whitespace property name preserved verbatim
• Deserialize → AsJsonString round-trip on issue-237 doc

Depth guard / stack-safety

• Array nesting at MaxJsonDepth succeeds
• Array nesting at MaxJsonDepth + 1 throws
• IDataItem nesting at MaxJsonDepth succeeds
• IDataItem nesting at MaxJsonDepth + 1 throws
• Mixed object/array nesting overflows correctly

All 125 Common.UnitTests pass (was 108; +17 new). Full solution test suite is green.

Compatibility

• WriteFieldValue and WriteDataItem public/internal signatures are unchanged — depth threading is via private overloads with default 0.
• No changes to Extensions/*.
• The arrayItem is not string && arrayItem is IEnumerable predicate is intentionally kept broad to preserve backward compatibility with the outer field-write path. Tightening it (e.g., to exclude byte[]) is left as a follow-up.

Closes #237
Closes #238

[mlankamp]

Thank you!

[philnach]

@markjbrown, can you take a look at this. thucnguyen77 is working with GeoJSON and has a scenario that can leverage dmt with.

[markjbrown]

Will approve.

I understand why someone would want/need this. My only concern is performance impact if someone goes nuts and starts writing data that has extraordinary number of nested levels.

Cosmos is optimized for indexing/search of nesting levels up to 3 layers deep. As the layers of nesting go up, the RU cost go up for both writes (due to indexing) and queries. (point reads are unaffected).

Non-blocking. Just wanted to call out potential edge case.