Skip to content

fix: Fix BigQuery timestamp literal format in SQL unparsing#144

Merged
sgrebnov merged 1 commit intospiceai-52from
sgrebnov/0320-fix-bigquery-dialect
Mar 20, 2026
Merged

fix: Fix BigQuery timestamp literal format in SQL unparsing#144
sgrebnov merged 1 commit intospiceai-52from
sgrebnov/0320-fix-bigquery-dialect

Conversation

@sgrebnov
Copy link
Copy Markdown

@sgrebnov sgrebnov commented Mar 20, 2026

Which issue does this PR close?

The default Dialect::timestamp_with_tz_to_string uses dt.to_string() which produces timestamps with a space before the TimeZone offset:

2016-08-06 20:05:00 +00:00 <- invalid for BigQuery:
invalid timestamp: '2016-08-06 20:05:00 +00:00'; while executing the filter on column 'startTime' (query) (sqlstate: [0, 0, 0, 0, 0], vendor_code: -2147483648)

BigQuery rejects this format. Per the BigQuery timestamp docs, the offset must be attached directly to the time:

2016-08-06 20:05:00+00:00 <- valid

This causes filter pushdown to fail when unparsing timestamp predicates for BigQuery.

Changes

Override timestamp_with_tz_to_string in BigQueryDialect using chrono's %:z format specifier, which produces the offset without a leading space.

Before (default dt.to_string()):

CAST('2016-08-06 20:05:00 +00:00' AS TIMESTAMP)  -- BigQuery error

After (%:z format):

CAST('2016-08-06 20:05:00+00:00' AS TIMESTAMP)  -- valid BigQuery timestamp

Similar to DuckDB recent fix: apache#17653

@sgrebnov sgrebnov marked this pull request as ready for review March 20, 2026 10:43
Copilot AI review requested due to automatic review settings March 20, 2026 10:43
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes BigQuery SQL unparsing for TIMESTAMP values with time zone offsets by ensuring the UTC offset is formatted without a leading space, preventing BigQuery “invalid timestamp” errors during filter pushdown.

Changes:

  • Override Dialect::timestamp_with_tz_to_string in BigQueryDialect.
  • Format timestamp-with-TZ strings using chrono’s %:z to attach the offset directly to the time, with precision based on TimeUnit.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread datafusion/sql/src/unparser/dialect.rs
Comment thread datafusion/sql/src/unparser/dialect.rs
@sgrebnov sgrebnov self-assigned this Mar 20, 2026
@sgrebnov sgrebnov merged commit 497da28 into spiceai-52 Mar 20, 2026
4 checks passed
@sgrebnov sgrebnov deleted the sgrebnov/0320-fix-bigquery-dialect branch March 20, 2026 11:05
@sgrebnov
Copy link
Copy Markdown
Author

Upstream PR: apache#21103

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants