Skip to content

feat(bigquery-jdbc): support picosecond timestamps in Arrow Storage Read API and nested types - #14332

Open
keshavdandeva wants to merge 1 commit into
mainfrom
jdbc-picosecond-support-3
Open

feat(bigquery-jdbc): support picosecond timestamps in Arrow Storage Read API and nested types#14332
keshavdandeva wants to merge 1 commit into
mainfrom
jdbc-picosecond-support-3

Conversation

@keshavdandeva

Copy link
Copy Markdown
Contributor

b/544839155

This PR enables picosecond precision (TIMESTAMP(12)) support across the BigQuery Storage Read API (Arrow stream) and nested data structures (ARRAY, STRUCT, and RANGE) when EnableTimestampPicos=true.

Key Changes

  • Storage Read API Session Configuration: Configured TableReadOptions.arrowSerializationOptions with TIMESTAMP_PRECISION_PICOS in BigQueryStatement when EnableTimestampPicos is active.
  • Arrow Deserialization & Formatting:
    • Wired enableTimestampPicos into BigQueryArrowResultSet, formatting picosecond timestamps directly to 12 fractional digits for column retrieval.
    • Normalized Arrow Text instances to String across hot paths to prevent downstream BigQueryTypeRegistry conversion failures.
    • Formatted RANGE and nested RANGE bounds using schema-derived element types and precision flags.
  • Nested Types Propagation:
    • Propagated precision flags to BigQueryArrowArray and BigQueryArrowStruct.
    • Overrode getTargetClass() in BigQueryArrowArray to return String.class when picoseconds are enabled, preventing Array.newInstance mismatch exceptions.
  • Unit Tests:
    • Added unit test suites verifying picosecond vs. microsecond behavior across BigQueryArrowResultSet, BigQueryArrowArrayOfPrimitives, BigQueryArrowStruct, and BigQueryStatement.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for picosecond timestamp precision when retrieving Arrow data in the BigQuery JDBC driver. It propagates the enableTimestampPicos flag through BigQueryArrowArray, BigQueryArrowResultSet, and BigQueryArrowStruct, configures the Arrow serialization options in BigQueryStatement, and formats timestamps with high precision when enabled. Comprehensive unit tests are also added to verify this behavior. The review feedback suggests improving the robustness of getRangeElementType in BigQueryArrowResultSet by standardizing the type name to uppercase and adding a null check to prevent a potential NullPointerException when range bounds are unbounded.

@keshavdandeva

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for picosecond timestamp precision when reading Arrow data in the BigQuery JDBC driver. It propagates the 'enableTimestampPicos' flag across Arrow-related classes (BigQueryArrowArray, BigQueryArrowResultSet, BigQueryArrowStruct), configures ArrowSerializationOptions in BigQueryStatement, and updates BigQueryTemporalUtility to format timestamps with up to 12 fractional digits. The review feedback suggests making the range element type resolution more robust by converting the type string to uppercase before parsing, and warns about potential memory leaks in the unit tests due to unclosed RootAllocator and VectorSchemaRoot instances.

@keshavdandeva
keshavdandeva marked this pull request as ready for review September 9, 2026 16:49
@keshavdandeva
keshavdandeva requested review from a team as code owners September 9, 2026 16:49
@keshavdandeva
keshavdandeva added this pull request to stack #14335 September 9, 2026 17:52
@keshavdandeva
keshavdandeva force-pushed the jdbc-picosecond-support-3 branch from 0fdf436 to 7235227 Compare September 9, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant