API, Core: implement types timestamp_ns and timestamptz_ns

[jacobmarble]

Helps #8657

This change is divided into two commits:

• API
• Core

This change adds field ChronoUnit unit to TimestampType, such that TimestampType now represents four specified types:

• timestamp (existing)
• timestamptz (existing)
• timestamp_ns (added in Spec: add nanosecond timestamp types #8683)
• timestamptz_ns (added in Spec: add nanosecond timestamp types #8683)

[jacobmarble]

I've tracked down the source of the four failing "core-tests" and will push a fix tomorrow.

[jacobmarble]

Fixing four failing tests caused me to do most of the new type work for Parquet and ORC as well. Please note that the ORC type attributes need an update in the spec, else it isn't possible to correctly convert an ORC timestamp to an Iceberg timestamp.

[emkornfield]

A few high level questions. Don't the Avro and Parquet details need to be updated?

Also, are these changes intended to be backdate into spec version 1 and 2 or part of v3?

[jacobmarble]

Don't the Avro and Parquet details need to be updated?

Yes.

(I'm new to the project.) My original intent here was to create multiple PRs, each with a different scope (API/Core, Avro, ORC, Parquet, Spark/Flink/Hive/etc). However, the cascade of failing tests in this PR suggests that one big change might be necessary. Do you have any feedback on this?

are these changes intended to be backdate into spec version 1 and 2 or part of v3?

These changes affect v3 only.

iceberg/format/spec.md

Lines 184 to 185 in 7c4bdaa

	| [v3](#version-3) | **`timestamp_ns`** | Timestamp, nanosecond precision, without timezone | [2] |
	| [v3](#version-3) | **`timestamptz_ns`** | Timestamp, nanosecond precision, with timezone | [2] |

iceberg/format/spec.md

Line 1302 in 7c4bdaa

Types `timestamp_ns` and `timestamptz_ns` are added in v3.

[jbonofre]

I'm starting to review. Thanks !

[rdblue]

@jacobmarble can you break this into smaller commits? There are a ton of files changed here and I'm concerned about catching problems with such a large PR.

[jbonofre]

I agree with @rdblue

For partition stat we had a PR dedicated for spec.
As it's not necessary the same reviewers, I think it's good to have one PR for spec (even if it's very small) and PRs for api and impl.

[jacobmarble]

@jacobmarble can you break this into smaller commits? There are a ton of files changed here and I'm concerned about catching problems with such a large PR.

I agree with @rdblue

For partition stat we had a PR dedicated for spec. As it's not necessary the same reviewers, I think it's good to have one PR for spec (even if it's very small) and PRs for api and impl.

Happy to split this up. I tried to scope this to API and Core, but the cascade of failing tests led to this.