Skip to content

test: add full types boundary consistency coverage#351

Merged
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:full-types-boundary-tests
Jun 2, 2026
Merged

test: add full types boundary consistency coverage#351
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:full-types-boundary-tests

Conversation

@QuakeWang
Copy link
Copy Markdown
Contributor

Purpose

Linked issue: #258

The existing Spark-provisioned full types test covered mixed Parquet, ORC, and Avro reads, but it only used ordinary non-null values and did not explicitly assert that the scan plan included all three data file formats. This left boundary values, top-level nulls, and selected nested null/empty collection cases under-covered for Java/Spark-generated data files.

This PR adds a focused consistency fixture and read test for those cases.

Brief change log

  • Add full_types_boundary_table to the Spark provisioning script.
  • Write Java/Spark-generated Paimon data files in Parquet, ORC, and Avro formats.
  • Cover primitive min/max-style values, decimal boundaries, empty string/binary, pre-epoch date, timestamp micros, top-level nulls, nested nulls, and empty collections.
  • Add a Rust integration test that:
    • verifies the scan plan includes parquet, orc, and avro data files;
    • reads the table through FileSystemCatalog;
    • checks Arrow arrays and decoded row values.

Tests

  • cargo test -p paimon-integration-tests --test read_tables test_read_full_types_boundary_table -- --nocapture
  • cargo test -p paimon-integration-tests --test read_tables

API and Format

Documentation

@JingsongLi JingsongLi closed this Jun 2, 2026
@JingsongLi JingsongLi reopened this Jun 2, 2026
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the new Spark fixture and boundary read coverage. The table exercises parquet/orc/avro, null values, empty collections, primitive min/max boundaries, binary/date/timestamp/decimal cases, and the Rust assertions compile cleanly against the integration test crate. I did not find blocking issues.

Verification:

  • cargo test -p paimon-integration-tests test_read_full_types_boundary_table --no-run

@JingsongLi JingsongLi merged commit f99f5b7 into apache:main Jun 2, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants