Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

awkward-array doesn't work with pyarrow 19.0.0 #3402

Open
pfackeldey opened this issue Feb 20, 2025 · 1 comment
Open

awkward-array doesn't work with pyarrow 19.0.0 #3402

pfackeldey opened this issue Feb 20, 2025 · 1 comment
Assignees
Labels
bug The problem described is something that must be fixed

Comments

@pfackeldey
Copy link
Collaborator

Version of Awkward Array

2.7.4

Description and code to reproduce

awkward-array doesn't work with pyarrow 19.0.0, however, it does work with 18 and earlier.

Running the test-suite with version 19, yields:

FAILED tests/test_1125_to_arrow_from_arrow.py::test_recordarray[False-False] - pyarrow.lib.ArrowInvalid: Column 'x' is declared non-nullable but contains nulls
FAILED tests/test_1125_to_arrow_from_arrow.py::test_recordarray[True-False] - pyarrow.lib.ArrowInvalid: Column 'x' is declared non-nullable but contains nulls
FAILED tests/test_1125_to_arrow_from_arrow.py::test_recordarray[True-True] - pyarrow.lib.ArrowInvalid: Column '0' is declared non-nullable but contains nulls
FAILED tests/test_1294_to_and_from_parquet.py::test_recordarray[False-through_parquet-True] - pyarrow.lib.ArrowInvalid: Column '0' is declared non-nullable but contains nulls
FAILED tests/test_1294_to_and_from_parquet.py::test_recordarray[True-through_parquet-True] - pyarrow.lib.ArrowInvalid: Column '0' is declared non-nullable but contains nulls
FAILED tests/test_1440_start_v2_to_parquet.py::test_recordarray[True-True] - RuntimeError: file metadata is only available after writer close

We should make sure that awkward's dependencies are pinned correctly, and also update the pyarrow version dependency for the test-suite here: https://github.com/scikit-hep/awkward/blob/main/requirements-test-full.txt#L6

@pfackeldey pfackeldey added the bug (unverified) The problem described would be a bug, but needs to be triaged label Feb 20, 2025
@bnavigator
Copy link
Contributor

Can confirm the same error on openSUSE with apache-arrow 19.0.1

Arrow reference: apache/arrow#41667 apache/arrow#44921

@ianna ianna added bug The problem described is something that must be fixed and removed bug (unverified) The problem described would be a bug, but needs to be triaged labels Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug The problem described is something that must be fixed
Projects
None yet
Development

No branches or pull requests

3 participants