Skip to content

[bug](iceberg) fix iceberg sink writer with spill report error#62899

Open
zhangstar333 wants to merge 1 commit intoapache:masterfrom
zhangstar333:iceberg_spill_error
Open

[bug](iceberg) fix iceberg sink writer with spill report error#62899
zhangstar333 wants to merge 1 commit intoapache:masterfrom
zhangstar333:iceberg_spill_error

Conversation

@zhangstar333
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary:

spill thread with write thread is different, so add mutex in the write function.

mysql> create table web_sales
    -> order by (ws_sold_date_sk, ws_item_sk, ws_order_number)
    -> PROPERTIES (
    ->   'write-format'='parquet'
    -> )
    -> as select * from tpcds_sf1000_parquet.web_sales;
ERROR 1105 (HY000): errCode = 2, detailMessage = (172.20.49.239)[INTERNAL_ERROR][E6] Size of permutation (40800) is less than required (48960)        0#  doris::Exception::Exception(int, std::basic_string_view<char, std::char_traits<char> > const&, bool) at /home/zcp/repo_center/doris_branch-4.1/doris/be/src/common/exception.cpp:0
        1#  doris::Exception::Exception(int, std::basic_string_view<char, std::char_traits<char> > const&) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_stri
mysql>

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant