Skip to content

Upgrade pandas dependency in databricks provider#67341

Closed
Prab-27 wants to merge 1 commit into
apache:mainfrom
Prab-27:upgrade-pandas
Closed

Upgrade pandas dependency in databricks provider#67341
Prab-27 wants to merge 1 commit into
apache:mainfrom
Prab-27:upgrade-pandas

Conversation

@Prab-27
Copy link
Copy Markdown
Contributor

@Prab-27 Prab-27 commented May 22, 2026

Upgrade pandas to >=3.0.0 in databricks pyproject.toml

closes : #52543
Related :pypi


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@eladkal
Copy link
Copy Markdown
Contributor

eladkal commented May 22, 2026

Upgrade pandas to >=3.0.0 in databricks pyproject.toml

I don't know if we want to force version 3.
Version 3 is very new and pandas is very sensitive

@Prab-27
Copy link
Copy Markdown
Contributor Author

Prab-27 commented May 23, 2026

Upgrade pandas to >=3.0.0 in databricks pyproject.toml

I don't know if we want to force version 3. Version 3 is very new and pandas is very sensitive

@eladkal Do we need to close this now (Without merge) ?
WDYT ?

@eladkal
Copy link
Copy Markdown
Contributor

eladkal commented May 23, 2026

Upgrade pandas to >=3.0.0 in databricks pyproject.toml

I don't know if we want to force version 3. Version 3 is very new and pandas is very sensitive

@eladkal Do we need to close this now (Without merge) ? WDYT ?

The goal is to find what is best way to unblock the dependecy. It requires some research. Sometimes the fix is not in airflow but with some of the upstream libraries.

@potiuk
Copy link
Copy Markdown
Member

potiuk commented May 24, 2026

@Prab-27 A few things need addressing before review — see our Pull Request quality criteria.

  • 15 failing checks across shared package tests (e.g. Basic tests / Shared providers_discovery tests, Basic tests / Shared secrets_backend tests, Basic tests / Shared observability tests). A pandas dependency upgrade in a single provider shouldn't normally break shared-package tests — worth verifying the diff is scoped correctly and that the pandas bump doesn't drag in incompatible transitive deps. See the Checks tab.

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@eladkal
Copy link
Copy Markdown
Contributor

eladkal commented May 25, 2026

I think we don't need to do anything. When databricks/databricks-sql-python#768 will be released CI should be resolve everything automaticly.

@Prab-27
Copy link
Copy Markdown
Contributor Author

Prab-27 commented May 25, 2026

Sounds Good @eladkal ! I'll close this PR and we can close this issue as completed !

@Prab-27 Prab-27 closed this May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pandas cannot be upgraded because of databricks-sql-connector

3 participants