Skip to content

feat: support 2-argument ceil(value, scale) for Spark compatibility#21562

Closed
SAY-5 wants to merge 4 commits intoapache:mainfrom
SAY-5:feat/ceil-two-arg-scale
Closed

feat: support 2-argument ceil(value, scale) for Spark compatibility#21562
SAY-5 wants to merge 4 commits intoapache:mainfrom
SAY-5:feat/ceil-two-arg-scale

Conversation

@SAY-5
Copy link
Copy Markdown

@SAY-5 SAY-5 commented Apr 12, 2026

Extends ceil to accept an optional scale parameter for Spark compatibility: ceil(3.145, 2) returns 3.15.

Implementation is ceil(value * 10^scale) / 10^scale. Handles Float32/Float64 with Int64 scale, both scalar and array inputs.

Closes #21560

Extends the ceil function to accept an optional scale parameter,
matching Spark SQL behavior: ceil(value, scale) rounds up to the
specified number of decimal places.

Implementation: ceil(value, scale) = ceil(value * 10^scale) / 10^scale

Supports Float32 and Float64 with Int64 scale, both scalar and array.

Closes apache#21560
@github-actions github-actions Bot added the functions Changes to functions implementation label Apr 12, 2026
@SAY-5
Copy link
Copy Markdown
Author

SAY-5 commented Apr 13, 2026

The cargo fmt and docs check failures need the full workspace to resolve locally. I can't run cargo fmt with a sparse checkout. If a maintainer could run cargo fmt --all and cargo doc to regenerate the function docs, that should clear both checks. Happy to rebase if needed.

@kumarUjjawal
Copy link
Copy Markdown
Contributor

The cargo fmt and docs check failures need the full workspace to resolve locally. I can't run cargo fmt with a sparse checkout.

I don't know what that means but you can run the fmt using ci/scripts/rust_fmt.sh similarly there is for the docs update ./dev/update_function_docs.sh.

@SAY-5
Copy link
Copy Markdown
Author

SAY-5 commented Apr 13, 2026

Thanks for the pointer to the scripts. I checked out the full workspace and ran cargo fmt -- no changes needed, the code was already formatted. The docs script needs a full cargo build to generate function tables, which I can't do from a sparse checkout. Could a maintainer run ./dev/update_function_docs.sh to regenerate the math function docs? That should clear the second check.

SAY-5 added 3 commits April 13, 2026 18:51
Run ./dev/update_function_docs.sh to regenerate scalar_functions.md,
aggregate_functions.md, and window_functions.md after adding 2-argument
ceil() in the previous commit.
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Apr 15, 2026
@SAY-5
Copy link
Copy Markdown
Author

SAY-5 commented Apr 15, 2026

Ran ./dev/update_function_docs.sh locally to regenerate the function docs. The updated scalar_functions.md now includes the 2-argument signature ceil(numeric_expression[, scale]) and example for ceil(3.145, 2) → 3.15. Latest commit regenerates all three doc files (aggregate, scalar, window).

@coderfender
Copy link
Copy Markdown
Contributor

@SAY-5 , could you run cargo fm --all on your repo and see if that doesnt help ?

@Jefffrey
Copy link
Copy Markdown
Contributor

This PR seems to be modifying the datafusion version of ceil and not the datafusion-spark function. Looks like we have a PR for the datafusion-spark version here:

@SAY-5
Copy link
Copy Markdown
Author

SAY-5 commented Apr 27, 2026

@Jefffrey thanks — you're right, I was modifying the core datafusion ceil and not the datafusion-spark function. Closing in favour of #21710 which addresses the right module. Apologies for the noise.

@SAY-5 SAY-5 closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support 2-argument ceil(value, scale) for Spark

4 participants