Skip to content

feat(flux): convert walltime to minutes/FSD for batch_system="flux" - #5019

Open
jasonb5 wants to merge 3 commits into
masterfrom
fix/flux-walltime-minutes
Open

feat(flux): convert walltime to minutes/FSD for batch_system="flux"#5019
jasonb5 wants to merge 3 commits into
masterfrom
fix/flux-walltime-minutes

Conversation

@jasonb5

@jasonb5 jasonb5 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a flux-specific walltime conversion in EnvBatch.set_job_defaults(): Flux's -t/--time-limit accepts minutes or Flux Standard Duration (FSD, RFC 23), not HH:MM:SS
  • On the flux path, %H/%M/%S(/%D) in walltime_format expand to the total duration in that unit, so a matching FSD suffix yields a valid value (e.g. %Mm on 01:10:0070m, %Hh1.17h); no format defaults to FSD minutes (70m)
  • Accepted formats are whitelisted (%M, %Ss, %Mm, %Hh, %Dd); anything else (e.g. %H — a bare number flux would misread as minutes — or %H:%M:%S) fails loudly at case setup instead of submitting a wrong time limit
  • Partial values round up at two decimal places so a job is never allotted less time than requested
  • The generic format_time()/walltime_format path for all other batch systems is untouched
  • Existing deployed flux config (<walltime_format>%Mm</walltime_format> on Tuolumne) works unchanged

Validation

  • New parametrized unit tests for _flux_walltime covering all accepted formats, rounding edge cases, and rejected formats, plus mock-based set_job_defaults tests for the flux branch (01:10:0070m/70/1.17h, 00:10:3011m)
  • All existing test_unit_xml_env_batch.py tests pass, confirming non-flux behavior is unchanged
  • Docs build cleanly; new "Walltime Format" section documents both the positional and flux semantics
  • End-to-end validation on Tuolumne still to be confirmed by a maintainer with access

Closes #5017

jasonb5 added 3 commits July 28, 2026 14:01
Flux's -t/--time-limit accepts minutes or Flux Standard Duration
(RFC 23), not HH:MM:SS. On the flux path %H/%M/%S in walltime_format
expand to the total duration in that unit so a matching FSD suffix
yields a valid value (e.g. %Mm on 01:10:00 gives 70m). Formats that
would produce values flux rejects or misreads are rejected at setup.

Closes #5017
@github-actions

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Flux-specific walltime rendering path so CIME emits a Flux-compatible --time-limit value (minutes or Flux Standard Duration) without changing the existing format_time() / positional walltime_format behavior for other batch systems. Also updates documentation and adds unit tests to validate conversion behavior and error handling.

Changes:

  • Add EnvBatch._flux_walltime() and use it from EnvBatch.set_job_defaults() when batch_system is flux.
  • Document Flux-specific walltime_format semantics and accepted formats.
  • Add unit tests for Flux walltime conversion and the set_job_defaults() Flux branch.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
doc/source/ccs/model-configuration/variables/batch.rst Documents walltime_format, including Flux-specific total-duration semantics and accepted formats.
CIME/XML/env_batch.py Implements Flux-only walltime conversion/validation during job default setup.
CIME/tests/test_unit_xml_env_batch.py Adds pytest coverage for _flux_walltime() and Flux behavior in set_job_defaults().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +136 to +141
<batch_system type="flux" >
<batch_submit>flux batch</batch_submit>
<walltime_format>%Mm</walltime_format>
<submit_args>
<arg flag="--time" name="$JOB_WALLCLOCK_TIME"/>
</submit_args>
@jasonb5

jasonb5 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

Needs testing on actual machine still.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(flux): convert walltime to minutes/FSD for batch_system="flux"

2 participants