Skip to content

Add two-stream radiation model with simplified surface energy balance (force-restore) - #3950

Merged
asalmgren merged 229 commits into
erf-model:developmentfrom
hgopalan:ERF-Radiation
Sep 14, 2026
Merged

asalmgren merged 229 commits into
erf-model:developmentfrom
hgopalan:ERF-Radiation

Conversation

@hgopalan

@hgopalan hgopalan commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a two-stream atmospheric radiation model as an alternative to RRTMGP, together with an optional simplified surface energy balance (SEB) module that evolves surface temperature and moisture by a force-restore method where Noah-MP is not driving them. The model is selected with erf.radiation_model = "TwoStream", the same selector as RRTMGP and Simple, and runs through its own class rather than the RRTMGP interface.

Everything new is off by default, so a run that does not ask for two-stream radiation is unaffected.

Shortwave

  • Direct beam by Beer-Lambert attenuation, F_dir(z) = S0 cos(zenith) exp(-tau_cum(z) / cos(zenith)).
  • Diffuse field from the Meador-Weaver (1980) two-stream solution with Zdunkowski PIFM coefficients, parameterized by single-scattering albedo and asymmetry factor, with independent clear-sky and cloud values.
  • The layer solutions are combined by the adding method, so the surface albedo reflects the beam back into the column rather than being applied only as an absorbed fraction. Direct and diffuse albedo can differ (surface_albedo_sw_diffuse).
  • Optional Earth-Sun distance factor (Spencer 1971) scaling S0 by day of year.

Longwave

  • Gray-gas two-stream sweep on layer interfaces (Toon et al. 1989), with Stefan-Boltzmann surface emission at a configurable emissivity and reflection of the downwelling flux at the surface.
  • Cooling to space is set by the flux divergence, with no isothermal override.

Optical depth

Two models, selected by erf.radiation.tau_model:

  • per_layer (default): a fixed optical depth per layer, so the column value scales with the number of vertical cells.
  • mass: built from each layer's mass path rho dz, so the column optical depth is set by the atmosphere rather than by the grid. Shortwave mixes dry absorption, Rayleigh scattering, water vapour and cloud water with extinction-weighted single-scattering albedo and asymmetry factor. Longwave uses the corresponding mass absorption coefficients, with the cloud term expressed as the Stephens (1978) emissivity.

On top of either model: a prescribed cloud band, moisture- and cloud-aware dynamic optical depth diagnosed from qv and qc, prognostic cloud fraction diagnosed from relative humidity and cloud water, and a prescribed bulk aerosol optical depth that is constant or exponential with height.

Coupling

  • Heating rates are stored as potential-temperature tendencies, dtheta/dt = (dT/dt) / pi, matching the convention the RhoTheta source term and the RRTMGP path already use. They are written to qheating_rates and exposed as the qsrc_sw and qsrc_lw plot variables.
  • Applied once per slow step, during slow-RHS construction only, so the tendency is consistent across all fast substeps.
  • Time-varying solar zenith angle from simulation time, latitude, longitude and day of year, with a static fallback.
  • Per-column surface albedo, emissivity and temperature from an LSM when present, falling back to scalars.
  • Non-uniform vertical grids: layer thicknesses are the distances between the layer interfaces of the nodal heights z_phys_nd, with the top layer's own thickness.

Simplified surface energy balance

  • Allocates the surface flux fields, passing Noah-MP values through when it is active and using scalar fallbacks otherwise.
  • Diagnostic mode reports the residual R_net - H - LE - G.
  • Prognostic mode evolves surface temperature and moisture by force-restore with bounded explicit Euler updates, and switches itself off at any level where Noah-MP is already driving the surface state.

Grid requirement

The column sweep integrates a whole atmospheric column in one pass, so each grid must span the domain in the vertical. ERF only decomposes in z when amr.max_grid_size_z is smaller than the number of cells in z; setting it at least that large is sufficient. A vertically decomposed grid aborts with a message naming the input rather than returning heating rates that look plausible and are wrong. Results do not depend on the horizontal decomposition or on the MFIter tile size.

Input validation

Several misconfigurations used to be accepted in silence. None of them aborts, so existing inputs keep running, but each now prints a warning naming the input, the valid values, and what the run will actually do instead:

  • an unrecognized tau_model, tau_profile_type, aerosol_profile_type or diag_callsite_mode falls back to its default, which is stated in the warning;
  • aerosol_profile_type = table is not implemented and contributes no aerosol optical depth;
  • seb_prognostic_enable reports that it also switches on seb_enable and seb_diagnostic_enable, which it depends on;

Cost and scaling

Exec/CanonicalTests/Radiation/TwoStream_vs_RRTMGP_Timing/ measures what each solver costs per radiation call and how that cost scales, with everything except the solver held fixed.

Radiation solver cost and scaling

Measured on one CPU core, marginal cost per call:

Grid Cells Two-stream RRTMGP
42 x 4 x 42 7,056 1.08 ms 433 ms
84 x 4 x 42 14,112 2.07 ms 881 ms
84 x 4 x 84 28,224 4.27 ms 1,812 ms
168 x 4 x 84 56,448 8.55 ms 3,581 ms

The two solvers compute different things: RRTMGP resolves 224 shortwave and 256 longwave g-points with gas optics from lookup tables, while the two-stream solver does a single gray sweep per column. The difference is the cost of spectral detail, so this is a guide to choosing a solver for a given run rather than a ranking. Cost per cell is flat for both across a factor of eight in problem size, which means either one extrapolates predictably to a production grid.

Both radiation paths previously had no BL_PROFILE region, so radiation cost could not be separated from the dycore for either solver. One is now added at each entry point, and the measurement reads exclusive time from AMReX's TinyProfiler.

Keeping it fair: both inputs pull in one shared settings block through AMReX's FILE include, pinning the grid, timestep, sounding, moisture model and surface temperature; radiation is called every slow step on both sides; plotfiles and both diagnostic logs are off so file I/O stays outside the measured region. The reported figure is the marginal cost across two step counts, which cancels one-time work, notably the roughly 150 ms RRTMGP spends reading its lookup tables on the first call. check_timing_fairness.py enforces those invariants and needs no RRTMGP build.

Caveats: a single CPU core, RRTMGP targets GPUs, and cost per call is not cost per unit of accuracy.

Testing

  • 34 GoogleTest cases covering the shortwave and longwave kernels and the column sweep, including energy-budget closure, the Exner convention, surface reflection, and resolution independence of the mass model.
  • Two ctest regressions: a column-heating check that verifies the vertical structure and orientation of the heating rates, and a plotfile check that qsrc_sw and qsrc_lw are written.
  • 17 canonical cases under Exec/CanonicalTests/Radiation/, each with a Python checker, covering clear-sky and cloudy shortwave, scattering, longwave, non-uniform spacing, dynamic optical depth, prognostic cloud fraction, aerosol, diurnal solar geometry, surface heterogeneity, the mass optical-depth model, RhoTheta and YSUNew coupling, and all three SEB modes.

Documentation

  • Docs/sphinx_doc/theory/Radiation.rst covers the shortwave and longwave formulations, the optical-depth models, the potential-temperature convention, the SEB equations and the grid requirement.
  • Docs/sphinx_doc/Inputs.rst documents every input.
  • Exec/CanonicalTests/Radiation/inputs_twostream_master is the reference input file and lists all 79 erf.radiation.* parameters with defaults, units and descriptions.
  • The development notes that accompanied this work (RAD_DEVELOPMENT.md, RAD_MPI_SKILLS.md) and the benchmark suite's generated outputs (benchmark_summary.*, the radiation_*_diag.dat files under its cases) are not part of the PR; they are listed in .gitignore, as the Canonical_RANS notes were handled in Remove Canonical_RANS development notes from the case directory #3993. The theory and inputs pages above are the documentation of record, and nothing in the tree refers to the notes.

Changes since the development merge of 11 September

An audit of the branch, and the follow-ups it produced, in the commits after a6dbe9a.

Defects fixed

  • The development merge had dropped the HOL_abs declaration in the MRF scheme that MRF, YSUNew and YSU: stay inside MFIter tiles (tiled runs abort or store a garbage PBLH); restore #3961's HOL_abs bases #3972 relies on, so every CI build failed; the file is development's again.
  • The force-restore surface update multiplied its tendency by the absolute simulation time instead of the step size (2x to 72x the Euler step over a 36 s run). The driver now takes time and dt separately.
  • The column sweep ran twice per step: the post-dycore call re-swept the same old state. It now runs once, at the pre-dycore call; the post-dycore call reports that sweep's cached flux diagnostics and steps the surface state, so the diagnostics CSV is unchanged.
  • The prognostic surface temperature and moisture were not in the checkpoint, so a restart reset them to the scalar defaults. They are written as Level_<lev>/TwoStream_TSfc and TwoStream_QSfc and read back when present.
  • Tests/Unit/Utils/ERF_GTestTerrainFineColumns.cpp (from BTF fine level: make the terrain-fitted mesh independent of the z split #3976) odr-used an anonymous-namespace constexpr inside a device lambda, which nvcc rejects; the lambda now captures a local copy. This is the CUDA failure that showed on the merged head and will show on development until this merges.

SEB coupling and inputs

  • New opt-in erf.radiation.seb_use_radiation_fluxes: where the LSM does not supply them, the net surface shortwave and longwave fluxes of the energy balance are the sweep's own per-column surface fluxes rather than the seb_*_default constants. Noah-MP fields are mapped as sav + sag for absorbed shortwave and -fira for absorbed longwave; the deep-soil reservoir values are the scalar defaults, since no LSM exposes them by name (the previous mapping had read soil-moisture fields).
  • erf.radiation_model = TwoStream with amr.max_level > 0 now stops at start-up with the reason (the sweep needs whole columns of its level) instead of asserting on max_grid_size_z at the first fine-level call. The theory page gains a Limitations section (single level, solar time base of 00:00 UTC on day_of_year, the diagnostics file written by default).
  • erf.ysu_rad_tend_smooth_strength was read and never used and is removed; a non-positive ysu_rad_tend_limiter_magnitude warns instead of silently becoming 1. Both YSUNew keys and the new SEB key are in Inputs.rst and the master deck.

Structure

  • The model is now a class, TwoStreamRadiation (Source/Radiation/ERF_TwoStreamRadiation.H/.cpp), owning the per-level surface and SEB fields, the flux cache and the diagnostics writer. ERF keeps one member and calls define_level, advance (pre- and post-dycore), write_checkpoint and read_checkpoint, each behind an explicit solverChoice.radChoice.rad_type == RadType::TwoStream test; every method re-tests it. The heating-rate array stays in ERF because RRTMGP fills the same one.
  • Assertions where a broken contract would otherwise surface as a wrong number elsewhere: call site, level defined, the 2D fields laid out like the 3D state (the sweep indexes them with the state's MFIter), the heating array's shape, SEB fields present, positive finite dt; aborts naming the level and step on a non-finite state into the sweep, non-finite heating rates out of it, or a non-finite surface state after the force-restore step.
  • Ghost and fill audit: every read in the sweep is at the column's own (i, j) with k inside the box; the 2D fields are read on valid cells; no FillBoundary or FillPatch is needed and none is done.

Tests

  • TwoStream_ColumnHeating is registered only with MPI and not on Windows, like the other cmake -P tests; the RRTMGP Radiation regtest that the ERF_ENABLE_RRGMTP typo fix had re-enabled, with no input files, is additionally gated on its input directory.
  • The SEB prognostic case gains a time-integration check (each step must change T_s by dt times the tendency), a restart mode with its own deck, and a coupled mode for the new flux option. All 17 canonical cases, the benchmark suite and the two CTests pass on the refactored code, with no unused erf.radiation or YSUNew key in any run.

Tree hygiene

  • The development notes and the run outputs that had been committed (benchmark summaries, diagnostics CSVs, data logs) are untracked and in .gitignore, as Remove Canonical_RANS development notes from the case directory #3993 did for the Canonical_RANS notes; the stray build/cmake_uninstall.cmake and the googletest submodule bump are gone; development-phase wording is removed from the branch's files.

Review response (13 September)

All ten inline findings are addressed in 5efc5eb; each thread has a reply saying what changed. In short: every domain-mean and maximum in the diagnostics is MPI-reduced, with a terrain-fitted CTest (TwoStream_ColumnHeating_Terrain) whose runner requires the diagnostics CSV to be identical on 1 rank and on the CI rank count; the prognostic cloud fraction no longer discards the dynamic optical depth; cloud_fraction_qc_scale is the saturating cloud water, cf += min(1, qc / scale); layer thicknesses come from the interface heights, the top layer included; the azimuth formula is corrected (and its sine sign convention) with gtests; the per-column scratch is a per-box FArrayBox instead of ~37 kB of device stack; the diagnostics writer persists; ERF_SimplifiedSEB.H is in Make.package; the plotfile gtest drives the real capability predicate. The same defect classes were searched for elsewhere in the model: the 46 silent input clamps in RadChoice::init_params now warn with the key and the value used; no other rank-local diagnostic, fixed-size device array or unused exported function was found. The driver now also refuses a finite but non-positive density or rho·theta before the sweep (a collective minimum), so the kernel's placeholder substitutions are unreachable from a run. Five gtests were added (553 pass), and the 17 canonical cases, the benchmark suite and the three CTests pass on the fixed code.

Selector unification (Slack request, 13 September). The branch had introduced a second selector, erf.radiation_type, next to development's erf.radiation_model. That is gone: TwoStream is now a value of erf.radiation_model (the RadiationType enum), RadType and erf.radiation_type no longer exist, and the erf.radiation.* sub-options are read only when that value is selected. The IRadiation models (RRTMGP, Simple) are gated on SolverChoice::rad_uses_interface(), the two-stream model on rad_type == RadiationType::TwoStream, and everything that any radiation model shares (heating-rate storage, the RhoTheta source, the plotfile variables) on rad_type != None. Every deck, README, checker and doc page uses erf.radiation_model = "TwoStream".

Copilot AI and others added 30 commits August 6, 2026 23:51
…nd Solvers

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…m-radiation

Phase 1: Two-stream atmospheric radiation module—infrastructure and clear-sky solver
Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
- Add ERF_RadiationDiagnostics.cpp and ERF_AdvanceTwoStreamRadiation.cpp to CMake build
- Add Source/Radiation include directory to CMake
- Add ERF_AdvanceTwoStreamRadiation.cpp to GNUmake Make.package
- Add compute_twostream_radiation_diagnostics() function declaration to ERF.H
- Implement compute_twostream_radiation_diagnostics() for Phase 1 diagnostic output
- Call compute_twostream_radiation_diagnostics() from ERF::Advance() at end of timestep
- radChoice member and init_params() call already in place (Phase 1 PR)

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…iation-module

Wire Phase 1 Radiation Module into Build System and Timestep Loop
…n test folders

Previously sounding_us_standard_atm lived at Exec/CanonicalTests/Radiation/
(outside both case folders) and was shared by reference across
SW_ClearSky_Analytical and LW_Isothermal. Per repo convention (see
Exec/CanonicalTests/ABL/*, e.g. input_sounding_GABLS1, mrf_sounding_unstable),
sounding files must live inside their own case folder and be named after
that case. This adds input_sounding_sw_clearsky and
input_sounding_lw_isothermal inside their respective folders and updates
the inputs files to reference them. The old shared file is removed.
Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
… kernel

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…AD_MPI_SKILLS.md

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…radiation

Phase 2: Real per-column vertical integration for two-stream radiation
…plementation

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…olumn-kernel

Wire per-column radiation kernel into diagnostics driver
… real dz

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…nown issues

Co-authored-by: hgopalan <1108371+hgopalan@users.noreply.github.com>
…completion

Phase 2c: GPU-safe ParallelFor, real LW downward sweep, and documentation updates
…raction masking)

- Add erf.radiation.tau_profile_type (constant|cloud_layer) to RadChoice
- Add cloud_base_height_m, cloud_top_height_m, cloud_tau_per_layer params
- Add erf.radiation.cloud_fraction (default 0.0, byte-identical when 0)
- Extend vertical_two_stream_sweep() to compute clear-sky and cloudy-column
  SW/LW fluxes and blend via F = (1-cf)*F_clear + cf*F_cloudy
- Add SW_Cloud_Layer RegTest
- Update RAD_DEVELOPMENT.md and RAD_MPI_SKILLS.md with Phase 3 + Phase 2d sections
asalmgren and others added 11 commits September 11, 2026 01:56
Commit 6b6f8e0 removed the third-site HOL_abs in ERF_ComputeDiffusivityMRF.cpp
while development had it unused; erf-model#3972 then restored pow(1 + 16*HOL_abs, -1/4)
at that site, and the merge kept the use without the declaration, so every CI
build job failed on an undeclared identifier. This takes development's copy of
the file; that deletion was this branch's only change to it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ce state

Three defects in the two-stream driver, each found and reproduced on the
TwoStream_SEB_Prognostic canonical case:

- The force-restore Euler update multiplied the tendency by the argument
  named time_step, which the call sites filled with the absolute simulation
  time (t_old at pre_dycore, time + dt at post_dycore). With dt = 0.5 s the
  per-step change of T_s grew from 2x to 72x the Euler step over the 36 s
  run, and T_s fell 1.08 K instead of 0.03 K. The driver now takes time and
  dt separately and steps by dt.

- The post_dycore call re-ran the full column sweep on vars_old, which is
  still the old state at that point, so radiation was computed twice per
  step for identical results (144 sweeps for 72 steps). The sweep now runs
  only at pre_dycore; the post call reports the cached domain-mean flux
  diagnostics of that sweep and runs the surface update, so the CSV rows
  are unchanged.

- twostream_t_sfc and q_sfc were not in the checkpoint, so a restart reset
  the prognostic surface state to the scalar defaults (299.98 K against
  299.70 K in the uninterrupted run at the restart step). Both are now
  written as Level_<lev>/TwoStream_TSfc and TwoStream_QSfc and read back
  when present; older checkpoints keep the defaults.

The canonical checker gains a time-integration check (each post_dycore row
must advance T_s by dt times the force-restore tendency, allowing for the
CSV's 1e-4 K quantisation, plus a cumulative check) and a "restart" mode
driven by the new inputs_seb_prognostic_restart deck, which restarts from
the checkpoint the enabled deck now writes at step 36. Against the output of
the previous binary the time-integration check fails on 71 of 71 steps and
the restart check by 0.28 K; with this change both pass, the restarted
surface state agreeing to 0.0 with the fresh run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Surface energy balance
- New opt-in erf.radiation.seb_use_radiation_fluxes: where the LSM does not
  supply them, the SEB's net surface shortwave and longwave fluxes are the
  two-stream sweep's own per-column surface fluxes (absorbed, positive
  down) instead of the seb_sw_flux_default / seb_lw_flux_default constants.
  The sweep writes them at the pre-dycore call and the post-dycore call
  leaves them in place. Default off, so existing decks are unchanged.
- Noah-MP mapping corrected: absorbed shortwave is sav + sag, and fira (net
  longwave to the atmosphere, positive up) enters with its sign flipped.
  t_deep and q_deep were being filled from smstav and smstot, which are
  soil-moisture availability and total column water; no LSM exposes a
  deep-soil temperature or moisture in kg/kg by name, so the reservoir
  values are the scalar defaults, and the docs say so.
- TwoStream_SEB_Prognostic gains inputs_seb_prognostic_coupled and a
  "coupled" checker mode that verifies SEB_residual_mean equals
  SW_surface - LW_net_surface - (H + LE + G) on every row and that the
  fluxes differ from the defaults.

Multi-level runs
- The sweep asserts that every box spans its level's domain in z, which no
  fine-level box does, so any max_level > 0 run aborted at the first
  fine-level radiation call with a message about amr.max_grid_size_z.
  init_params now aborts at start-up with the actual reason; the theory
  docs gain a Limitations section (single level, solar time base of 00:00
  UTC on day_of_year, radiation_diag.dat written by default).

Tests
- TwoStream_ColumnHeating runs through cmake -P / execute_process and is
  now registered only with ERF_ENABLE_MPI and not on WIN32, like the other
  script-driven tests; it failed in 0.03 s on the Windows job.
- The ERF_ENABLE_RRGMTP typo fix had re-registered add_test_r(Radiation),
  whose Tests/test_files/Radiation inputs have never existed, so the
  RRTMGP ON job failed it; the test is now also gated on that directory.

Inputs
- erf.ysu_rad_tend_smooth_strength was read, clamped, printed and never
  used; removed from TurbStruct, the coupling deck, its README and the
  development notes. A non-positive ysu_rad_tend_limiter_magnitude now
  warns instead of silently becoming 1. Both YSUNew keys and the new SEB
  key are documented in Inputs.rst and the master deck.

Unrelated diffs dropped
- build/cmake_uninstall.cmake (a build artefact), the googletest submodule
  pointer (back to development's v1.17.0 commit), the commented-out copy of
  the old radiation source block and its "two Add" heading in
  ERF_MakeSources.cpp, the duplicate angle-bracket includes in
  ERF_DataStruct.H, and three stale Source/Radiation/RAD_DEVELOPMENT.md
  references (the file lives under Exec/CanonicalTests/Radiation).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…wording

After merging development (through erf-model#3995):

- Tests/CMakeLists.txt: both sides had added a checker executable to the
  CUDA foreach list; the merge resolution keeps erf_two_stream_radiation_check
  and erf_cloud_chamber_wall_dt_guard_check.
- The HIP job of the previous head flagged -Wshadow on the new dt argument of
  compute_twostream_radiation_diagnostics, which shadows the ERF::dt member;
  renamed to dt_step.

Design notes and generated outputs leave the tree. RAD_DEVELOPMENT.md and
RAD_MPI_SKILLS.md under Exec/CanonicalTests/Radiation are development notes,
not documentation of the model (the theory and inputs docs are), and the
benchmark suite's benchmark_summary.{json,md} and the radiation_*_diag.dat
files under its cases are the outputs of running it. All are removed from git
and listed in .gitignore, and every "see RAD_DEVELOPMENT.md" pointer in the
headers, the master deck and the case READMEs is dropped, the same way
development handled the Canonical_RANS notes in erf-model#3993.

Development-phase wording is gone from the PR's files: the "Phase 11" line in
the SurfaceHeterogeneity README, "previous phases (1-9)" in the NonuniformDZ
deck, "prior phases" in ERF_SolarGeometry.H, the PHASE6_TIMING key of the
benchmark runner, and the "smoother" wording the coupling case still carried
for the input removed earlier. The three "Phase 12/13" comments in
ERF_ComputeDiffusivityYSUNew.cpp are development's own, not this branch's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ace constexpr

The fill_interpolated_mesh device lambda in ERF_GTestTerrainFineColumns.cpp
(added in erf-model#3976) passes the anonymous-namespace constexpr nz to amrex::min,
which takes its arguments by reference, so the lambda odr-uses nz and nvcc
stops with "identifier nz is undefined in device code". This failed the CUDA
SINGLE job on the merged head of this branch and fails on development the
same way. The lambda now captures a local copy of the value.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The model's state and driver used to live in ERF itself: eleven per-level
MultiFab vectors, a flux-diagnostics cache and a member function of ERF that
reached into vars_old, geom, lsm, z_phys_cc and qheating_rates. They are now
one class, TwoStreamRadiation (Source/Radiation/ERF_TwoStreamRadiation.H/.cpp,
replacing ERF_AdvanceTwoStreamRadiation.cpp), and ERF holds a single member
and calls it only behind an explicit test of
solverChoice.radChoice.rad_type == RadType::TwoStream:

- ERF_MakeNewArrays: define_level(lev, radChoice, ba2d, dm) allocates and
  default-fills the 2D surface and SEB fields;
- ERF_AdvanceRadiation: advance(..., "pre_dycore", vars_old cons, z_phys_cc,
  geom, lsm, qheating_rates) runs the column sweep and writes the heating rates;
- ERF_Advance: advance(..., "post_dycore", ...) reports the cached flux
  diagnostics and steps the force-restore state by dt;
- ERF_Checkpoint: write_checkpoint / read_checkpoint carry T_s and q_s.

Every method also tests the predicate itself and returns otherwise. The
heating-rate MultiFab stays in ERF because RRTMGP fills the same array and
the RhoTheta source and the plotfile read it regardless of the model.

Assertions added where a wrong contract would otherwise surface as a wrong
number several routines away: call_site must be pre_dycore or post_dycore;
the level must have been defined; the 2D fields must have the state's box
count and distribution (the sweep indexes them with the state's MFIter);
qheating must be the 2-component field on the state's grids; the SEB fields
must exist when seb_enable is set; the force-restore step needs a positive
finite dt. Aborts with the level and step: a non-finite density or
rho*theta handed to the sweep (the kernel would clamp it silently), a
non-finite heating rate after the sweep (it feeds the RhoTheta source next
step), and a non-finite surface state after the force-restore update.

Ghost and fill audit: every read in the sweep is at the column's own (i, j)
and at k within the box (the cell-centre heights are read up to kmax), the
2D fields are read at (i, j, 0) of valid cells, the source term and plotfile
read the heating rates on valid cells; no FillBoundary or FillPatch is
needed and none is done. The one-cell x/y ghost ring of the 2D fields is
never read; it is kept so existing checkpoints read back unchanged.

Results are unchanged: SEB baseline/feature_on/restart/coupled checks,
the column-heating deck with FPE traps, 1- vs 4-rank plotfile parity and
the unit tests all pass as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The sweep of all 17 canonical cases, the benchmark suite and the two
CTests against the TwoStreamRadiation class left every checker passing and
no unused erf.radiation or YSUNew key in any run log; the 80 erf.radiation.*
keys the master deck lists are exactly the 80 RadChoice reads.

What it also showed: nine radiation_*_diag.dat files under LW_Isothermal,
SW_*, TwoStream_MassOpticalDepth and TwoStream_TimeIntegration were still
tracked although the checkers regenerate them from the runs, and each run
also writes ERF's *_hist.dat / *_profiles.dat data logs and the benchmark
suite's _runs/ directory into the case directories. All of those are now
untracked and covered by .gitignore, so running the cases leaves the tree
clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The "Implementation Summary" sections of the SEB_Diagnostic and
SEB_Prognostic READMEs were change-log material (new and modified files,
design decisions), the same kind of content the RAD_DEVELOPMENT notes held,
and the Prognostic one was stale: it said the surface state feeds nothing
back, which the coupling option and the LW surface emission both
contradict. The test design, validation criteria and run instructions are
unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@asalmgren asalmgren left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: two-stream radiation + simplified SEB

Reviewed the ~6,400 lines of new code under Source/, Tests/, CMake/, Exec/Make.ERF. The remaining ~75k lines (canonical test decks, Python checkers, docs) were skimmed rather than audited line by line.

The core physics holds up. The Meador-Weaver layer solution and the adding-method upward/downward passes match the RRTMGP formulation, including the src(sfc) = alpha_dir * F_dir(0) boundary condition and the placement of the denom = 1 - R_dif * A factor. The LW downward-then-upward ordering correctly precedes the gray-surface reflection term. Orientation (k=0 at the surface), the Exner conversion, and the once-per-slow-step injection into make_sources all look right.

10 findings inline: 1 high, 5 medium, 4 low. The high one is #1 — it's the claim the PR makes about itself in the description.

Also verified as correct: the diffuse/direct albedo split vs. RRTMGP's sw_solver_2stream; &hetero_alb_sw_arr taken inside the [=] device lambda refers to the closure copy rather than a host stack address; ba2d[lev] has setRange(2,0) and preserves box order, so (i,j,0) indexing of the 2D fields against the 3D state's MFIter is valid; std::swap(vars_old, vars_new) happens at the top of ERF::Advance so the post-dycore call sees the intended old state; iteration equals istep[lev], so the two CSV call sites agree on the step index; read_checkpoint is placed after MakeNewLevelFromScratch so define_level has run on restart; get_qv_from_state/get_qc_from_state guard correctly against RhoQ1_comp/RhoQ2_comp when moisture is off; and the ERF_ENABLE_RRGMTP -> ERF_ENABLE_RRTMGP typo fix uses the real option name from CMakeLists.txt:141.

// True horizontal heterogeneity (e.g., patchy clouds varying by column)
// remains deferred to future work.
if (do_sweep) {
if (n_columns_total > 0) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[high] Every "domain mean" diagnostic is rank-local; no MPI reduction.

sw_surface_sum, sw_up_toa_sum, lw_net_sum, lw_up_toa_sum and n_columns_total here — and max_heating_global (line 649), seb_residual_sum/n_seb_columns (line 730), t_s_sum/q_s_sum/n_prog_columns (lines 847-851) — are accumulated over the MFIter of the local boxes only, then divided by the local column count and printed/written by the IOProcessor. There is no ParallelDescriptor::ReduceRealSum / ReduceRealMax anywhere in this file.

Run on 4 ranks with erf.radiation.seb_enable and a heterogeneous surface: SW_surface, LW_net_surface, T_s_mean and heating_rate_max in radiation_diag.dat describe rank 0's subdomain only, and change when you change the decomposition — which contradicts the "results do not depend on the horizontal decomposition" claim in the PR description.

heating_rate_max is also the safety diagnostic, so a runaway column on rank 3 is invisible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. Every accumulator formed over the MFIter (the four flux sums, the column count, max heating, the SEB residual sum/max/count, the prognostic T_s/q_s sums/maxima/count) is now reduced with ParallelDescriptor::Reduce{Real,Long}{Sum,Max} before the means and maxima are formed; the collectives sit under input-driven conditions that are identical on all ranks.

To keep it that way, RunTwoStreamRadiation.cmake now also runs the deck on one rank and requires the diagnostics CSV to match the NRANKS run byte for byte, and a new CTest TwoStream_ColumnHeating_Terrain runs the column over a Witch-of-Agnesi hill on a terrain-fitted mesh with two boxes of unequal width, so the columns differ and rank 0's share has a different mean from the whole. On the previous binary the two CSVs differ from the first row (3.193070e+02 vs the global SW_surface); on this one they are identical. A centred hill on a 4x4 domain did not discriminate: the two boxes are mirror images.

Comment thread Source/Radiation/ERF_TwoStreamColumn.H Outdated
is_cloud_level(z_center, rad_choice)) {
amrex::Real cf_prog = diagnose_cloud_fraction_prognostic(i, j, k, state_arr, rad_choice);
amrex::Real tau_clear_k = tau_layer_value(z_center, base, rad_choice, /*apply_cloud=*/false);
tau = tau_clear_k + cf_prog * rad_choice.cloud_tau_per_layer;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] The prognostic-cloud-fraction branch throws away the dynamic moisture optical depth.

diagnose_layer_tau computes tau including diagnose_tau_dynamic(...), and then this line unconditionally overwrites it with tau_clear_k + cf_prog * cloud_tau_per_layer, where tau_clear_k is the base value computed with no dynamic term.

With tau_sw_dynamic_enable = true, cloud_fraction_prog_enable = true and tau_profile_type = cloud_layer, every layer inside the cloud band silently loses its coeff_qv*qv + coeff_qc*qc contribution. The doc comment says the cloud fraction only "replaces the cloud-band enhancement", so code and documented intent disagree.

The same loss propagates into the mass model, where additions = diagnose_layer_tau(..., 0.0, ...) - band then evaluates to just the aerosol term.

Suggest adding the dynamic term back onto tau_clear_k, or computing the replacement relative to the already-computed tau rather than re-deriving from the base.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. The cloud-band enhancement is now scaled in place, tau += (cf - 1) * cloud_tau_per_layer, on the tau that already carries the dynamic term, so nothing is re-derived from the clear-sky base and the mass model's additions term keeps the moisture contribution. gtest DynamicOpticalDepthSurvivesPrognosticCloudFraction checks that switching the dynamic term on changes a cloud-band layer by exactly coeff_qv * qv with the prognostic cloud fraction active, and that it equals the fixed-cloud-fraction value at cf = 1.


// qc contribution: scaled by qc_scale
if (qc_scale > 0.0) {
amrex::Real cf_qc = qc_scale * qc;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] cloud_fraction_qc_scale is applied as a multiplier, making the qc term dead at its default.

cf_qc = qc_scale * qc with the default qc_scale = 1.0e-3 and a realistic qc ~ 1e-3 kg/kg gives cf_qc = 1e-6 — i.e. no contribution at all.

ERF_RadStruct.H:549 describes this as the factor that "converts qc [kg/kg] to dimensionless cloud fraction contribution", which implies qc / qc_scale (a threshold), not qc * qc_scale. As written, a user who turns on cloud_fraction_prog_enable for a cloudy column gets only the RH ramp; the liquid-water path is ignored.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb: cf += min(1, qc / cloud_fraction_qc_scale), so the input is the cloud water at which the liquid-water term alone saturates (1e-3 kg/kg by default). The RadStruct comment, Inputs.rst, the theory page's formula and the master deck now say that; gtest CloudFractionLiquidWaterTermIsAThreshold pins 1e-3 -> 1, 5e-4 -> 0.5. The ProgCloudFraction canonical case still passes its checker.

Comment thread Source/Radiation/ERF_TwoStreamColumn.H Outdated
}
if (z_phys_cc) {
for (int m = 0; m + 1 < nlev; ++m) {
amrex::Real dz_computed = z_phys_cc(i, j, kmin + m + 1) - z_phys_cc(i, j, kmin + m);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] Layer thickness on a non-uniform grid is the cell-centre spacing, not the layer thickness.

dz_computed = z_phys_cc(i,j,kmin+m+1) - z_phys_cc(i,j,kmin+m) is the centre-to-centre distance, but dz_level[m] is then used both as the layer thickness in the flux divergence dF/(rho*cp*dz) — where dF is taken across the layer's two interfaces — and as the mass path rho*dz for the tau_model = mass optical depths.

On a stretched grid (erf.terrain_z_levels) those two differ by the stretch ratio, so heating is misallocated between layers and the column optical depth is biased.

z_phys_nd carries the true interface heights and is available at the call site.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. The sweep now takes z_phys_nd: the height of face k over column (i,j) is the mean of its four nodes, the layer thickness is the distance between the layer's two interfaces, and the layer centre is their midpoint. That thickness feeds both the heating divergence and the mass path. TwoStreamRadiation::advance takes z_phys_nd instead of z_phys_cc (both call sites updated). gtest MassModelIsIndependentOfTheStretching runs a constant-density column with a geometric stretching of 1.25 against the uniform layering of the same height and requires the same SW and LW fluxes to 1e-9 of the incident flux; with the centre-to-centre thickness the mass path does not add up to the column and this fails.

Comment thread Source/Radiation/ERF_TwoStreamColumn.H Outdated
}
}
// Top layer: no level above; reuse the thickness of the layer below.
if (nlev > 1) dz_level[nlev - 1] = dz_level[nlev - 2];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium, same finding as above] The topmost layer's thickness is copied from the layer below.

On a stretched grid this is simply wrong — and the top layer is the one the regression checker requires to be the strongest LW cooler, so the check is validating a thickness that doesn't correspond to the grid. Deriving the top layer's thickness from z_phys_nd interfaces fixes both this and the centre-spacing issue above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed together with the one above in 5efc5eb: the top layer's thickness is now the distance between its own two interfaces from z_phys_nd, not a copy of the layer below. The new TwoStream_ColumnHeating_Terrain CTest runs the column-structure checker on a terrain-fitted mesh, so the top-layer cooling check now sees a thickness that belongs to the grid.

Comment thread Source/Radiation/ERF_TwoStreamColumn.H Outdated
// from the uniform spacing otherwise. z_level locates the cloud band
// and evaluates the exponential aerosol profile.
// ------------------------------------------------------------------
amrex::Real dz_level[MAX_RAD_LEVELS];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] ~37 KB of per-thread stack arrays inside a GPU device lambda.

With MAX_RAD_LEVELS = 512, the arrays dz_level, z_level, tau_sw_level, F_dir, A, S, F_lw_up_iface, F_lw_down_iface and tau_lw_level total roughly 37 KB per thread — and vertical_two_stream_sweep is AMREX_FORCE_INLINEd twice into the same lambda (clear + cloudy), so the frame may double.

In a CUDA build every one of those accesses is local memory. Occupancy collapses, and the GPU path will likely be slower than the CPU one, defeating the cost comparison in the PR description. The arrays are also allocated at full capacity regardless of the actual nlev.

Worth either sizing to a realistic maximum, or hoisting the scratch into a device-allocated FArrayBox/Gpu::DeviceVector indexed by column.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. The nine per-column arrays are now a scratch FArrayBox the driver allocates once per box (TwoStreamScratch::NCOMP = 9 components on the box grown by one cell at the top, so nlev + 1 interface entries per column) and the sweep addresses as scratch(i, j, kmin + m, field) through small reference-returning accessors, so the kernel body is unchanged. The clear and cloudy evaluations of a column share the buffer; MAX_RAD_LEVELS and its assert are gone and any nlev works; the unit-test harness allocates the same buffer. The device lambda now holds only scalars.

// No column sweep runs here; that happened in advance_radiation above.
// ***********************************************************************************************
if (solverChoice.radChoice.rad_type == RadType::TwoStream) {
two_stream_rad.advance(lev, iteration, time + dt_lev, dt_lev, "post_dycore",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] This call and the pre-dycore one are gated differently, so setting both solvers half-runs the two-stream model.

The pre-dycore call in ERF_AdvanceRadiation.cpp:113 is an else if on the RRTMGP branch; this post-dycore call is gated only on radChoice.rad_type == RadType::TwoStream.

Set erf.radiation_model (RRTMGP) and erf.radiation_type = TwoStream with seb_prognostic_enable: the column sweep never runs, m_flux_diag[lev] stays at its zero-initialised value, and this call still advances the force-restore surface state every step from those zeros — and, with seb_use_radiation_fluxes, from m_sw_flux_sfc values that are never written.

RadChoice::init_params warns about radiation_type = RRTMGP but says nothing about this combination. Either abort there, or give this call the same rad_type == None condition the pre-dycore branch effectively carries.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. SolverChoice::init_params aborts at start-up when erf.radiation_model and erf.radiation_type = TwoStream are both set (Inputs.rst says so), and the post-dycore call in ERF::Advance now carries the same rad_type == None condition the pre-dycore else-if branch implies, so the two gates match even without the abort.

return;
}

// Guard against duplicate writes using 3-tuple identity:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] The duplicate-write guard is dead code.

RadiationDiagnostics rad_diag(...) is constructed fresh inside every TwoStreamRadiation::advance() call (ERF_TwoStreamRadiation.cpp:266), so m_last_write_step / m_last_write_call_site / m_last_write_time always start at their defaults and this (step, call_site, time) dedup never fires.

m_header_written resets the same way, so the ifstream existence probe runs on every CSV row.

Either make the logger a member of TwoStreamRadiation, or drop the dedup state along with the diag_dedup_tol input it reads.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb: the writer is a member of TwoStreamRadiation (std::unique_ptr m_diag), created on the first advance() call, so m_last_write_* and m_header_written carry across calls and the header probe runs once.

CEXE_headers += ERF_RadiationDiagnostics.H
CEXE_headers += ERF_PrognosticCloudFraction.H
CEXE_headers += ERF_AerosolOpticalDepth.H
CEXE_headers += ERF_SolarGeometry.H

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] ERF_SimplifiedSEB.H is missing from CEXE_headers.

It's included by ERF_TwoStreamRadiation.cpp, and every other header in Source/Radiation/ is listed here, so the GNUmake build doesn't track it for install/dependency listing.

(ERF_RadiationCoupling.H from the intermediate commits is correctly absent from the final tree.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb: ERF_SimplifiedSEB.H is in CEXE_headers.

// radiation is disabled. This regression test ensures the capability check correctly
// recognizes both solverChoice.rad_type (RRTMGP) and solverChoice.radChoice.rad_type
// (TwoStream) as sources of radiation heating.
TEST(Plotfile3DSelection, RadiationHeatingStorageRecognizesTwoStreamAndRRTMGP)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] This regression test doesn't exercise the thing it claims to.

RadiationHeatingStorageRecognizesTwoStreamAndRRTMGP sets caps.radiation_heating_storage = true by hand for both the "RRTMGP" and "TwoStream" cases and then asserts the same predicate twice. It never touches setPlotVariables, which is where the new || radChoice.rad_type == RadType::TwoStream mapping actually lives — so the test would still pass if that line were reverted.

To be a regression test it needs to drive setPlotVariables (or whatever factory populates caps) from a SolverChoice with radChoice.rad_type = TwoStream, and assert that radiation_heating_storage comes back true.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5efc5eb. The predicate is now a free function, erf_plotfile::radiation_heating_storage_available(rad_model, rad_type), which ERF::setPlotVariables uses to populate the capability, and the test drives it with the two selector enums: (None, None) -> false, (RRTMGP, None) -> true, (None, TwoStream) -> true, and the resulting capabilities through plot3d_fixed_variable_available. Dropping the TwoStream half of the mapping now fails the test.

hgopalan and others added 5 commits September 13, 2026 15:28
All ten findings, in the reviewer's order, plus what the same classes of
defect turned up elsewhere in the model.

[high] Domain-mean diagnostics were rank-local. Every sum, count and max
formed over the MFIter (surface and TOA fluxes, column count, max heating
rate, SEB residual, prognostic T_s/q_s) is now reduced with
ParallelDescriptor::Reduce{Real,Long}{Sum,Max} before the means and maxima
are formed. New CTest TwoStream_ColumnHeating_Terrain runs the column over
a Witch-of-Agnesi hill on a terrain-fitted mesh with two boxes of unequal
width; RunTwoStreamRadiation.cmake now also runs the deck on one rank and
requires the diagnostics CSV to match the NRANKS run byte for byte. On the
previous binary the two files differ from the first row; they are identical
now. (A centred hill on a 4x4 domain does not discriminate: the two boxes
are mirror images and rank 0's mean equals the whole.)

[medium] Prognostic cloud fraction discarded the dynamic optical depth: the
cloud-band layers were rebuilt from the clear-sky base. The enhancement is
now scaled in place, tau += (cf - 1) * cloud_tau_per_layer, so the
moisture term survives; gtest DynamicOpticalDepthSurvivesPrognosticCloudFraction.

[medium] cloud_fraction_qc_scale acted as a multiplier (qc * 1e-3 = 1e-6).
It is now the cloud water at which the liquid-water term saturates,
cf += min(1, qc / qc_scale); RadStruct, Inputs.rst, the theory page and
the master deck say so; gtest CloudFractionLiquidWaterTermIsAThreshold.

[medium] Layer thickness on a non-uniform grid was the centre-to-centre
spacing with the top layer copied from below. The sweep now takes the
nodal heights: each interface height over a column is the mean of its
four nodes, the thickness is the distance between the layer's two
interfaces, and the top layer has its own. TwoStreamRadiation::advance
takes z_phys_nd instead of z_phys_cc; gtest
MassModelIsIndependentOfTheStretching (constant-density column, geometric
stretching 1.25, same fluxes as the uniform layering).

[medium] compute_solar_azimuth_angle had sin and cos of the zenith swapped
in cos(azimuth), and its sin(azimuth) term had the wrong sign for the
clockwise-from-north convention it documents (the hour angle is negative
before noon). Both fixed; new gtests in ERF_GTestSolarGeometry.cpp pin due
south at solar noon, east-of-south in the morning, due east at equinox
sunrise, and the pole.

[medium] ~37 kB of per-thread stack arrays in the device lambda. The nine
per-column arrays now live in a scratch FArrayBox the driver allocates per
box (TwoStreamScratch::NCOMP components on the box grown by one at the top)
and the sweep addresses as scratch(i, j, kmin + m, field); MAX_RAD_LEVELS
and its assert are gone, any nlev works, and the clear and cloudy
evaluations of a column share the buffer.

[low] Both radiation selectors set half-ran the two-stream model. SolverChoice
now aborts at start-up when erf.radiation_model and
erf.radiation_type = TwoStream are both set, and the post-dycore call
carries the same rad_type == None condition as the pre-dycore branch.

[low] The diagnostics writer was rebuilt on every call, so its duplicate
guard and header flag never carried over. It is a member of
TwoStreamRadiation, created on the first call.

[low] ERF_SimplifiedSEB.H added to CEXE_headers.

[low] The plotfile-selection gtest set the capability flag by hand. The
predicate is now erf_plotfile::radiation_heating_storage_available(rad_model,
rad_type), used by ERF::setPlotVariables and driven directly by the test
with the two selector enums.

Same classes elsewhere: the 46 silent clamps in RadChoice::init_params now
go through clamp_low / clamp_high / reset_nonpositive, which print the key,
the value and what is used instead. No other rank-local diagnostic, no
other fixed-size device array and no other unused exported function was
found in the radiation sources.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ite one

The column kernel substitutes placeholders (rho = 1, rho*theta of 288 K) for
a non-finite or non-positive density or rho*theta and carries on. The driver
refused only NaN and inf, so a finite but non-positive value could still
reach the kernel and produce plausible heating rates from a placeholder.
advance() now also takes the minimum of rho and rho*theta over the level (a
collective reduction) at the pre-dycore call and aborts, naming the level,
step and minima, when either is not positive. The kernel guards stay for
direct callers (the unit tests) and are marked unreachable from the driver.

ERF's own HSE initialisation rejects a sounding with a negative potential
temperature before radiation runs, so this abort is reachable only through
a corrupted state, which is what it is for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…diation_type

Development selects a radiation model with one input, erf.radiation_model
(RadiationType: None, RRTMGP, Simple). This branch had added a second
selector for the two-stream model, erf.radiation_type (RadType), so a deck
could name two models and the code had to reason about both enums at every
gate. On the maintainer's request the second selector is gone:

- TwoStream is a value of RadiationType; erf.radiation_model = TwoStream
  selects it. RadType and erf.radiation_type no longer exist.
- RadChoice::init_params takes the selection from SolverChoice (a bool) and
  reads the erf.radiation.* sub-options only when the model is selected;
  RadChoice::enabled replaces RadChoice::rad_type, and the start-up abort
  for max_level > 0 names erf.radiation_model.
- SolverChoice::rad_uses_interface() (RRTMGP or Simple) gates everything
  that goes through IRadiation: the rad vector, its Init on new levels, the
  data log, the pre-dycore branch. rad_type == TwoStream gates the
  TwoStreamRadiation calls. Storage, the RhoTheta source and the plotfile
  variables, which every model shares, are gated on rad_type != None, and
  the plotfile capability predicate takes the single enum.
- The both-selectors abort is moot and removed. The constructor's "unknown
  radiation model" check stays and now knows TwoStream.
- Every deck (17 canonical cases, the benchmark suite, the three CTest
  decks, the master deck), README, checker and doc page uses
  erf.radiation_model = "TwoStream"; the Inputs.rst row lists it.

A deck that still sets erf.radiation_type runs with no radiation and the
key shows up in AMReX's unused-parameter report at exit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@hgopalan

Copy link
Copy Markdown
Contributor Author

Following up on the Slack request about the two selectors: the branch now uses erf.radiation_model only. TwoStream is a value of the existing RadiationType enum, RadType and erf.radiation_type are gone, and the erf.radiation.* sub-options are read only when that value is selected. Gates: SolverChoice::rad_uses_interface() (RRTMGP, Simple) for everything that goes through IRadiation, rad_type == RadiationType::TwoStream for the two-stream class, rad_type != None for what every model shares (heating-rate storage, the RhoTheta source, qsrc_sw/qsrc_lw). All 17 canonical decks, the CTest decks, the master deck, the READMEs, the checkers and the docs use erf.radiation_model = "TwoStream"; a deck still carrying the old key runs without radiation and the key appears in AMReX's unused-parameter report. Verified with 553 gtests, the three CTests and the full canonical sweep, on top of today's development merge (commit 3eb26c6, merged with the branch head as e2a67ad).

hgopalan and others added 5 commits September 13, 2026 18:50
RRTMGP has erf.rad_freq_in_steps; the two-stream model has no such input.
That is deliberate: one gray sweep per column is cheap (about a millisecond
per ten thousand cells), so it runs every slow step from the old state and
skipping steps would buy nothing. The Limitations section says so, next to
the single-level and solar-time notes, so the missing input is not read as
an omission.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ee hygiene

Seven changes from the review of PR erf-model#3950, in four groups.

Input documentation

- Inputs.rst documented erf.radiation.solar_zenith_deg, but RadChoice reads
  erf.radiation.solar_zenith. ParmParse ignores the documented spelling
  silently, so a deck that set the zenith angle as documented ran at the 45
  degree default with no error. The row now names the key the code reads.
- Two rows of the two-stream parameter table were misaligned against the grid
  border: a stray "|" inside the tau_model cell and a sw_kabs_dry row one
  character too wide. reStructuredText drops a whole malformed grid table, so
  all 80 documented parameters were rendering as an error block rather than a
  table. Both rows are rewrapped; docutils now parses Inputs.rst with no
  malformed-table error, as it does on development.
- The Doxygen block on diagnose_tau_aerosol_exponential described
  tau = tau_surface * exp(-z/H), but the function returns the profile
  integrated across the layer, (tau_surface/H) * dz * exp(-z/H). The comment
  now matches the code, gains the missing @PARAM dz, and the signature loses
  its leftover "// NEW:" marker.

Diagnostics off by default

erf.radiation.diag_enable now defaults to false, so a two-stream run that did
not ask for diagnostics no longer writes a CSV row and a stdout block on every
step. The per-stream switches keep their true defaults, so setting this one
input turns the whole set on. Every canonical deck that relies on the
diagnostics states it explicitly; the two column-heating CTests already passed
it on the command line. Inputs.rst and the master deck record the new default.

One reduction instead of three per step

The pre-sweep state check ran contains_nan() plus two MultiFab::min() calls
every step per level: three passes over the state and three collectives in the
hot path of a solver that costs about a millisecond per call. It is now a
single fused pass and one ReduceRealMin of two values. A non-finite value maps
to -infinity so the same minimum answers both questions, which keeps the two
abort messages distinct. Note this differs from what the review proposed:
gating the check on erf.radiation.v >= 1 would have left a corrupt state
undetected in a default run, undoing the fix this check was added for, so the
cost is cut without turning the check off.

Tree hygiene

- TwoStream_Benchmark_Suite/cases/{lw_isothermal,sw_clearsky,sw_cloud_layer,
  sw_scattering} were byte-identical copies of four standalone canonical
  cases. benchmark_config.py already named the standalone directories; only
  run_benchmark_suite.py's CASE_PATHS pointed at the copies. It now names the
  standalone cases too and the copies are gone, along with two unreferenced
  duplicates at the cases/ root. run_case() stages a copy into _runs/, so the
  canonical cases are still never written to.
- radiation_timing_comparison.png was a committed 115 KB run output. It is
  removed, added to .gitignore beside the other generated outputs, and the
  case README points at the script that regenerates it.

Verified on this branch: 573 gtests (570 pass, 3 pre-existing skips), all
three radiation CTests, all 17 canonical cases with their checkers, the
5-case benchmark suite reading from the canonical directories, and the
timing fairness checker. The full regression suite is 121/122; the one
failure, Terrain2Lev_BTF_ZSplit, is unrelated and fixed on development by
a8af231, which this branch does not yet contain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings the branch up to development 06ee49b, which it was behind by the
base-state-at-lateral-ghost-heights work (erf-model#4002).

One conflict, in Tests/Unit/Utils/ERF_GTestTerrainFineColumns.cpp. Both sides
carry the identical fix for the nvcc rejection -- the device lambda takes a
local copy of the anonymous-namespace constexpr rather than odr-using it --
and differ only in the comment above it, because the fix landed independently
here and on development in erf-model#4000. Development's wording is kept, so the merge
leaves that file byte-identical to development.

Everything else merged cleanly, including Source/ERF_MakeNewArrays.cpp, which
both sides had touched, and the gold files erf-model#4002 regenerated.

Merging erf-model#4002 also supplies erf.vert_implicit = false to the
Terrain2Lev_BTF_ZSplit deck, which is what that test needs against the
implicit-vertical-diffusion guard from erf-model#3998. It was the one regression
failure on this branch before the merge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The diagnostics are off by default now, so the bullet describing
radiation_diag.dat as written by default and turned off with
diag_csv_enable had it backwards. It names diag_enable as the switch that
turns them on, and keeps the append-on-rerun behaviour explicit, since that
matches ERF's other data logs rather than deviating from them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@asalmgren
asalmgren merged commit 50b55ec into erf-model:development Sep 14, 2026
3 of 47 checks passed
asalmgren pushed a commit that referenced this pull request Sep 14, 2026
Commit bee7f94 on the two-stream branch replaced every entry under
Submodules/ (AMReX, Noah-MP, RRTMGP, WW3, ekat, googletest) with a symbolic
link to /home/almgren/TEMP/ERF/Submodules/<name>, a path on one machine,
and the merge of #3950 carried that onto development. A fresh clone then
has no AMReX or googletest source, so CMake stops at add_subdirectory.

This puts the gitlinks back at the commits recorded just before that
change (7ff1672), which are the pointers development had.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
asalmgren added a commit that referenced this pull request Sep 14, 2026
Restore the Submodules gitlinks that the #3950 merge turned into symlinks
hgopalan added a commit to hgopalan/ERF that referenced this pull request Sep 14, 2026
…ory page

ERF_IBSEBSolar.H carried a copy of the ERF-Radiation solar geometry so the
branch had no dependency on it; with erf-model#3950 merged the five functions are
wrappers around Source/Radiation/ERF_SolarGeometry.H (same Spencer
coefficients, hour angle, zenith, azimuth and Earth-Sun distance). The
sunrise case (10800 steps, sun_mode = solar) is bitwise identical before
and after: every report line and the final face dumps.

The longwave kernel uses the shared Stefan-Boltzmann constant instead of
a local constexpr captured into the device lambda; the reserved
two_stream provider note names erf.radiation_model; a figure caption in
the theory page ran into the next paragraph (Sphinx error).

Checks from the audit notes: the seven touched units compile in single
precision; no MSVC long/M_PI patterns in the diff; the IBSEB_Cube and
tiling decks leave no unused ParmParse keys; IBSEB_Cube split into 8-cell
boxes in x and y agrees with its gold to 2e-10 (a z split is refused by
the implicit vertical diffusion guard of erf-model#3998); the Debug tree of the
merged head builds without warnings and passes 553 unit tests and the
three IBSEB CTests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants