Mariadb support - #56
Merged
Merged
Conversation
bmfmancini
commented
Jul 14, 2026
bmfmancini
commented
Owner
- Huge fix for statistical methods causing forecasts to have accuracy issues (Thanks Codex!!)
- Ability to set embedding model in settings
- Report fixes
- Add ability for admin to set max reports per user
- better prompt sent to LLM for forecast
Added typed forecasting contracts and fit statuses in [contracts.py](/home/sean/Documents/Github/data_forecasting_agent/data_forecaster/backend/forecasting/contracts.py). Added explicit ok, degraded, failed, and not_estimable states. Replaced fabricated zero-error fallbacks with unavailable metrics. Centralized RMSE, MAE, MAPE, WAPE, and MASE calculations. MAPE is now unavailable when actual values contain zero. Added consistent WAPE and MASE calculations to model holdouts. Degraded models can no longer enter model ranking. The pipeline fails explicitly when no model has valid evaluation evidence. Added forecast status information to the API schema.
All four model adapters (ARIMA, SARIMA, Holt-Winters, EWMA) now return ForecastAdapterResult instead of loose dictionaries. The forecasting agent uses typed attribute access throughout and transitional dict lookups are removed. Key changes: - ARIMA/SARIMA preserve with_intercept configuration through refit - Holt-Winters selects additive/multiplicative seasonal on the training split only (fixes test-data leakage in model-form selection) - EWMA estimates alpha by minimizing one-step SSE on the training split instead of using a fixed alpha=0.3; uses centralized metrics instead of the mislabeled perform_rolling_origin_validation - fitted_configuration populated with order, seasonal_order, trend, intercept, seasonal type/period, alpha, and initialization provenance - _calculate_additional_metrics removed (dead code); WAPE/MASE now computed centrally in forecasting/metrics.py - _has_required_metrics operates on ForecastAdapterResult typed objects - Deterministic fallback selection uses lowest RMSE (LLM never ranks) - Updated test_forecasting_metrics.py for typed results Validation: 105 data_forecaster tests passed, 8 metrics tests passed, compileall passed, git diff --check passed.
Deterministic synthetic fixtures (forecasting/fixtures.py) covering: - constant and near-constant series - stationary AR(1) and random walk - additive and multiplicative seasonality - trend without seasonality - zeros and negative values - missing and duplicate timestamps - short seasonal series (< 2 cycles) - isolated anomalies and structural breaks All fixtures use a fixed seed (42) for reproducibility. Failure-state tests (tests/test_forecast_failure_states.py) verify: - Failed/degraded/not_estimable models cannot win ranking - Missing holdout metrics remain None (never zero) - Short-series persistence output is explicitly not_estimable - No fabricated evaluation after fitting exception - All ForecastAdapterResult objects serialize to JSON - Fitted configuration (order, seasonal_order, trend, alpha) survives refit Regression fixture tests (tests/test_forecast_fixtures.py) verify: - Every fixture is deterministic across calls - Each fixture has expected statistical properties - All four adapters survive every fixture without crashing Tests will be validated in the final batch run.
The function perform_rolling_origin_validation performed only a single terminal holdout split, not rolling-origin validation. Renamed to terminal_holdout_validation with accurate docstring. No backward-compatible alias retained (greenfield project). Phase 2 will replace this with a proper expanding-window backtesting service. No code currently imports this function — all four adapters now use centralized metrics directly.
Document all completed R1 tasks: typed contracts, centralized metrics, typed adapter migration, forecasting agent cleanup, obsolete metric logic removal, regression fixtures, and failure-state tests. List remaining R1 work (nullable consumer hardening, test stall diagnosis, final validation).
All report builders, renderers, visualizations, and agents now handle nullable metrics (rmse/mae/mape/wape/mase) safely: - report/models.py: ForecastMetrics and ModelComparisonEntry rmse/mae/mape are now float | None; added format_metric() helper that returns 'not available' for None/NaN/inf values - report/builder.py: _compute_confidence, _compute_health_indicators, _build_forecast_metrics, and _build_model_comparison all guard None metrics before comparison/formatting; model comparison entries no longer mask unavailable metrics as 0.0 - report/dashboard.py: primary_risk guards None mape before comparison - report/renderers/html_renderer.py: model comparison table uses format_metric() instead of bare :.4f formatting - report/renderers/markdown_renderer.py: removed _finite_or_zero (which masked None as 0.0); uses format_metric() for all metric columns - utils/visualization.py: chart title handles None mape/rmse gracefully - agents/report_generation_agent.py: visual strategy MAPE check guards None - agents/model_selection_agent.py: _format_metrics_text handles None/NaN metrics as 'not available' instead of formatting nan Also updated implementation_phases.md with R4 skip note (Phases 6-7).
…tches - arima_model.py/sarima_model.py: Replace getattr(full_model.model, 'trend', None) with trend='c' if with_intercept else 'n' — pmdarima ARIMA objects don't expose a .model attribute - fixtures.py: Fix series names for missing_timestamps and duplicate_timestamps to match their ALL_FIXTURES keys
instead of hard coded
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.