2026-07-23 Engineering Team

[joecastiglione]

Agenda

• Open Pull Request Follow-ups
  • Jeff review / check-in next week's Engineering meeting
• ActivitySim Application & Analysis Guide (CS)
  • Status
    • Telecommuting change
    • Network change
• Explicit Telecommute Implementation (WSP)
  • Follow up from June 30 presentation
• EET Implementation and Improvement (Outer Loop)
  • Status: complete?
    • PR?
    • Documentation updates?
• EET Investigation of Inconsistencies (Outer Loop)
  • Status: complete?
  • Presentation vs Technical Report
• Model Calibration – Partial Automation (RSG)
  • PSRC + Transport for NSW testing status
  • FineTune vs. ActivitySim comparison table
• EET Updated Random Number Generation (RSG)
  • Status: complete?
• EET Output Analysis (RSG)
  • Status: complete?
• Telecommute Frequency Model Update and Tests
  • Presentation?
  • MWCOG testing coordination

Notes

Meeting Purpose

• Joe explained he's substituting for Jeff, who can't make the later meeting times for the next few months.
• Format shifted to a project-management review of Phase 11C task status, aiming to close out remaining work before pivoting to Phase 12 planning in September.

Task-by-Task Status

ActivitySim Application and Analysis Guide (CS)

• Joe Flood has largely finished responding to comments on the land use change and still needs to add analysis metrics.
• Has not yet shared the telecommute and network change write-ups.
• Action: Joe Flood to continue land use analysis metrics; next check-in scheduled for Aug 25.

Explicit Telecommute Model (WSP)

• Completed review of ~200 worker-day records with conflicting telework/away-from-home hours.
• Ran full-scale SANDAG model with newly estimated telecommute models; results look reasonable, with one model showing better fit than the other (hypotheses being explored).
• Incorporating industry/occupation data into finalization.
• Action: Final presentation targeted for Aug 11.
• Lower priority: investigate telecommute-frequency/MGRA "same zone" data anomaly flagged by David and Joel — Sijia will address only if time remains after primary tasks.

Telecommute Frequency Model (RSG)

• Parallel TransLink-funded work produced a much larger, correctly-signed mode-choice logsum coefficient than the SANDAG estimation, raising questions about regional/temporal/survey differences.
• Discussed options: share TransLink's specification with MWCOG for comparison testing, or have MWCOG apply the logsum coefficient to existing models.
• Action: David/Joel to send an email/short comparison summary of SANDAG vs. TransLink specifications and recommendations within 2–3 weeks (~week of Aug 11).
• Action: Coordinate with MWCOG to run sensitivity test on current spec; targeting **September * (~5 weeks out) for a consortium check-in including MWCOG's test results.
• David/Joel will also prepare documentation/presentation summarizing the telecommute-frequency investigation walkthrough (follow-up to yesterday's meeting).

EET (Explicit Error Term) Work (Outer Loop)

• Pull request reviewed by David last week; only minor changes remain for Jan to incorporate.
• Documentation reviewed by David — thorough, no major comments.
• Considered essentially complete; some inconsistency-investigation findings and a Monte Carlo vs. explicit-error-term MAZ sampling prototype will be opened as a separate issue (PR?) for future consideration, not funded further now.
• Action: Jan to incorporate minor PR changes and submit invoice once complete.
• Action: David to run one additional test (full scenario without shadow pricing) to confirm small numerical changes hold; tentatively scheduled for Aug 4, budget-permitting.
• Poisson sampling method: confirmed tested successfully, folded into EET deliverable; Jan noted priority sampling explored as an alternative with similar noise properties — flagged as a topic for future discussion, not immediate action.

Model Calibration / Partial Automation (RSG)

• Joe C asked about PSRC and TfNSW possibly testing new RSG calibration tool
• Amir: TfNSW is mid-calibration for their ABM release (due in about a week) and prioritizing their internal FineTune tool over RSG's log-ratio method, due to its support for more optimizers and better visualization/dashboards.
• TfNSW not expected to test the RSG (log-ratio) method given their current focus.
• Discussion on open-sourcing FineTune: Amir noted TfNSW leadership (Shaun) is pursuing this internally with legal, but no timeline yet.
• Amir proposed distinguishing two products: (1) RSG's embedded market-share calibration (fast, good for policy/scenario testing) vs. (2) a more comprehensive optimizer library needed for complex calibrations with competing targets (e.g., tour mode choice, distance model soft-capping).
• David pushed back that the log-ratio method has successfully calibrated all consortium models to date, including tour mode choice.
• Action: No immediate task; to be revisited as part of Phase 12 prioritization — a joint Product/Community and Engineering discussion on the future vision for calibration tooling.

Random Number Generation Task

• Considered closed/complete at investigation and implementation level, even though results didn't yield an integrated change.
• Action: David to open a documentation branch/issue (similar to EET's prior approach) capturing findings and code for potential future pickup; no further check-ins needed on calendar.

Coordination Discussion

• Joe raised concern about redundant/parallel work streams (e.g., consortium-funded calibration work vs. TfNSW FineTune tool development; consortium / MWCOG telecommute estimation work vs. TransLink-funded telecommute estimation work) and asked whether better coordination is needed.
• General agreement that coordination is difficult due to differing agency timelines, but should be pursued where possible; some past examples (e.g., park-and-ride feature) worked out well without a priori coordination.

Broader Discussion: Investigation Rigor & Testing

• Joe raised a general question about how much effort/money to invest in chasing down unexplained anomalies (e.g., shadow pricing discrepancies in SANDAG's model, ongoing memory-release issue).
• David suggested many such issues could be addressed via consortium support funds.
• Sijia raised whether the consortium should more actively "own" validation of the SANDAG example model, since it currently relies on SANDAG staff to catch bugs.
• Joe expressed that he thought it would be impractical for the consortium to fully own model validation (resource constraints, no dedicated survey/count data, no local knowledge), but agreed the consortium should be more rigorous in verifying that funded work is properly validated (cited a specific gap: he and Bhargav review found no clear validation of shadow-pricing convergence beyond "zones not reopening").
• David suggested this rigor/support-fund model could be a value proposition to attract new agencies to join the consortium.

Summary of Key Dates

Date	Item
Aug 4 (tentative)	David: re-run EET test without shadow pricing
Aug 11	Sijia: final explicit telecommute presentation
~Aug 11-14	David/Joel: SANDAG vs. TransLink telecommute-frequency spec summary/email
Aug 25	Joe Flood: Application & Analysis Guide check-in
Sept 1	Telecommute frequency: consortium check-in w/ MWCOG test results
Ongoing/Phase 12	Calibration tooling vision (fine-tune vs. RSG) discussion