Implement multiprocessing for the CVM model

[yueshuaing]

Proposed changes

This PR adds multiprocessing to the existing commercial vehicle model (CVM). The main changes include: adding a multiprocessing configuration to the settings, registering CVM tables with the RNG channel, table checkpointing, and using reproducible random draws so that multiprocess output is correct and consistent.

Impact

All output tables now match exactly between the old single-process and the new multiprocess runs, except cv_trips. This is because the old single-process implementation used global random draws in route_stops that weren't fully reproducible. The difference versus the old baseline is small, about 0.14% of total trips, with differences in OD pairs with 0-10 trips. With this PR, single-process and multiprocess results are consistent with each other.

Multiple runs were tested on a 512 GB memory, 64 process machine. Performance is as follows:

Processes	Runtime
1	2h 38m
12	40 min
20	38min
30	38min

Types of changes

What types of changes does your code introduce to ABM?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

How has this been tested?

Please describe the tests that you ran to verify your changes.

• Run single-process, save output as baseline.
• Run multiprocess with the updated multiprocess_steps config and confirm identical output

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

[JiaXu1024]

@yueshuaing Just saw this PR and assigned myself as reviewer.
I'm wondering if this CVM multi-processing update is ready for testing? Since this PR is draft and not publish yet.

[yueshuaing]

@yueshuaing Just saw this PR and assigned myself as reviewer. I'm wondering if this CVM multi-processing update is ready for testing? Since this PR is draft and not publish yet.

Hi Susan, thanks for taking care of the reviewer assignment. I changed the PR ready for review. Please test run it and let me know if you run into any issues.

[yueshuaing]

@i-am-sijia , the PR is ready for your review.

[JiaXu1024]

I implemented the multi‑process configuration in the SANDAG CVM model and verified that the runtime dropped from 3 hours to 35 minutes—an enormous improvement.

I also compared the model output (final_trips.csv) from the single‑process and multi‑process versions of CVM.

Files Compared:

• File 1 (Single process CVM): T:\STORAGE-63T\abm3_runs\2022_JavaRemoval\output\CVM\final_trips.csv
• File 2 (Multi-process CVM): T:\STORAGE-63T\abm3_runs\2022_CVM_mp\CVM\final_trips.csv

I understood that the differences are expected and intentional. The multi-process CVM doesn't replicate the old single-process results because the old version had non-reproducible random number generation.

Below is the detailed statistical analysis:

1. Trip Volume Differences

Metric	File 1 (Single process CVM)	File 2 (Multi-process CVM)	Difference
Total Trips	1,173,815	1,176,094	+2,279 (+0.19%)
Unique Routes	171,168	171,184	+16 (+0.01%)
Routes Only in File 1	3	-	-3
Routes Only in File 2	-	19	+19
Common Routes	171,165	171,165	-
Common Routes with Different Trip Counts	148,286	148,286	86.6% of common routes

2. Vehicle Type Distribution

Vehicle Type	File 1 Count	File 1 %	File 2 Count	File 2 %	Difference
DRIVEALONE	599,966	51.11%	601,060	51.11%	+1,094
LHDT	311,945	26.58%	311,605	26.49%	-340
MHDT	133,537	11.38%	134,138	11.41%	+601
HHDT	128,367	10.94%	129,291	10.99%	+924

Key Insight: DRIVEALONE and HHDT vehicles increased while LHDT decreased slightly. The distribution percentages remained remarkably stable.

3. Time of Day (TOD) Distribution

Time Period	File 1 Count	File 1 %	File 2 Count	File 2 %	Difference
Early AM (EA)	25,102	2.14%	25,095	2.13%	-7
AM	196,350	16.73%	196,637	16.72%	+287
Midday (MD)	751,838	64.05%	752,734	64.00%	+896
PM	141,355	12.04%	141,487	12.03%	+132
Evening (EV)	59,170	5.04%	60,141	5.11%	+971

Key Insight: Most additional trips occurred during Midday (MD) and Evening (EV) periods.

4. Trip Purpose Distribution

Origin Purpose

Purpose	File 1 Count	File 1 %	File 2 Count	File 2 %	Difference
service	401,309	34.19%	401,971	34.18%	+662
goods_delivery	271,840	23.16%	273,517	23.26%	+1,677
maintenance	246,383	20.99%	245,890	20.91%	-493
originate	171,168	14.58%	171,184	14.56%	+16
base	45,751	3.90%	45,854	3.90%	+103
goods_pickup	28,739	2.45%	29,102	2.47%	+363
home	8,625	0.73%	8,576	0.73%	-49

Key Insight: Significant increase in goods_delivery trips (+1,677), partially offset by reduction in maintenance trips (-493).

Destination Type

Type	File 1 Count	File 1 %	File 2 Count	File 2 %	Difference
commercial	587,520	50.05%	589,423	50.12%	+1,903
residential	396,462	33.78%	396,328	33.70%	-134
base	140,576	11.98%	140,691	11.96%	+115
warehouse	45,429	3.87%	45,689	3.88%	+260
intermodal	3,828	0.33%	3,963	0.34%	+135

5. Performance Metrics Summary - System-Wide Totals

Metric	File 1	File 2	Difference	Impact
Total VMT	8,721,265 mi	8,733,018 mi	+11,752 mi	+0.13%
Total Travel Time	265,041 hrs	265,013 hrs	-28 hrs	-0.01%
Total Operating Cost	$533,715,582	$534,895,151	+$1,179,569	+0.22%

Critical Finding: Despite slightly less total travel time, File 2 costs $1.18 million more due to additional trips and different route patterns.

cc @bhargavasana @johnpgliebe @i-am-sijia

[i-am-sijia]

Thank you @JiaXu1024 for the comprehensive comparisons! Quick questions: Do we have a new single process run to compare with the multi process run to confirm they have the same results? I think they should.

Are we using the same inputs including skims as the old single process run? Can some of the difference be because of the differences in inputs?

[JiaXu1024]

@yueshuaing @i-am-sijia Just wanna confirm if this section of code change fixes the issue of non-reproducible random number generation?

[yueshuaing]

Hi Susan, yes, we removed the previous global random draw np.random.seed(seed=42), as in multiprocessing this code runs in each of the process, seed to the same draw. And the previous scipy.stats.beta.rvs() function draws a batch of len(df) random numbers, assigns them to the row based on its position, so reproducibility now depends on df's row order being identical every time which is not guaranteed in multiprocessing. So we switched to the ActivitySim's reproducible method state.get_rn_generator().random_for_df(df) from SimpleChannel

[JiaXu1024]

Hi @yueshuaing , thanks for the clarification. So the previous global random‑draw settings in the single‑process setup didn’t have any issues, right?

[yueshuaing]

Yes, the code wasn't broken in the single-process cause the single-process processes the same data in the same order every time. The issue only appears when introducing multiprocessing, which became not reproducible between single vs multi process.