Tutorials on building PyRenew multisignal models#819
Conversation
| The logit-scale representation keeps sampled rates between 0 and 1 after transformation back to the probability scale. | ||
|
|
||
| We use a common generic baseline rate and a logit-scale standard deviation of $0.3$, matching the prior scale used by custom-H-E. | ||
| The prior correlation of $0.5$ says that the two rates may differ, but prior draws that are high for hospital ascertainment also tend to be high for ED ascertainment. |
There was a problem hiding this comment.
Mention that correlated ascertainment is important for identifiability.
There was a problem hiding this comment.
noted.
we could show this by introducing another model which estimates each separately. there's a second tutorial not included in this PR on workflow which builds successive versions of the model.
There was a problem hiding this comment.
another model which estimates each separately
is this not what Pyrenew-h-e is?
There was a problem hiding this comment.
each observation process estimates the ascertainment rate without linking the two -
cf the ascertainment tutorial section: https://cdcgov.github.io/PyRenew/tutorials/ascertainment.html#independent-scalar-ascertainment
| rt_comparison_df = pd.concat(rt_comparison_parts, ignore_index=True) | ||
| ``` | ||
|
|
||
| ```{python} |
There was a problem hiding this comment.
This figure is quite cramped. Override the default size?
There was a problem hiding this comment.
fixed this and several other similar plots. the tutorial theme fonts seem too small - I made pretty much the same set of fixes to all the plots:
+ p9.theme(
text=p9.element_text(size=16),
plot_title=p9.element_text(size=18, weight="bold"),
axis_title=p9.element_text(size=15),
axis_text=p9.element_text(size=13),
strip_text=p9.element_text(size=14),
needs more investigation.
according to the AI, mkdocs is generating the png and which is then restricted to column width, but if the theme specifies the figure size, they should all be the same ???
| ```{python} | ||
| #| label: synthetic-recovery-check | ||
|
|
||
| rhat = pyrenew_parameter_summary["r_hat"].astype(float) | ||
| ess = pyrenew_parameter_summary["ess_bulk"].astype(float) | ||
| print(f"Max R-hat: {rhat.max():.3f}") | ||
| print(f"Min bulk ESS: {ess.min():.0f}") | ||
|
|
||
| latent_inf = idata_trimmed.posterior["latent_infections"] | ||
| inf_q05 = latent_inf.quantile(0.05, dim=["chain", "draw"]).values | ||
| inf_q50 = latent_inf.quantile(0.50, dim=["chain", "draw"]).values | ||
| inf_q95 = latent_inf.quantile(0.95, dim=["chain", "draw"]).values | ||
|
|
||
| true_infections = daily_infections["true_infections"].to_numpy() | ||
| n_compare = min(len(true_infections), len(inf_q05)) | ||
| inf_covered = (true_infections[:n_compare] >= inf_q05[:n_compare]) & ( | ||
| true_infections[:n_compare] <= inf_q95[:n_compare] | ||
| ) | ||
| print(f"90% interval coverage for true infections: {np.mean(inf_covered):.1%}") | ||
| ``` |
There was a problem hiding this comment.
This table needs more commentary. I'm not sure what it's about.
| ) | ||
| + theme_tutorial | ||
| ) | ||
| ``` |
There was a problem hiding this comment.
I think it would make more sense here to show the posterior predictive for E and H, not just a simple MA.
| ) | ||
| + theme_tutorial | ||
| ) | ||
| ``` |
There was a problem hiding this comment.
What do we learn from this plot that we didn't learn from the previous one? Why bin into two week estimates?
There was a problem hiding this comment.
After further reading, I see that this introduces the idea of relative widths, which are an important comparison point between the models later in the document.
| ``` | ||
|
|
||
| Rt is useful as a companion to latent infections because it focuses on epidemic shape rather than absolute infection scale. | ||
| The plot below compares Rt across all three models against the synthetic data-generating Rt trajectory. |
There was a problem hiding this comment.
The intro only mentions 2 models. Perhaps I was not reading carefully enough. Can we state what the three models are at the very beginning?
There was a problem hiding this comment.
I think the names could be a bit clearer, as well.
Something like
pyrenew-H-E -> independent-ascertainment
linked-pyrenew-H-E -> correlated-ascertainment
custom-H-E -> ratio-ascertainment
?
| .reset_index() | ||
| ) | ||
| latent_width_summary | ||
| ``` |
There was a problem hiding this comment.
The above table needs much more detail, I think. Mean relative width of what?
|
|
||
| ```{python} | ||
| #| label: linked-latent-infections | ||
| #| fig-cap: Posterior latent infections under the two PyRenew parameterizations |
There was a problem hiding this comment.
I just realized these figure captions aren't showing in the html (I have just been reviewing the html, not the qmd). We should figure out how to display them.
There was a problem hiding this comment.
another glitch in the toolchain. made all captions into italicized paragraph directly under the plot. not elegant, but it works.
There was a problem hiding this comment.
See #826. Let's decide how to address this repo-wide before merging this PR.
| ) | ||
| + theme_tutorial | ||
| ) | ||
| ``` |
There was a problem hiding this comment.
yikes. They all fail to recover the IEDR and IHR. Can we plot the priors too? Perhaps in a facet above?
| ) | ||
| + theme_tutorial | ||
| ) | ||
| ``` |
There was a problem hiding this comment.
Would love to see priors here too.
There was a problem hiding this comment.
Really nice work @cdc-mitzimorris! I have only reviewed the new tutorial.
Co-authored-by: Damon Bayer <damonmbayer@gmail.com>
for more information, see https://pre-commit.ci
…into mem_tutorial_h_e_approx
for more information, see https://pre-commit.ci
…into mem_tutorial_h_e_approx
3 participants
This PR adds tutorial "Multi-signal Model for Weekly Hospital and Daily ED Data" which compares a PyRenew
MultiSignalModelto the production H+E model fitting both on synthetic data.