diff --git a/CHANGELOG.md b/CHANGELOG.md index 70fa0aa5..4e17a329 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- **`ChaisemartinDHaultfoeuille.predict_het` × `placebo`: R-parity on both global and per-path surfaces.** R-verified — `did_multiplegt_dyn(predict_het, placebo)` emits heterogeneity OLS results on backward (placebo) horizons via R's `DIDmultiplegtDYN:::did_multiplegt_main` placebo block (`effect = matrix(-i, ...)` rbind site); the same block runs per-by_level under `did_multiplegt_dyn(by_path, predict_het, placebo)`, so both global `res$results$predict_het` and per-by_level `res$by_level_i$results$predict_het` slots emit backward rows. R's predict_het syntax with `placebo > 0` requires the `c(-1)` sentinel in the horizon vector to trigger "compute heterogeneity for ALL forward (1..effects) AND ALL placebo (1..placebo) positions" — passing positive-only horizons errors with "specified numbers in predict_het that exceed the number of placebos". Python mirrors via `_compute_heterogeneity_test(..., placebo=L_max)` (set automatically from `self.placebo` at both global and per-path call sites in `fit()`) — the function iterates forward (1..L_max) and backward (-1..-L_max) horizons in a single loop with an explicit `out_idx < 0` eligibility guard for backward horizons whose `F_g` is too small (would otherwise silently misread `N_mat` via numpy negative indexing). `results.heterogeneity_effects` uses negative-int keys for backward horizons; `path_heterogeneity_effects` does the same per path. Placebo rows in `to_dataframe(level="by_path")` have non-NaN `het_*` columns when `placebo=True` and `heterogeneity=` are both set. **Survey gate (warn + skip):** `survey_design + placebo + heterogeneity` emits a `UserWarning` at fit-time and falls back to forward-horizon-only heterogeneity on both surfaces — the Binder TSL cell-period allocator's REGISTRY justification is tied to **post-period** attribution; backward-horizon attribution puts ψ_g mass on a pre-period cell, a separate library-extension claim that needs its own derivation. Forward-horizon `predict_het + survey_design` continues to work unchanged on both global and per-path surfaces. The function-level `_compute_heterogeneity_test` keeps a per-iteration `NotImplementedError` backstop for direct callers that bypass fit(). Pre-period allocator derivation deferred to a follow-up methodology PR (tracked in TODO.md). R parity confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityHeterogeneityWithPlacebo` (scenario 23, `multi_path_reversible_predict_het_with_placebo_global`, `placebo=2, effects=3, no by_path`) and `::TestDCDHDynRParityByPathHeterogeneityWithPlacebo` (scenario 22, same DGP plus `by_path=3`); pinned at `BETA_RTOL=1e-6` / `SE_RTOL=1e-5` for `beta` / `se` / `t_stat` / `n_obs` and `INFERENCE_RTOL=1e-4` for `p_value` / `conf_int` across 3 paths × (3 forward + 2 placebo) = 15 horizons + 1 global × 5 horizons. Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPredictHetPlacebo` (placebo het column population, survey-gate warn+skip behavior, forward+survey anti-regression, `out_idx<0` eligibility guard, single-path telescope `path_heterogeneity_effects[(only_path,)] == heterogeneity_effects` bit-exactly, summary rendering, direct-call `NotImplementedError` backstop). Closes TODO #422. + +### Changed +- **`ChaisemartinDHaultfoeuille.predict_het` inference: t-distribution df threading (closes TODO pilot-412).** `_compute_heterogeneity_test` now passes `df = n_obs - n_params` to `safe_inference` on the non-survey OLS path, matching R `did_multiplegt_dyn(predict_het=...)`'s t-distribution inference (`DIDmultiplegtDYN:::did_multiplegt_main` `t_stat <- qt(0.975, df.residual(model))` site). Pre-PR Python used `df=None` (normal Z critical), producing 0.1-2% rtol gaps on `p_value` and `conf_int` vs R. Parity tolerance tightened on the existing forward-horizon scenarios (`multi_path_reversible_predict_het`, `multi_path_reversible_by_path_predict_het`) from "unpinned" to `INFERENCE_RTOL=1e-4` on `p_value` and `conf_int`; `beta` / `se` / `t_stat` continue at `BETA_RTOL=1e-6` / `SE_RTOL=1e-5`. **Rank-deficient caveat:** `n_params = design.shape[1]` is the pre-drop column count; under near-rank-deficient designs that `solve_ols` retains rather than NaN-out, the actual rank may be lower than `n_params` (R's `df.residual` uses post-drop rank). Fully rank-deficient designs are NaN-filled by the existing short-circuit; the gap only affects near-rank-deficient edge cases (tracked as a Low TODO follow-up). The Z-vs-t REGISTRY deviation note is replaced with an "R parity (post-2026-05-15 df threading)" positive-claim note. + ## [3.3.3] - 2026-05-15 ### Added diff --git a/TODO.md b/TODO.md index 7572e088..8dcdca76 100644 --- a/TODO.md +++ b/TODO.md @@ -60,8 +60,8 @@ Deferred items from PR reviews that were not addressed before merge. | dCDH: Survey cell-period allocator's post-period attribution is a library convention, not derived from the observation-level survey linearization. MC coverage is empirically close to nominal on the test DGP; a formal derivation (or a covariance-aware two-cell alternative) is deferred. Documented in REGISTRY.md survey IF expansion Note. | `chaisemartin_dhaultfoeuille.py`, `docs/methodology/REGISTRY.md` | PR 2 | Medium | | dCDH: Parity test SE/CI assertions only cover pure-direction scenarios; mixed-direction SE comparison is structurally apples-to-oranges (cell-count vs obs-count weighting). | `test_chaisemartin_dhaultfoeuille_parity.py` | #294 | Low | | dCDH by_path: negative-baseline path regression (e.g. `(-1, 0, 0, 0)`) is not yet exercised. The existing negative-D test (`test_negative_integer_D_supported`) only covers paths with negative values in non-baseline positions like `(0, -1, -1, -1)`, which does not trigger the R `substr(path, 1, 1)` bug regime (the bug needs a multi-character baseline). Add a switcher fixture with `D_{g,1} = -1` and assert the resulting path tuple key. | `tests/test_chaisemartin_dhaultfoeuille.py` | #419 | Low | -| dCDH by_path: per-path placebo heterogeneity (`predict_het` rows for negative horizons) is currently NaN-filled in `to_dataframe(level="by_path")` `het_*` columns and unpopulated in `path_heterogeneity_effects`. R `did_multiplegt_dyn(..., by_path, predict_het)` forwards `predict_het` into each per-path `did_multiplegt_main` call alongside `placebo`, so R likely emits placebo het rows we do not yet mirror. Validate R's actual placebo predict_het output, then either implement parity or document the deviation explicitly. | `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/chaisemartin_dhaultfoeuille_results.py`, `tests/test_chaisemartin_dhaultfoeuille_parity.py` | #422 | Medium | -| dCDH heterogeneity: `_compute_heterogeneity_test` passes `df=None` to `safe_inference`, so Python uses the normal Z critical value (~1.96) for `t_stat`-derived `p_value` and `conf_int`. R `did_multiplegt_dyn(..., predict_het)` uses the t-distribution with df = n - k from the OLS regression, producing ~0.1-2% rtol gaps on CIs and p-values vs Python. Documented as a deviation in the heterogeneity R-parity Note; parity tests pin only `beta`, `se`, `t_stat`, and `n_obs`. Either thread the OLS df into `safe_inference` to match R, or formalize a separate inference-tolerance constant for the heterogeneity surface. | `diff_diff/chaisemartin_dhaultfoeuille.py`, `tests/test_chaisemartin_dhaultfoeuille_parity.py` | pilot-412 | Low | +| dCDH by_path: survey-aware backward-horizon (`placebo + predict_het + survey_design`) raises `NotImplementedError` because the Binder TSL cell-period allocator's REGISTRY justification is tied to post-period attribution. Backward horizons would put ψ_g mass on a pre-period cell. Deriving the pre-period cell allocator (or adding a covariance-aware two-cell alternative) is deferred to a follow-up methodology PR. | `diff_diff/chaisemartin_dhaultfoeuille.py`, `docs/methodology/REGISTRY.md` | follow-up | Medium | +| dCDH heterogeneity: rank-deficient designs use `df = n_obs - n_params` (pre-drop column count) in the t-distribution inference. R's `lm(predict_het=...)` uses `df.residual = n - rank(design)` post-drop. Fully rank-deficient designs are NaN-filled by the rank-deficient short-circuit at `_compute_heterogeneity_test:5141-5150`, so the gap only affects near-rank-deficient designs where `solve_ols` retains the design. Thread actual rank from `solve_ols` to close the gap. | `diff_diff/chaisemartin_dhaultfoeuille.py` | follow-up | Low | | CallawaySantAnna: consider materializing NaN entries for non-estimable (g,t) cells in group_time_effects dict (currently omitted with consolidated warning); would require updating downstream consumers (event study, balance_e, aggregation) | `staggered.py` | #256 | Low | | ImputationDiD dense `(A0'A0).toarray()` scales O((U+T+K)^2), OOM risk on large panels | `imputation.py` | #141 | Medium (deferred — only triggers when sparse solver fails) | | Multi-absorb weighted demeaning needs iterative alternating projections for N > 1 absorbed FE with survey weights; unweighted multi-absorb also uses single-pass (pre-existing, exact only for balanced panels) | `estimators.py` | #218 | Medium | diff --git a/benchmarks/R/generate_dcdh_dynr_test_values.R b/benchmarks/R/generate_dcdh_dynr_test_values.R index a5caa5ee..9c57cadc 100644 --- a/benchmarks/R/generate_dcdh_dynr_test_values.R +++ b/benchmarks/R/generate_dcdh_dynr_test_values.R @@ -622,12 +622,30 @@ extract_dcdh_by_path <- function(res, n_effects, n_placebos = 0) { # res$results$predict_het, a data.frame with columns # {effect, covariate, Estimate, SE, t, LB, UB, N, pF}. Estimate is the # WLS coefficient on the heterogeneity covariate. +# +# `n_effects` retained for backward-compat with scenarios 20/21 callers +# but unused: we iterate ALL rows in ph$effect and partition by sign so +# placebo (negative-effect) rows are captured separately. Scenario 22 +# probes whether R emits negative-effect rows when called with +# `predict_het + placebo`; resolves TODO #422. extract_dcdh_predict_het <- function(res, n_effects) { ph <- res$results$predict_het - horizons <- list() - if (is.null(ph) || nrow(ph) == 0) return(list(predict_het = horizons)) - for (h in seq_len(min(n_effects, nrow(ph)))) { - horizons[[as.character(ph$effect[h])]] <- list( + # `structure(list(), names = character(0))` produces a named list with + # zero entries; jsonlite serializes it as `{}` (object) rather than + # `[]` (array). Plain `list()` would serialize as `[]`, which gives + # the JSON contract a type-unstable shape (object when populated, array + # when empty). Type stability matters for generic consumers — see + # `tests/test_chaisemartin_dhaultfoeuille_parity.py::_as_dict` for the + # defensive Python-side coercion that backstops this. + forward_horizons <- structure(list(), names = character(0)) + placebo_horizons <- structure(list(), names = character(0)) + if (is.null(ph) || nrow(ph) == 0) { + return(list(predict_het = forward_horizons, + placebo_predict_het = placebo_horizons)) + } + for (h in seq_len(nrow(ph))) { + effect_val <- as.numeric(ph$effect[h]) + entry <- list( beta = as.numeric(ph$Estimate[h]), se = as.numeric(ph$SE[h]), t = as.numeric(ph$t[h]), @@ -636,8 +654,15 @@ extract_dcdh_predict_het <- function(res, n_effects) { n_obs = as.numeric(ph$N[h]), p_value = as.numeric(ph$pF[h]) ) + if (effect_val > 0) { + forward_horizons[[as.character(effect_val)]] <- entry + } else if (effect_val < 0) { + placebo_horizons[[as.character(effect_val)]] <- entry + } + # effect_val == 0: skip (not a valid event-study horizon). } - list(predict_het = horizons) + list(predict_het = forward_horizons, + placebo_predict_het = placebo_horizons) } # Helper: extract per-path predict_het results. Under by_path=k + @@ -650,10 +675,16 @@ extract_dcdh_by_path_predict_het <- function(res, n_effects) { for (i in seq_along(by_levels)) { slot <- res[[paste0("by_level_", i)]] ph <- slot$results$predict_het - horizons <- list() + # See extract_dcdh_predict_het comment for the named-list rationale. + forward_horizons <- structure(list(), names = character(0)) + placebo_horizons <- structure(list(), names = character(0)) if (!is.null(ph) && nrow(ph) > 0) { - for (h in seq_len(min(n_effects, nrow(ph)))) { - horizons[[as.character(ph$effect[h])]] <- list( + # Iterate ALL rows; partition by sign so placebo (negative-effect) + # rows are captured under `placebo_horizons`. Scenario 22 probes + # whether R emits negative-effect rows on the per-path surface. + for (h in seq_len(nrow(ph))) { + effect_val <- as.numeric(ph$effect[h]) + entry <- list( beta = as.numeric(ph$Estimate[h]), se = as.numeric(ph$SE[h]), t = as.numeric(ph$t[h]), @@ -662,12 +693,18 @@ extract_dcdh_by_path_predict_het <- function(res, n_effects) { n_obs = as.numeric(ph$N[h]), p_value = as.numeric(ph$pF[h]) ) + if (effect_val > 0) { + forward_horizons[[as.character(effect_val)]] <- entry + } else if (effect_val < 0) { + placebo_horizons[[as.character(effect_val)]] <- entry + } } } out[[i]] <- list( path = by_levels[i], frequency_rank = i, - horizons = horizons + horizons = forward_horizons, + placebo_horizons = placebo_horizons ) } list(by_path_predict_het = out) @@ -1201,6 +1238,87 @@ cat(" Scenarios 20/21: multi_path_reversible_predict_het + by_path version\n") dont_drop_larger_lower = TRUE), results = extract_dcdh_by_path_predict_het(res21, n_effects = 3) ) + + # Scenario 23: GLOBAL predict_het + placebo (no by_path). Mirrors + # scenario 22's syntax minus by_path so we have a parity anchor for + # the GLOBAL `results.heterogeneity_effects` surface emitting both + # forward and backward (placebo) horizons. Resolves codex R1 P1 #2: + # the Phase 1A change extended the global heterogeneity loop to + # cover backward horizons, so a global-surface parity test was + # required to lock that contract independently of the per-path + # dispatcher. Same `c(-1)` sentinel as scenario 22 (computes ALL + # forward + ALL placebo positions); reuses `d20` for DGP parity. + res23 <- did_multiplegt_dyn( + df = d20, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, placebo = 2, + dont_drop_larger_lower = TRUE, + predict_het = list("het_x", c(-1)), + ci_level = 95, graph_off = TRUE + ) + scenarios$multi_path_reversible_predict_het_with_placebo_global <- list( + data = list( + group = as.numeric(d20$group), + period = as.numeric(d20$period), + treatment = as.numeric(d20$treatment), + outcome = as.numeric(d20$outcome), + het_x = as.numeric(d20$het_x) + ), + params = list(pattern = "multi_path_reversible_predict_het_with_placebo_global", + n_switchers = n_switchers20, n_controls = n_controls20, + n_groups = n_groups20, n_periods = n_periods20, + seed = 120L, effects = 3, placebo = 2, + predict_het_var = "het_x", + predict_het_horizons = c(-1), + ci_level = 95, + dont_drop_larger_lower = TRUE), + results = extract_dcdh_predict_het(res23, n_effects = 3) + ) + + # Scenario 22: by_path + predict_het + placebo (probes TODO #422). Reuses + # d20 from scenarios 20/21 for DGP parity. Tests whether R's + # did_multiplegt_dyn(by_path=k, predict_het, placebo=N) per-by_level + # dispatcher emits predict_het rows on backward (placebo) horizons. + # + # R's predict_het syntax with `placebo > 0` (per did_multiplegt_main + # source `DIDmultiplegtDYN:::did_multiplegt_main` lines 1907 / 2030): + # the SAME horizon vector is used for BOTH forward effects AND placebo + # positions. Passing `c(1, 2, 3)` with `placebo=2` errors because + # `max(c(1, 2, 3)) > placebo=2`. The `c(-1)` sentinel triggers "compute + # heterogeneity for ALL forward (1..effects) AND ALL placebo + # (1..placebo) positions" by replacing `het_effects` with `1:l_XX` in + # the forward block and `1:l_placebo_XX` in the placebo block. Forward + # rows are emitted with positive `effect` values (1, 2, 3); placebo + # rows with NEGATIVE values (-1, -2) per `effect = matrix(-i, ...)` at + # the placebo block's rbind site. + # + # The extended extract_dcdh_by_path_predict_het partitions the per-path + # predict_het table by `effect` sign: forward into `horizons`, placebo + # into `placebo_horizons`. + res22 <- did_multiplegt_dyn( + df = d20, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, placebo = 2, by_path = 3, + dont_drop_larger_lower = TRUE, + predict_het = list("het_x", c(-1)), + ci_level = 95, graph_off = TRUE + ) + scenarios$multi_path_reversible_predict_het_with_placebo <- list( + data = list( + group = as.numeric(d20$group), + period = as.numeric(d20$period), + treatment = as.numeric(d20$treatment), + outcome = as.numeric(d20$outcome), + het_x = as.numeric(d20$het_x) + ), + params = list(pattern = "multi_path_reversible_predict_het_with_placebo", + n_switchers = n_switchers20, n_controls = n_controls20, + n_groups = n_groups20, n_periods = n_periods20, + seed = 120L, effects = 3, placebo = 2, by_path = 3, + predict_het_var = "het_x", + predict_het_horizons = c(-1), + ci_level = 95, + dont_drop_larger_lower = TRUE), + results = extract_dcdh_by_path_predict_het(res22, n_effects = 3) + ) } # --------------------------------------------------------------------------- diff --git a/benchmarks/data/dcdh_dynr_golden_values.json b/benchmarks/data/dcdh_dynr_golden_values.json index fe4e6f44..700c5ffe 100644 --- a/benchmarks/data/dcdh_dynr_golden_values.json +++ b/benchmarks/data/dcdh_dynr_golden_values.json @@ -1488,7 +1488,8 @@ "n_obs": 90, "p_value": 0.13394537794 } - } + }, + "placebo_predict_het": {} } }, "multi_path_reversible_by_path_predict_het": { @@ -1546,6 +1547,230 @@ "n_obs": 30, "p_value": 0.53612734365 } + }, + "placebo_horizons": {} + }, + { + "path": "0,1,1,0", + "frequency_rank": 2, + "horizons": { + "1": { + "beta": 3.3963706812, + "se": 0.28651602615, + "t": 11.8540338799, + "ci_lo": 2.8074285548, + "ci_hi": 3.9853128076, + "n_obs": 30, + "p_value": 5.495025198e-12 + }, + "2": { + "beta": 3.1871911509, + "se": 0.26329034164, + "t": 12.1052338305, + "ci_lo": 2.6459901027, + "ci_hi": 3.728392199, + "n_obs": 30, + "p_value": 3.4532476901e-12 + }, + "3": { + "beta": 0.46828316092, + "se": 0.27123345051, + "t": 1.7264948701, + "ci_lo": -0.089245181354, + "ci_hi": 1.0258115032, + "n_obs": 30, + "p_value": 0.096124084653 + } + }, + "placebo_horizons": {} + }, + { + "path": "0,1,1,1", + "frequency_rank": 3, + "horizons": { + "1": { + "beta": 3.1122409756, + "se": 0.28295503686, + "t": 10.9990654702, + "ci_lo": 2.5306185675, + "ci_hi": 3.6938633837, + "n_obs": 30, + "p_value": 2.8157393205e-11 + }, + "2": { + "beta": 3.1939245728, + "se": 0.23480746457, + "t": 13.6023127656, + "ci_lo": 2.711270917, + "ci_hi": 3.6765782286, + "n_obs": 30, + "p_value": 2.4833856221e-13 + }, + "3": { + "beta": 2.8295623299, + "se": 0.27042368033, + "t": 10.4634413913, + "ci_lo": 2.2736984941, + "ci_hi": 3.3854261657, + "n_obs": 30, + "p_value": 8.1857621596e-11 + } + }, + "placebo_horizons": {} + } + ] + } + }, + "multi_path_reversible_predict_het_with_placebo_global": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + "treatment": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "outcome": [-0.38947445306, -0.23568361568, 0.21162698298, 9.4874432555, 9.5272086491, 9.8732152644, 10.4966606852, 10.7269218628, 11.3041347822, 12.3075808547, -0.53540224144, -0.85896484972, 0.9039919656, 8.4216226165, 0.32926420824, 1.5187015075, 2.2466260038, 2.3752794078, 2.5489603023, 3.8839539563, -1.6531086427, -0.48172021176, -1.1588810115, 7.5449719691, 8.2423444411, 0.16054588328, 1.34512733, 2.3267248703, 2.907302291, 2.7362604935, 3.9790870637, 5.1832195997, 4.9765720574, 5.9562138217, 14.7385510317, 14.1883967913, 14.9982098406, 16.5614484131, 16.3735973947, 17.2722352384, 2.4672139194, 3.181736844, 2.8966044002, 3.8788938746, 12.0094366558, 4.5284429927, 4.7586747034, 6.3688938717, 6.8655832555, 6.6200148023, 1.366346809, 2.2681542407, 1.0986738641, 1.3627616225, 10.60127055, 10.2334877677, 3.7661446414, 2.8049023737, 5.3088063004, 5.7352273543, -2.1462960524, -1.562262737, -0.68045628748, -0.39170645702, 0.51893991954, 8.2908071534, 9.041118565, 9.1952101148, 10.5230194116, 10.4974242438, 3.8654236028, 3.9905881505, 4.2598845829, 5.4183992861, 5.9314932187, 13.2206109279, 5.3935881935, 7.4081365763, 7.5828582314, 7.359520808, 0.32944110732, 1.3451564511, 0.47252530565, 1.86498985, 2.4477442051, 10.7154679806, 10.8414260484, 3.9189896786, 4.2352551524, 4.3157247407, 1.3062082738, 2.2840554655, 2.5999388791, 11.2750783874, 11.7106216584, 11.6742402808, 13.1041698079, 13.2031556633, 13.5015002952, 14.1776065603, 0.11158785236, 1.054725582, 2.5473237287, 9.9685092, 2.5553042659, 3.1672544975, 3.2044163382, 3.279807654, 4.9738691668, 6.0325408926, -0.080780006181, 2.1294224371, 1.8617613349, 9.4372859439, 10.6907812039, 3.0731760034, 3.9983448664, 3.9479302438, 4.5770928564, 4.759461957, 1.2233010078, 1.5007797084, 1.8169053771, 3.0011971575, 11.1825713053, 12.4024767365, 12.1157117449, 12.2510645066, 13.4088151694, 13.9401105065, -2.7956639463, -1.9178517996, -1.8381646645, -1.0116116427, 7.2114118991, -0.47283326103, -0.2375849953, -0.21983407988, 0.83491171322, 1.7623326366, -2.4904909619, -1.1251647276, -0.0021674062124, 0.20812688743, 9.9195669513, 9.5480968853, 2.3956781408, 2.1183487474, 3.0646069867, 3.808704102, -2.0779624805, -1.9780684504, -0.33525118838, -0.057425341904, 1.0012356804, 8.1271505508, 8.7896794462, 8.8251678355, 10.1813290206, 10.1022892304, 3.6502716683, 3.7201719177, 3.7674199724, 5.1980443964, 5.7139097866, 14.6401937191, 7.0192780984, 7.0612922863, 7.5443921558, 8.7556031236, -0.26132650677, -0.50936928911, 0.99992467827, 0.80556665801, 1.4907623966, 9.7831198927, 9.9980487229, 3.7279599817, 3.0358585022, 3.9339454859, 1.7055670675, 2.6615688448, 2.0238218508, 10.8780753336, 11.4600783337, 12.1587520022, 13.7945646254, 13.1589222575, 13.342547118, 13.9311259257, -0.98078405525, -0.63223319399, -1.929094269, 8.5579341992, -0.79975861867, 0.6987605024, 2.0093938808, 1.957238027, 2.5023783296, 2.9270198103, 3.616503637, 5.3290175157, 5.0181873208, 13.0651636363, 13.3235606524, 6.4172576786, 7.1614692798, 7.607304798, 8.2321568553, 8.7568274818, -0.55566389977, 0.36757246427, 0.28018998819, 1.261105748, 8.9955481188, 10.1666747189, 9.8200303073, 10.1898516286, 11.6332278043, 11.4423294544, 0.043742524857, 1.4317108525, 1.446779116, 1.7235245936, 9.6155857044, 2.6780724807, 3.8338588243, 3.0809574447, 3.8233716995, 5.2736169021, 0.23027752914, -0.26124341972, -0.046130533987, 1.0311827165, 10.2695519873, 9.1487918458, 1.7938108203, 3.856934424, 3.8212591017, 3.3125876647, 3.2444220758, 3.0145873594, 3.5693841475, 4.527761117, 5.100913301, 12.6708939616, 13.4343961383, 14.8168409647, 14.6450726832, 15.5364022794, -1.8347375996, -1.4876759328, -1.2163971975, -0.29855416949, 0.092900206374, 8.2832035096, 0.013082856576, 0.89557660897, 1.7138465012, 2.2530638019, 0.76113222743, 1.043661851, 1.7984771121, 3.0208714656, 2.8743377143, 11.8256263263, 12.6746604421, 3.7250139312, 5.4491969792, 5.7317854828, 0.074535831744, 1.4428167027, 1.1066800877, 9.2238521871, 10.5115502601, 11.3772966396, 10.9541345172, 12.5793579545, 12.3657504826, 13.0027281071, -2.4127593869, -2.3899477395, -1.5435450717, 7.6410352797, -0.072105997553, -0.44113885708, -0.14677767369, 0.52052741992, 1.2023035084, 2.5550904186, -5.2176366841, -4.7239899175, -4.077171198, 4.735683732, 5.1687376682, -1.7818546965, -1.4377782451, -1.6932244105, -0.76675510256, 0.24508795491, 2.8905378674, 4.0481598896, 2.4793004476, 3.9117013242, 14.0021662289, 13.3506459755, 14.3634437928, 14.0247306384, 14.2254401205, 15.8489624363, 1.4579466797, 3.0292382559, 2.1459296903, 2.4742188578, 11.8428716637, 4.1913754533, 4.9542969898, 4.7906239328, 5.4839864806, 6.3573411246, 1.5502157812, 2.7273867786, 3.6829326297, 3.2500867309, 12.1787740115, 12.8266235821, 5.2117032225, 5.5086421172, 6.4287441774, 7.1957751897, 1.5717105195, 2.1034404835, 2.9853482577, 1.9922540467, 3.1024341108, 12.0144685484, 11.3633060429, 12.4330985545, 13.0966613228, 13.9743134509, 2.9326237068, 2.5054078142, 4.389182362, 4.6156128698, 5.1185159097, 13.6500763459, 5.7165365624, 6.8283484181, 7.2726425166, 7.1139668596, -2.7510252383, -1.7881562897, -0.7569242708, -0.66254116073, -1.0130202411, 8.2760144771, 7.6821223629, 0.55272479404, 0.88966618002, 1.6990596386, -1.1075252055, -1.0653834352, -0.43050860649, 7.9330480831, 8.497039261, 8.4097442995, 8.6939911965, 9.203283756, 10.6833768049, 11.668019098, 5.2983061789, 5.4612207703, 5.8971662603, 15.8508309236, 7.6326584579, 7.8929180033, 9.3436098284, 8.0966602405, 9.6916022556, 10.8633361911, 0.80205529395, 0.20107471189, 0.69348362951, 8.9602497569, 10.456290067, 3.2517643718, 3.3978156634, 3.1825880623, 4.0252656414, 3.81567534, -1.7209120673, -1.4829479201, -1.163442575, -1.790056039, 8.1935086645, 8.1984467002, 8.6224411669, 9.1043002177, 10.2683555294, 9.8689868745, 0.29561696587, 0.8554221211, 2.0339642264, 2.6531321176, 10.1813390641, 3.6104252101, 2.9532447436, 4.7696134091, 4.8295133364, 5.0527991602, 0.58705806724, 1.4180117625, 1.4244496058, 2.6400076277, 10.8060851593, 11.4310602002, 3.3611866291, 4.2666006678, 5.7305407586, 4.6888574948, -1.0815272324, -0.23665106862, 0.28232657309, 0.81491270698, 1.6876098526, 10.9938513032, 10.2613641269, 10.5614397285, 11.6892560277, 12.0693711136, -1.2744124685, -0.25160282178, 0.97134830573, 0.2398470695, 0.79407021378, 9.6954405575, 2.2737730981, 2.9067112029, 2.9139167009, 4.4474104736, 2.8877037347, 2.5995466583, 3.9802611614, 3.6293335277, 4.4413698643, 12.3784220024, 12.4012837999, 5.7105252229, 6.6018115119, 6.6402465829, 0.29239262974, 1.2220047033, 2.3159182171, 6.3110713658, 7.4969704119, 7.4467125172, 8.4492773221, 8.9485553238, 9.3296967175, 9.2584034658, 0.68581620777, 1.599784379, 1.1636924299, 6.6758938993, 1.702525925, 2.997307299, 4.1814522946, 4.5412303497, 4.6175597579, 4.2182195995, -1.5932447414, -0.68014300614, -0.6830571828, 4.3718025501, 4.3824694244, 0.098011860586, 0.77397454044, 0.73867374234, 2.4780776778, 1.453609304, 0.8477297338, -0.055091676225, 1.2279035812, 1.0540105073, 6.7232069649, 7.574659705, 8.6119120026, 8.965879519, 9.8579466434, 10.1375765135, 0.19069758436, 1.0157922636, 2.6483364818, 1.8677601978, 8.9249418047, 3.0859216384, 3.4905406714, 4.3722898374, 4.4431670254, 5.3993971226, -3.3670833707, -1.4697253234, -1.6150207132, -1.3246951404, 4.1797020113, 4.8251407527, 0.28412515654, 0.91461882204, 0.99263226799, 1.4414939648, -0.25098842145, -0.085274367311, -0.1373956682, 0.15144191374, 1.4329104513, 7.648234139, 6.8556590112, 6.5019130638, 8.3512621801, 8.8399798905, 1.8232880237, 1.0445447676, 1.6789949488, 2.2221513625, 2.9716852472, 8.8662419521, 3.5103702855, 4.1750677138, 5.5765981069, 6.1889800904, 0.37819640531, 2.3085907134, 2.2703432485, 2.0339267798, 2.4933037099, 7.2969353222, 8.6685670728, 3.9244198419, 4.1597588735, 4.7480514032, 2.3197377468, 2.9066032253, 3.0275584801, 9.0037335396, 7.9370045254, 9.4697266898, 10.1525937348, 10.5695573614, 10.9108106702, 10.5700879261, -2.0411429691, -1.730527702, -0.33798743459, 3.6419238489, 0.92486595565, 1.148748046, 1.2579016489, 2.753824807, 1.7992774598, 3.2821386074, 1.9269115148, 2.1414404672, 2.7275337912, 8.0888173297, 9.0665421161, 3.9158916232, 5.793690125, 5.6687501361, 5.5801231283, 5.5029519768, 1.9462441818, 2.587106672, 2.2657292461, 3.725756554, 8.8071198087, 8.8919966727, 10.4295998433, 11.2295422809, 10.992419767, 12.3528988981, 0.51858760917, 0.91477075195, 1.0080317414, 2.3184150701, 8.311700509, 3.6885278127, 3.4003729172, 4.6786132486, 4.0686884409, 4.915805778, -0.51524152113, -0.58645941226, -0.18816426967, -0.1146746434, 5.2180432031, 6.362684678, 1.9185794208, 2.758895588, 3.2557682801, 3.5754196536, -1.4128881009, -1.1117636987, -0.22622997905, 0.34854714406, 0.53510934436, 6.3081790903, 6.8667069477, 7.1597174792, 7.0436647989, 8.1714976953, 1.8119864596, 1.5601421163, 2.4154945667, 2.8617976566, 4.0783180801, 9.5651932194, 5.270553974, 5.0677497909, 5.684842441, 6.0440030115, -4.0527566651, -4.3064178115, -3.4440934523, -3.2411382271, -1.5680894026, 3.5066065231, 4.0222421096, -1.8773911565, -0.26988754805, 0.26043138596, 2.8361206552, 3.8286696348, 3.7203584628, 9.7198598921, 9.0413047461, 11.5538392884, 11.30481579, 12.1053735548, 11.4067468353, 12.8609925775, -0.45512969419, -0.10209549375, 0.68755271212, 5.988299857, 0.79090267123, 3.0182794574, 2.6212420009, 2.6720411405, 3.2406228495, 3.6639011017, 3.277243578, 4.5561359837, 4.1140656646, 10.0312637383, 10.6523891786, 6.5488403692, 6.367970651, 7.1791200758, 7.5460429143, 8.020449256, -2.6950947559, -3.0761058934, -2.4352818908, -1.8047061817, 2.8460391719, 4.387045321, 4.9861574992, 5.6441444576, 5.4262427911, 5.9915006491, -2.7025124396, -2.0006968552, -1.4619125359, -0.78214005868, 4.7204380238, 0.14475544389, 0.31247146682, 0.81393622044, 2.1864289018, 1.2845520714, -1.6017611223, -0.63409687164, -0.61757075983, 1.3645308869, 5.0822395083, 5.5951077433, 1.861764545, 1.8682365106, 2.7495963329, 3.4486828226, 1.4498521375, 2.252985795, 2.5042359162, 3.3001439812, 3.7402133884, 9.1921958482, 10.0247020114, 9.9641095372, 10.5841137951, 10.7239198288, -3.508178256, -2.9298772963, -2.7993204407, -2.0102210017, -2.1316030015, 4.4468867886, -0.49269407384, 0.68321152785, 0.098691593493, 0.44118759045, -2.0920700385, -1.590574067, -0.86172015186, -1.2531938485, 0.33645216964, 5.7867090665, 5.3428352061, 1.4600703271, 1.456727736, 3.0492924763, -0.48978688819, -0.18337661381, 0.72365944284, 6.6551141084, 6.9412464716, 7.2842518826, 7.5096654312, 7.9465656395, 8.6876180297, 9.0223244405, -4.0331048686, -4.6608913806, -3.2781301701, 1.6831261139, -3.439395894, -2.0717054903, -1.6038605293, -1.4175370871, -0.90858805401, -1.038290352, -2.7774091386, -2.3177608925, -0.85938575013, 5.5146980102, 5.7348025667, 0.5754581037, 0.65891969572, 1.2389167582, 2.0435927323, 1.8646853236, 1.13044316, 1.6152067423, 1.8433539367, 2.5184079189, 8.1693710149, 9.1298558756, 9.4690362017, 10.4772665744, 9.9178710379, 11.2768003797, -4.2201928233, -3.1296750739, -2.1963975207, -2.4282653157, 2.6559008893, -1.345311643, -0.55777399722, 0.039359733318, -0.091148987272, 1.1577176451, 2.1214982795, 2.1074229551, 3.2838730945, 5.122385666, 8.4827558178, 9.5363576609, 4.3805908855, 5.6180574956, 5.4885979348, 6.6574463826, 3.3044452322, 3.7907846417, 3.5434672216, 4.5968076373, 5.0973756463, 11.0498816954, 11.3102519271, 12.1880907787, 12.6804100484, 12.3694441975, -1.0776008576, -0.80639205059, -0.25968633449, -0.27319955367, -0.4020016233, 6.1943632529, 2.2952204585, 2.2180117426, 2.1132930971, 2.3116309818, -1.8467022583, -1.8965759207, -1.4672865528, -1.4475057995, -0.16632255853, 4.5370957083, 5.6290440436, 1.4496676538, 1.8695318093, 1.6356947361, -0.89664436868, 0.37852159348, 1.4254957278, 6.2304811947, 5.9777317, 7.1459413811, 7.5235867108, 8.6613019746, 8.1321174316, 9.5279854166, -1.4589153635, -0.78306139038, -0.20238002646, 5.5104827626, 0.49152010598, 0.86744374768, 1.9286384636, 2.5806103308, 2.8496930824, 2.4348250895, -0.036533972497, 0.207004254, -0.49018076779, 6.0529719853, 6.3484880533, 1.5371985143, 2.6551831784, 2.7127862584, 2.6929283426, 3.0481687887, 0.41533608345, 0.19412270959, 0.44662839578, 1.3063368129, 6.5002944896, 6.2187332039, 7.9403458529, 8.6118875247, 8.4482050796, 9.1856316845, 0.18053704127, -0.10682316028, 0.9128127945, 1.5004444945, 6.7936053535, 2.0289864485, 2.982605235, 2.8026053225, 3.5232163892, 4.8206101575, 3.2125051652, 3.9434954038, 4.5812676634, 5.181856726, 11.5547003011, 11.3794504772, 6.4794442193, 7.0864431319, 7.3520479742, 7.6067573977, 0.45400730084, 0.50469593165, -0.15334942425, 1.6789373293, 1.1334069731, 6.8475748307, 7.0292239561, 7.339011245, 8.4666051676, 8.5136696916, -1.2573536105, -0.13785707747, 0.58761560397, 1.4980853561, 1.6776613045, 7.8164304743, 1.8078839037, 2.3531997482, 3.4236838727, 3.719405626, 0.074032633417, 1.814538819, 1.3581908836, 2.3461222296, 3.1857112741, 8.0960479271, 7.5620221866, 3.9524023678, 5.0265240703, 4.8961197126, 1.2242853074, 1.4319331965, 2.1865840446, 1.2445095092, 3.2472439405, 3.5478312994, 4.3347472449, 3.7025538593, 3.7343055628, 5.2520598565, 0.42261622489, 1.5341546878, 2.1091275475, 2.6454517731, 2.9877288173, 3.7105099568, 3.371385426, 4.9902149205, 5.6603785771, 5.5144729766, 0.83738592206, 1.1336031968, 1.2955173463, 2.4042237907, 2.0025159071, 3.6017998716, 3.0922754444, 4.0010512866, 3.3814397228, 5.3317601623, -2.7224373238, -1.6068537104, -1.8182638402, -1.3495740912, -0.5678388365, -0.17789129917, -0.25266487419, 0.65726082751, 1.1299674607, 2.685825718, 2.4294472048, 2.69679639, 3.5074766318, 3.5273647629, 4.9542552887, 6.234232526, 6.0380579643, 6.0894049413, 6.2517694637, 7.3639076133, 0.67674781791, 2.3785713361, 2.1147102458, 2.6698045443, 4.6512799113, 2.9411541538, 4.550284476, 6.3724810856, 6.307971804, 5.7232910315, -1.6492585942, -1.7539792047, -1.9123520004, -0.40869695192, 0.74677096825, 0.13807947305, 1.0984134077, 2.4333951981, 3.0935415539, 2.855910576, 2.7513777021, 2.2988540544, 3.4339824897, 4.0025603665, 4.9161784384, 5.3750996072, 5.0699472188, 5.9682120541, 6.3750415099, 6.8659144838, -0.25008999524, -0.36155368448, 1.0730390213, 0.92233794381, 1.4353629051, 3.4427390551, 2.4755656479, 3.8880553371, 3.1890537717, 4.6918300874, 2.114627159, 1.6309141333, 3.1799739141, 4.1875171173, 3.2056631079, 4.3838886062, 5.1727617959, 4.2613681872, 5.2715451911, 6.5128933687, 2.0483819212, 3.9674454647, 2.7202020758, 3.5510473395, 4.1734635461, 3.8062951613, 5.3111025233, 5.2316163492, 5.614005842, 6.9321329919, 0.90297804009, 2.483627712, 3.0084619546, 2.9788799322, 3.6033174203, 3.4533849012, 3.9437255353, 4.9397819987, 4.8915899111, 4.8730162756, 0.040066616557, 0.6590094332, 0.97007462142, 0.68855093457, 2.6142505976, 2.7121691125, 3.4645319517, 3.5547984743, 4.4703968352, 4.798246022, -0.33000579984, -0.51179645192, 2.0683298992, 1.5578004908, 2.9519912561, 2.3182333687, 2.6519502439, 5.0200643644, 4.3979462621, 4.697667554, -0.60852106376, 0.12819634273, 0.65851446779, 2.334157038, 1.2507354539, 2.8736625902, 2.856854623, 3.8526650288, 3.6036816207, 4.363825628, -0.73128646947, -0.87705949197, 0.2068024588, 0.8102192234, 0.49489152403, 1.5117635517, 1.391564736, 3.0038229366, 3.247422302, 3.0927705043, -4.1039807396, -2.9931419689, -3.2088313683, -2.9148890748, -2.5175390588, -2.2039553654, -1.5537585763, -1.5038679763, -0.77320784305, -0.065851868564, 1.0313142498, 2.9571659829, 2.0190895471, 3.7268996701, 4.6589797789, 4.6889558678, 4.9110222729, 5.7559852625, 5.5423346785, 6.2178089257, 2.9428533374, 4.757281362, 4.6941910741, 4.478268561, 6.180925968, 5.971211092, 6.6300126261, 7.8431354173, 7.6138885889, 8.2870484982, -4.6486500232, -4.3293962547, -3.3837629389, -3.5033584064, -2.7672857308, -1.7697210179, -1.5745726688, -0.67578078035, -0.082928399663, 0.011366511, -0.49283294835, -0.38636689804, 0.83643989751, 1.212473889, 1.5125671082, 2.7632103096, 1.7720701418, 2.4320263932, 3.9936373848, 3.4689586505, 0.91627601873, 0.98105976401, 1.6130113126, 2.2386443746, 2.7213773977, 3.7001782294, 2.8956455331, 3.8890665453, 4.5542578852, 6.1909092137, -3.1401737915, -3.3173951931, -2.614595799, -1.948952474, -1.3420523774, -0.51325227315, -0.89496354307, 0.69317522502, 0.83387600449, 1.3736895326, -1.3991665799, -0.88071166454, -1.7935199456, -1.0327456384, 0.22797073004, 0.4750719784, 0.99050757786, 0.47105082401, 2.2144708992, 1.0455693417, -0.23698467418, 1.5335503698, 2.7875031807, 1.3339827803, 2.6836804393, 4.1014726763, 4.4595980479, 4.010149303, 4.8514460328, 5.0520554245, 2.2453832707, 3.296747875, 2.5233817543, 3.2384413823, 3.5365389376, 4.7254040415, 4.8124189642, 4.4508694895, 5.6664761848, 6.6236985335, 1.0230133467, 0.76044973396, 2.5663115776, 3.2345561469, 2.9240503032, 3.1886070679, 3.9852755792, 4.7770707231, 4.9810713019, 5.6255235495, -0.90286019841, -0.076239112849, 0.3715132453, 2.1594074749, 2.5021690364, 1.8118486327, 2.2917715352, 3.1525505143, 3.5313555246, 2.9499473078, 0.50211334146, 1.242218949, 0.87806989916, 2.5472634181, 2.4091157411, 2.0542727525, 2.1935037832, 3.6388483348, 4.8883468351, 3.6319662274, -2.6163110762, -2.0306604353, -0.74956124941, -1.1605560257, 0.057877418549, 0.21359397186, 0.81043487155, 1.6319201969, 2.0516515842, 3.361952844], + "het_x": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + }, + "params": { + "pattern": "multi_path_reversible_predict_het_with_placebo_global", + "n_switchers": 90, + "n_controls": 30, + "n_groups": 120, + "n_periods": 10, + "seed": 120, + "effects": 3, + "placebo": 2, + "predict_het_var": "het_x", + "predict_het_horizons": -1, + "ci_level": 95, + "dont_drop_larger_lower": true + }, + "results": { + "predict_het": { + "1": { + "beta": 3.1129329005, + "se": 0.16802462607, + "t": 18.5266467977, + "ci_lo": 2.7789109989, + "ci_hi": 3.4469548022, + "n_obs": 90, + "p_value": 9.0828759468e-32 + }, + "2": { + "beta": 2.0734964222, + "se": 0.69578948634, + "t": 2.9800628824, + "ci_lo": 0.69031270198, + "ci_hi": 3.4566801424, + "n_obs": 90, + "p_value": 0.0037461129962 + }, + "3": { + "beta": 1.0453185701, + "se": 0.69089010811, + "t": 1.5130026582, + "ci_lo": -0.32812550855, + "ci_hi": 2.4187626488, + "n_obs": 90, + "p_value": 0.13394537794 + } + }, + "placebo_predict_het": { + "-2": { + "beta": 0.32798894118, + "se": 0.15084446079, + "t": 2.1743519083, + "ci_lo": 0.028120077748, + "ci_hi": 0.62785780462, + "n_obs": 90, + "p_value": 0.032425356672 + }, + "-1": { + "beta": 0.12495390804, + "se": 0.14220705065, + "t": 0.87867589877, + "ci_lo": -0.15774435231, + "ci_hi": 0.40765216839, + "n_obs": 90, + "p_value": 0.38202545605 + } + } + } + }, + "multi_path_reversible_predict_het_with_placebo": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + "treatment": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "outcome": [-0.38947445306, -0.23568361568, 0.21162698298, 9.4874432555, 9.5272086491, 9.8732152644, 10.4966606852, 10.7269218628, 11.3041347822, 12.3075808547, -0.53540224144, -0.85896484972, 0.9039919656, 8.4216226165, 0.32926420824, 1.5187015075, 2.2466260038, 2.3752794078, 2.5489603023, 3.8839539563, -1.6531086427, -0.48172021176, -1.1588810115, 7.5449719691, 8.2423444411, 0.16054588328, 1.34512733, 2.3267248703, 2.907302291, 2.7362604935, 3.9790870637, 5.1832195997, 4.9765720574, 5.9562138217, 14.7385510317, 14.1883967913, 14.9982098406, 16.5614484131, 16.3735973947, 17.2722352384, 2.4672139194, 3.181736844, 2.8966044002, 3.8788938746, 12.0094366558, 4.5284429927, 4.7586747034, 6.3688938717, 6.8655832555, 6.6200148023, 1.366346809, 2.2681542407, 1.0986738641, 1.3627616225, 10.60127055, 10.2334877677, 3.7661446414, 2.8049023737, 5.3088063004, 5.7352273543, -2.1462960524, -1.562262737, -0.68045628748, -0.39170645702, 0.51893991954, 8.2908071534, 9.041118565, 9.1952101148, 10.5230194116, 10.4974242438, 3.8654236028, 3.9905881505, 4.2598845829, 5.4183992861, 5.9314932187, 13.2206109279, 5.3935881935, 7.4081365763, 7.5828582314, 7.359520808, 0.32944110732, 1.3451564511, 0.47252530565, 1.86498985, 2.4477442051, 10.7154679806, 10.8414260484, 3.9189896786, 4.2352551524, 4.3157247407, 1.3062082738, 2.2840554655, 2.5999388791, 11.2750783874, 11.7106216584, 11.6742402808, 13.1041698079, 13.2031556633, 13.5015002952, 14.1776065603, 0.11158785236, 1.054725582, 2.5473237287, 9.9685092, 2.5553042659, 3.1672544975, 3.2044163382, 3.279807654, 4.9738691668, 6.0325408926, -0.080780006181, 2.1294224371, 1.8617613349, 9.4372859439, 10.6907812039, 3.0731760034, 3.9983448664, 3.9479302438, 4.5770928564, 4.759461957, 1.2233010078, 1.5007797084, 1.8169053771, 3.0011971575, 11.1825713053, 12.4024767365, 12.1157117449, 12.2510645066, 13.4088151694, 13.9401105065, -2.7956639463, -1.9178517996, -1.8381646645, -1.0116116427, 7.2114118991, -0.47283326103, -0.2375849953, -0.21983407988, 0.83491171322, 1.7623326366, -2.4904909619, -1.1251647276, -0.0021674062124, 0.20812688743, 9.9195669513, 9.5480968853, 2.3956781408, 2.1183487474, 3.0646069867, 3.808704102, -2.0779624805, -1.9780684504, -0.33525118838, -0.057425341904, 1.0012356804, 8.1271505508, 8.7896794462, 8.8251678355, 10.1813290206, 10.1022892304, 3.6502716683, 3.7201719177, 3.7674199724, 5.1980443964, 5.7139097866, 14.6401937191, 7.0192780984, 7.0612922863, 7.5443921558, 8.7556031236, -0.26132650677, -0.50936928911, 0.99992467827, 0.80556665801, 1.4907623966, 9.7831198927, 9.9980487229, 3.7279599817, 3.0358585022, 3.9339454859, 1.7055670675, 2.6615688448, 2.0238218508, 10.8780753336, 11.4600783337, 12.1587520022, 13.7945646254, 13.1589222575, 13.342547118, 13.9311259257, -0.98078405525, -0.63223319399, -1.929094269, 8.5579341992, -0.79975861867, 0.6987605024, 2.0093938808, 1.957238027, 2.5023783296, 2.9270198103, 3.616503637, 5.3290175157, 5.0181873208, 13.0651636363, 13.3235606524, 6.4172576786, 7.1614692798, 7.607304798, 8.2321568553, 8.7568274818, -0.55566389977, 0.36757246427, 0.28018998819, 1.261105748, 8.9955481188, 10.1666747189, 9.8200303073, 10.1898516286, 11.6332278043, 11.4423294544, 0.043742524857, 1.4317108525, 1.446779116, 1.7235245936, 9.6155857044, 2.6780724807, 3.8338588243, 3.0809574447, 3.8233716995, 5.2736169021, 0.23027752914, -0.26124341972, -0.046130533987, 1.0311827165, 10.2695519873, 9.1487918458, 1.7938108203, 3.856934424, 3.8212591017, 3.3125876647, 3.2444220758, 3.0145873594, 3.5693841475, 4.527761117, 5.100913301, 12.6708939616, 13.4343961383, 14.8168409647, 14.6450726832, 15.5364022794, -1.8347375996, -1.4876759328, -1.2163971975, -0.29855416949, 0.092900206374, 8.2832035096, 0.013082856576, 0.89557660897, 1.7138465012, 2.2530638019, 0.76113222743, 1.043661851, 1.7984771121, 3.0208714656, 2.8743377143, 11.8256263263, 12.6746604421, 3.7250139312, 5.4491969792, 5.7317854828, 0.074535831744, 1.4428167027, 1.1066800877, 9.2238521871, 10.5115502601, 11.3772966396, 10.9541345172, 12.5793579545, 12.3657504826, 13.0027281071, -2.4127593869, -2.3899477395, -1.5435450717, 7.6410352797, -0.072105997553, -0.44113885708, -0.14677767369, 0.52052741992, 1.2023035084, 2.5550904186, -5.2176366841, -4.7239899175, -4.077171198, 4.735683732, 5.1687376682, -1.7818546965, -1.4377782451, -1.6932244105, -0.76675510256, 0.24508795491, 2.8905378674, 4.0481598896, 2.4793004476, 3.9117013242, 14.0021662289, 13.3506459755, 14.3634437928, 14.0247306384, 14.2254401205, 15.8489624363, 1.4579466797, 3.0292382559, 2.1459296903, 2.4742188578, 11.8428716637, 4.1913754533, 4.9542969898, 4.7906239328, 5.4839864806, 6.3573411246, 1.5502157812, 2.7273867786, 3.6829326297, 3.2500867309, 12.1787740115, 12.8266235821, 5.2117032225, 5.5086421172, 6.4287441774, 7.1957751897, 1.5717105195, 2.1034404835, 2.9853482577, 1.9922540467, 3.1024341108, 12.0144685484, 11.3633060429, 12.4330985545, 13.0966613228, 13.9743134509, 2.9326237068, 2.5054078142, 4.389182362, 4.6156128698, 5.1185159097, 13.6500763459, 5.7165365624, 6.8283484181, 7.2726425166, 7.1139668596, -2.7510252383, -1.7881562897, -0.7569242708, -0.66254116073, -1.0130202411, 8.2760144771, 7.6821223629, 0.55272479404, 0.88966618002, 1.6990596386, -1.1075252055, -1.0653834352, -0.43050860649, 7.9330480831, 8.497039261, 8.4097442995, 8.6939911965, 9.203283756, 10.6833768049, 11.668019098, 5.2983061789, 5.4612207703, 5.8971662603, 15.8508309236, 7.6326584579, 7.8929180033, 9.3436098284, 8.0966602405, 9.6916022556, 10.8633361911, 0.80205529395, 0.20107471189, 0.69348362951, 8.9602497569, 10.456290067, 3.2517643718, 3.3978156634, 3.1825880623, 4.0252656414, 3.81567534, -1.7209120673, -1.4829479201, -1.163442575, -1.790056039, 8.1935086645, 8.1984467002, 8.6224411669, 9.1043002177, 10.2683555294, 9.8689868745, 0.29561696587, 0.8554221211, 2.0339642264, 2.6531321176, 10.1813390641, 3.6104252101, 2.9532447436, 4.7696134091, 4.8295133364, 5.0527991602, 0.58705806724, 1.4180117625, 1.4244496058, 2.6400076277, 10.8060851593, 11.4310602002, 3.3611866291, 4.2666006678, 5.7305407586, 4.6888574948, -1.0815272324, -0.23665106862, 0.28232657309, 0.81491270698, 1.6876098526, 10.9938513032, 10.2613641269, 10.5614397285, 11.6892560277, 12.0693711136, -1.2744124685, -0.25160282178, 0.97134830573, 0.2398470695, 0.79407021378, 9.6954405575, 2.2737730981, 2.9067112029, 2.9139167009, 4.4474104736, 2.8877037347, 2.5995466583, 3.9802611614, 3.6293335277, 4.4413698643, 12.3784220024, 12.4012837999, 5.7105252229, 6.6018115119, 6.6402465829, 0.29239262974, 1.2220047033, 2.3159182171, 6.3110713658, 7.4969704119, 7.4467125172, 8.4492773221, 8.9485553238, 9.3296967175, 9.2584034658, 0.68581620777, 1.599784379, 1.1636924299, 6.6758938993, 1.702525925, 2.997307299, 4.1814522946, 4.5412303497, 4.6175597579, 4.2182195995, -1.5932447414, -0.68014300614, -0.6830571828, 4.3718025501, 4.3824694244, 0.098011860586, 0.77397454044, 0.73867374234, 2.4780776778, 1.453609304, 0.8477297338, -0.055091676225, 1.2279035812, 1.0540105073, 6.7232069649, 7.574659705, 8.6119120026, 8.965879519, 9.8579466434, 10.1375765135, 0.19069758436, 1.0157922636, 2.6483364818, 1.8677601978, 8.9249418047, 3.0859216384, 3.4905406714, 4.3722898374, 4.4431670254, 5.3993971226, -3.3670833707, -1.4697253234, -1.6150207132, -1.3246951404, 4.1797020113, 4.8251407527, 0.28412515654, 0.91461882204, 0.99263226799, 1.4414939648, -0.25098842145, -0.085274367311, -0.1373956682, 0.15144191374, 1.4329104513, 7.648234139, 6.8556590112, 6.5019130638, 8.3512621801, 8.8399798905, 1.8232880237, 1.0445447676, 1.6789949488, 2.2221513625, 2.9716852472, 8.8662419521, 3.5103702855, 4.1750677138, 5.5765981069, 6.1889800904, 0.37819640531, 2.3085907134, 2.2703432485, 2.0339267798, 2.4933037099, 7.2969353222, 8.6685670728, 3.9244198419, 4.1597588735, 4.7480514032, 2.3197377468, 2.9066032253, 3.0275584801, 9.0037335396, 7.9370045254, 9.4697266898, 10.1525937348, 10.5695573614, 10.9108106702, 10.5700879261, -2.0411429691, -1.730527702, -0.33798743459, 3.6419238489, 0.92486595565, 1.148748046, 1.2579016489, 2.753824807, 1.7992774598, 3.2821386074, 1.9269115148, 2.1414404672, 2.7275337912, 8.0888173297, 9.0665421161, 3.9158916232, 5.793690125, 5.6687501361, 5.5801231283, 5.5029519768, 1.9462441818, 2.587106672, 2.2657292461, 3.725756554, 8.8071198087, 8.8919966727, 10.4295998433, 11.2295422809, 10.992419767, 12.3528988981, 0.51858760917, 0.91477075195, 1.0080317414, 2.3184150701, 8.311700509, 3.6885278127, 3.4003729172, 4.6786132486, 4.0686884409, 4.915805778, -0.51524152113, -0.58645941226, -0.18816426967, -0.1146746434, 5.2180432031, 6.362684678, 1.9185794208, 2.758895588, 3.2557682801, 3.5754196536, -1.4128881009, -1.1117636987, -0.22622997905, 0.34854714406, 0.53510934436, 6.3081790903, 6.8667069477, 7.1597174792, 7.0436647989, 8.1714976953, 1.8119864596, 1.5601421163, 2.4154945667, 2.8617976566, 4.0783180801, 9.5651932194, 5.270553974, 5.0677497909, 5.684842441, 6.0440030115, -4.0527566651, -4.3064178115, -3.4440934523, -3.2411382271, -1.5680894026, 3.5066065231, 4.0222421096, -1.8773911565, -0.26988754805, 0.26043138596, 2.8361206552, 3.8286696348, 3.7203584628, 9.7198598921, 9.0413047461, 11.5538392884, 11.30481579, 12.1053735548, 11.4067468353, 12.8609925775, -0.45512969419, -0.10209549375, 0.68755271212, 5.988299857, 0.79090267123, 3.0182794574, 2.6212420009, 2.6720411405, 3.2406228495, 3.6639011017, 3.277243578, 4.5561359837, 4.1140656646, 10.0312637383, 10.6523891786, 6.5488403692, 6.367970651, 7.1791200758, 7.5460429143, 8.020449256, -2.6950947559, -3.0761058934, -2.4352818908, -1.8047061817, 2.8460391719, 4.387045321, 4.9861574992, 5.6441444576, 5.4262427911, 5.9915006491, -2.7025124396, -2.0006968552, -1.4619125359, -0.78214005868, 4.7204380238, 0.14475544389, 0.31247146682, 0.81393622044, 2.1864289018, 1.2845520714, -1.6017611223, -0.63409687164, -0.61757075983, 1.3645308869, 5.0822395083, 5.5951077433, 1.861764545, 1.8682365106, 2.7495963329, 3.4486828226, 1.4498521375, 2.252985795, 2.5042359162, 3.3001439812, 3.7402133884, 9.1921958482, 10.0247020114, 9.9641095372, 10.5841137951, 10.7239198288, -3.508178256, -2.9298772963, -2.7993204407, -2.0102210017, -2.1316030015, 4.4468867886, -0.49269407384, 0.68321152785, 0.098691593493, 0.44118759045, -2.0920700385, -1.590574067, -0.86172015186, -1.2531938485, 0.33645216964, 5.7867090665, 5.3428352061, 1.4600703271, 1.456727736, 3.0492924763, -0.48978688819, -0.18337661381, 0.72365944284, 6.6551141084, 6.9412464716, 7.2842518826, 7.5096654312, 7.9465656395, 8.6876180297, 9.0223244405, -4.0331048686, -4.6608913806, -3.2781301701, 1.6831261139, -3.439395894, -2.0717054903, -1.6038605293, -1.4175370871, -0.90858805401, -1.038290352, -2.7774091386, -2.3177608925, -0.85938575013, 5.5146980102, 5.7348025667, 0.5754581037, 0.65891969572, 1.2389167582, 2.0435927323, 1.8646853236, 1.13044316, 1.6152067423, 1.8433539367, 2.5184079189, 8.1693710149, 9.1298558756, 9.4690362017, 10.4772665744, 9.9178710379, 11.2768003797, -4.2201928233, -3.1296750739, -2.1963975207, -2.4282653157, 2.6559008893, -1.345311643, -0.55777399722, 0.039359733318, -0.091148987272, 1.1577176451, 2.1214982795, 2.1074229551, 3.2838730945, 5.122385666, 8.4827558178, 9.5363576609, 4.3805908855, 5.6180574956, 5.4885979348, 6.6574463826, 3.3044452322, 3.7907846417, 3.5434672216, 4.5968076373, 5.0973756463, 11.0498816954, 11.3102519271, 12.1880907787, 12.6804100484, 12.3694441975, -1.0776008576, -0.80639205059, -0.25968633449, -0.27319955367, -0.4020016233, 6.1943632529, 2.2952204585, 2.2180117426, 2.1132930971, 2.3116309818, -1.8467022583, -1.8965759207, -1.4672865528, -1.4475057995, -0.16632255853, 4.5370957083, 5.6290440436, 1.4496676538, 1.8695318093, 1.6356947361, -0.89664436868, 0.37852159348, 1.4254957278, 6.2304811947, 5.9777317, 7.1459413811, 7.5235867108, 8.6613019746, 8.1321174316, 9.5279854166, -1.4589153635, -0.78306139038, -0.20238002646, 5.5104827626, 0.49152010598, 0.86744374768, 1.9286384636, 2.5806103308, 2.8496930824, 2.4348250895, -0.036533972497, 0.207004254, -0.49018076779, 6.0529719853, 6.3484880533, 1.5371985143, 2.6551831784, 2.7127862584, 2.6929283426, 3.0481687887, 0.41533608345, 0.19412270959, 0.44662839578, 1.3063368129, 6.5002944896, 6.2187332039, 7.9403458529, 8.6118875247, 8.4482050796, 9.1856316845, 0.18053704127, -0.10682316028, 0.9128127945, 1.5004444945, 6.7936053535, 2.0289864485, 2.982605235, 2.8026053225, 3.5232163892, 4.8206101575, 3.2125051652, 3.9434954038, 4.5812676634, 5.181856726, 11.5547003011, 11.3794504772, 6.4794442193, 7.0864431319, 7.3520479742, 7.6067573977, 0.45400730084, 0.50469593165, -0.15334942425, 1.6789373293, 1.1334069731, 6.8475748307, 7.0292239561, 7.339011245, 8.4666051676, 8.5136696916, -1.2573536105, -0.13785707747, 0.58761560397, 1.4980853561, 1.6776613045, 7.8164304743, 1.8078839037, 2.3531997482, 3.4236838727, 3.719405626, 0.074032633417, 1.814538819, 1.3581908836, 2.3461222296, 3.1857112741, 8.0960479271, 7.5620221866, 3.9524023678, 5.0265240703, 4.8961197126, 1.2242853074, 1.4319331965, 2.1865840446, 1.2445095092, 3.2472439405, 3.5478312994, 4.3347472449, 3.7025538593, 3.7343055628, 5.2520598565, 0.42261622489, 1.5341546878, 2.1091275475, 2.6454517731, 2.9877288173, 3.7105099568, 3.371385426, 4.9902149205, 5.6603785771, 5.5144729766, 0.83738592206, 1.1336031968, 1.2955173463, 2.4042237907, 2.0025159071, 3.6017998716, 3.0922754444, 4.0010512866, 3.3814397228, 5.3317601623, -2.7224373238, -1.6068537104, -1.8182638402, -1.3495740912, -0.5678388365, -0.17789129917, -0.25266487419, 0.65726082751, 1.1299674607, 2.685825718, 2.4294472048, 2.69679639, 3.5074766318, 3.5273647629, 4.9542552887, 6.234232526, 6.0380579643, 6.0894049413, 6.2517694637, 7.3639076133, 0.67674781791, 2.3785713361, 2.1147102458, 2.6698045443, 4.6512799113, 2.9411541538, 4.550284476, 6.3724810856, 6.307971804, 5.7232910315, -1.6492585942, -1.7539792047, -1.9123520004, -0.40869695192, 0.74677096825, 0.13807947305, 1.0984134077, 2.4333951981, 3.0935415539, 2.855910576, 2.7513777021, 2.2988540544, 3.4339824897, 4.0025603665, 4.9161784384, 5.3750996072, 5.0699472188, 5.9682120541, 6.3750415099, 6.8659144838, -0.25008999524, -0.36155368448, 1.0730390213, 0.92233794381, 1.4353629051, 3.4427390551, 2.4755656479, 3.8880553371, 3.1890537717, 4.6918300874, 2.114627159, 1.6309141333, 3.1799739141, 4.1875171173, 3.2056631079, 4.3838886062, 5.1727617959, 4.2613681872, 5.2715451911, 6.5128933687, 2.0483819212, 3.9674454647, 2.7202020758, 3.5510473395, 4.1734635461, 3.8062951613, 5.3111025233, 5.2316163492, 5.614005842, 6.9321329919, 0.90297804009, 2.483627712, 3.0084619546, 2.9788799322, 3.6033174203, 3.4533849012, 3.9437255353, 4.9397819987, 4.8915899111, 4.8730162756, 0.040066616557, 0.6590094332, 0.97007462142, 0.68855093457, 2.6142505976, 2.7121691125, 3.4645319517, 3.5547984743, 4.4703968352, 4.798246022, -0.33000579984, -0.51179645192, 2.0683298992, 1.5578004908, 2.9519912561, 2.3182333687, 2.6519502439, 5.0200643644, 4.3979462621, 4.697667554, -0.60852106376, 0.12819634273, 0.65851446779, 2.334157038, 1.2507354539, 2.8736625902, 2.856854623, 3.8526650288, 3.6036816207, 4.363825628, -0.73128646947, -0.87705949197, 0.2068024588, 0.8102192234, 0.49489152403, 1.5117635517, 1.391564736, 3.0038229366, 3.247422302, 3.0927705043, -4.1039807396, -2.9931419689, -3.2088313683, -2.9148890748, -2.5175390588, -2.2039553654, -1.5537585763, -1.5038679763, -0.77320784305, -0.065851868564, 1.0313142498, 2.9571659829, 2.0190895471, 3.7268996701, 4.6589797789, 4.6889558678, 4.9110222729, 5.7559852625, 5.5423346785, 6.2178089257, 2.9428533374, 4.757281362, 4.6941910741, 4.478268561, 6.180925968, 5.971211092, 6.6300126261, 7.8431354173, 7.6138885889, 8.2870484982, -4.6486500232, -4.3293962547, -3.3837629389, -3.5033584064, -2.7672857308, -1.7697210179, -1.5745726688, -0.67578078035, -0.082928399663, 0.011366511, -0.49283294835, -0.38636689804, 0.83643989751, 1.212473889, 1.5125671082, 2.7632103096, 1.7720701418, 2.4320263932, 3.9936373848, 3.4689586505, 0.91627601873, 0.98105976401, 1.6130113126, 2.2386443746, 2.7213773977, 3.7001782294, 2.8956455331, 3.8890665453, 4.5542578852, 6.1909092137, -3.1401737915, -3.3173951931, -2.614595799, -1.948952474, -1.3420523774, -0.51325227315, -0.89496354307, 0.69317522502, 0.83387600449, 1.3736895326, -1.3991665799, -0.88071166454, -1.7935199456, -1.0327456384, 0.22797073004, 0.4750719784, 0.99050757786, 0.47105082401, 2.2144708992, 1.0455693417, -0.23698467418, 1.5335503698, 2.7875031807, 1.3339827803, 2.6836804393, 4.1014726763, 4.4595980479, 4.010149303, 4.8514460328, 5.0520554245, 2.2453832707, 3.296747875, 2.5233817543, 3.2384413823, 3.5365389376, 4.7254040415, 4.8124189642, 4.4508694895, 5.6664761848, 6.6236985335, 1.0230133467, 0.76044973396, 2.5663115776, 3.2345561469, 2.9240503032, 3.1886070679, 3.9852755792, 4.7770707231, 4.9810713019, 5.6255235495, -0.90286019841, -0.076239112849, 0.3715132453, 2.1594074749, 2.5021690364, 1.8118486327, 2.2917715352, 3.1525505143, 3.5313555246, 2.9499473078, 0.50211334146, 1.242218949, 0.87806989916, 2.5472634181, 2.4091157411, 2.0542727525, 2.1935037832, 3.6388483348, 4.8883468351, 3.6319662274, -2.6163110762, -2.0306604353, -0.74956124941, -1.1605560257, 0.057877418549, 0.21359397186, 0.81043487155, 1.6319201969, 2.0516515842, 3.361952844], + "het_x": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] + }, + "params": { + "pattern": "multi_path_reversible_predict_het_with_placebo", + "n_switchers": 90, + "n_controls": 30, + "n_groups": 120, + "n_periods": 10, + "seed": 120, + "effects": 3, + "placebo": 2, + "by_path": 3, + "predict_het_var": "het_x", + "predict_het_horizons": -1, + "ci_level": 95, + "dont_drop_larger_lower": true + }, + "results": { + "by_path_predict_het": [ + { + "path": "0,1,0,0", + "frequency_rank": 1, + "horizons": { + "1": { + "beta": 2.8301870449, + "se": 0.32413011552, + "t": 8.7316386517, + "ci_lo": 2.1639280505, + "ci_hi": 3.4964460393, + "n_obs": 30, + "p_value": 3.2987315589e-09 + }, + "2": { + "beta": -0.16062645702, + "se": 0.27377503227, + "t": -0.58670966336, + "ci_lo": -0.72337909543, + "ci_hi": 0.40212618138, + "n_obs": 30, + "p_value": 0.56245884519 + }, + "3": { + "beta": -0.16188978053, + "se": 0.25819780533, + "t": -0.62699905727, + "ci_lo": -0.69262297039, + "ci_hi": 0.36884340932, + "n_obs": 30, + "p_value": 0.53612734365 + } + }, + "placebo_horizons": { + "-2": { + "beta": 0.17845234804, + "se": 0.28623051863, + "t": 0.62345674701, + "ci_lo": -0.40990290924, + "ci_hi": 0.76680760533, + "n_obs": 30, + "p_value": 0.53841588057 + }, + "-1": { + "beta": -0.10541988314, + "se": 0.25556786142, + "t": -0.41249272328, + "ci_lo": -0.63074714587, + "ci_hi": 0.41990737958, + "n_obs": 30, + "p_value": 0.68335990755 + } } }, { @@ -1579,6 +1804,26 @@ "n_obs": 30, "p_value": 0.096124084653 } + }, + "placebo_horizons": { + "-2": { + "beta": 0.34827327298, + "se": 0.32052969, + "t": 1.0865554232, + "ci_lo": -0.31058494077, + "ci_hi": 1.0071314867, + "n_obs": 30, + "p_value": 0.28720457559 + }, + "-1": { + "beta": 0.51528032817, + "se": 0.24805394114, + "t": 2.0772914383, + "ci_lo": 0.0053981497919, + "ci_hi": 1.0251625066, + "n_obs": 30, + "p_value": 0.047792249305 + } } }, { @@ -1612,6 +1857,26 @@ "n_obs": 30, "p_value": 8.1857621596e-11 } + }, + "placebo_horizons": { + "-2": { + "beta": 0.45724120253, + "se": 0.18968126937, + "t": 2.4105764583, + "ci_lo": 0.067345769371, + "ci_hi": 0.84713663568, + "n_obs": 30, + "p_value": 0.023295538059 + }, + "-1": { + "beta": -0.034998720913, + "se": 0.22412500402, + "t": -0.15615714572, + "ci_lo": -0.4956942646, + "ci_hi": 0.42569682278, + "n_obs": 30, + "p_value": 0.87711532865 + } } } ] diff --git a/diff_diff/chaisemartin_dhaultfoeuille.py b/diff_diff/chaisemartin_dhaultfoeuille.py index 92ba1476..a1485efd 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille.py +++ b/diff_diff/chaisemartin_dhaultfoeuille.py @@ -980,14 +980,25 @@ def fit( heterogeneity : str, optional Column name for a time-invariant covariate to test for heterogeneous effects (Web Appendix Section 1.5, Lemma 7). - Partial implementation: post-treatment regressions only - (no placebo regressions or joint null test). Cannot be - combined with ``controls``, ``trends_linear``, or - ``trends_nonparam``. Requires ``L_max >= 1``. Under - ``by_path`` / ``paths_of_interest``, per-path - heterogeneity coefficients also surface on - ``results.path_heterogeneity_effects`` and on - ``to_dataframe(level="by_path")`` via ``het_*`` columns. + Per-horizon OLS regressions are computed for forward + horizons (1..L_max), and ALSO for backward (placebo) + horizons (-1..-L_max) when ``placebo=True`` is set + (post-2026-05-15: per-path placebo predict_het R-parity + against ``did_multiplegt_dyn(by_path, predict_het, placebo)``). + Joint Wald F-test across rows is NOT computed + (per-horizon inference only). Cannot be combined with + ``controls``, ``trends_linear``, or ``trends_nonparam``. + Requires ``L_max >= 1``. Under ``by_path`` / + ``paths_of_interest``, per-path heterogeneity coefficients + also surface on ``results.path_heterogeneity_effects`` and + on ``to_dataframe(level="by_path")`` via ``het_*`` columns + (positive AND negative-horizon rows populated when + ``placebo=True``). Under ``survey_design``, backward- + horizon (placebo) heterogeneity is NOT computed (the pre- + period Binder TSL cell allocator is deferred to a follow- + up methodology PR); a ``UserWarning`` fires at fit-time + and forward-horizon heterogeneity continues to compute + normally. design2 : bool, default=False If ``True``, identify and report switch-in/switch-out (Design-2) groups. Convenience wrapper (descriptive summary, @@ -3875,6 +3886,28 @@ def fit( # event_study_effects but the het test uses level outcomes. Y_het = Y_mat if not _is_trends_linear else y_pivot.to_numpy() N_het = N_mat_orig + # Survey + placebo + heterogeneity: backward-horizon + # (placebo) predict_het is deferred under survey designs + # because the Binder TSL cell-period allocator's REGISTRY + # justification is tied to post-period attribution + # (pre-period cell allocator derivation gap). Compute + # forward-horizon heterogeneity only and warn so the user + # knows backward-horizon results are NOT silently produced. + _het_placebo = L_max if self.placebo else 0 + if _het_placebo > 0 and _obs_survey_info is not None: + warnings.warn( + "survey_design + heterogeneity + placebo=True: " + "backward-horizon (placebo) predict_het is not yet " + "supported under survey designs (pre-period cell " + "allocator deferred — see REGISTRY.md " + "ChaisemartinDHaultfoeuille placebo predict_het " + "Note). Forward-horizon predict_het is computed " + "normally. Drop placebo, drop heterogeneity, or " + "drop survey_design to silence this warning.", + UserWarning, + stacklevel=2, + ) + _het_placebo = 0 heterogeneity_effects = _compute_heterogeneity_test( Y_mat=Y_het, N_mat=N_het, @@ -3884,6 +3917,7 @@ def fit( T_g=T_g_arr, X_het=X_het, L_max=L_max, + placebo=_het_placebo, alpha=self.alpha, rank_deficient_action=self.rank_deficient_action, group_ids_order=np.array(all_groups), @@ -3902,6 +3936,17 @@ def fit( if heterogeneity is not None and ( self.by_path is not None or self.paths_of_interest is not None ): + # Per-path placebo predict_het under survey: same gating + # rationale as the global call above. The global call's + # warning already fired (heterogeneity is computed at both + # surfaces in the same fit() flow), so per-path skips + # silently — no duplicate warning. Forward-horizon per-path + # predict_het + survey continues to work via the existing + # _compute_path_heterogeneity_test → _compute_heterogeneity_test + # forward path. + _path_het_placebo = L_max if self.placebo else 0 + if _path_het_placebo > 0 and _obs_survey_info is not None: + _path_het_placebo = 0 path_heterogeneity_effects = _compute_path_heterogeneity_test( Y_mat=Y_het, N_mat=N_het, @@ -3914,6 +3959,7 @@ def fit( by_path=self.by_path, paths_of_interest=self.paths_of_interest, D_mat=D_mat, + placebo=_path_het_placebo, alpha=self.alpha, rank_deficient_action=self.rank_deficient_action, group_ids_order=np.array(all_groups), @@ -4903,6 +4949,7 @@ def _compute_heterogeneity_test( T_g: np.ndarray, X_het: np.ndarray, L_max: int, + placebo: int = 0, alpha: float = 0.05, rank_deficient_action: str = "warn", group_ids_order: Optional[np.ndarray] = None, @@ -4918,6 +4965,15 @@ def _compute_heterogeneity_test( variance-weighted average of effect differences. Standard OLS inference is valid - no need to account for DID estimation error. + Forward horizons ``l in 1..L_max`` use ``out_idx = F_g - 1 + l`` (post- + period). Backward placebo horizons ``l in -1..-placebo`` use the same + construction with negated ``l``, so ``out_idx = F_g - 1 + l < F_g - 1`` + (pre-period). Lemma 7's OLS inference is symmetric — same regression + structure with negated time index — and mirrors R's + ``did_multiplegt_dyn(..., predict_het=list("X", c(-1)), placebo=N)`` + per-horizon dispatcher (``DIDmultiplegtDYN:::did_multiplegt_main`` + placebo block at the ``effect = matrix(-i, ...)`` rbind site). + Parameters ---------- Y_mat : np.ndarray, shape (n_groups, n_periods) @@ -4992,7 +5048,14 @@ def _compute_heterogeneity_test( results: Dict[int, Dict[str, Any]] = {} # Survey setup (once, before horizon loop). When inactive, df_s=None and - # the existing plain-OLS path runs unchanged. + # the existing plain-OLS path runs unchanged. Forward-horizon survey + # heterogeneity is supported; the per-iteration backward-horizon gate + # below catches the unsupported case (survey + l_h < 0) defensively. + # Under normal `fit()` flow the global / per-path call sites pass + # `placebo=0` when survey is active (with a UserWarning), so the + # backward iterations are never reached and the gate is a defensive + # backstop only — direct callers passing `placebo > 0 + survey` get + # the explicit raise. use_survey = obs_survey_info is not None and group_ids_order is not None if use_survey: from diff_diff.survey import ( @@ -5032,7 +5095,31 @@ def _compute_heterogeneity_test( else: df_s = None - for l_h in range(1, L_max + 1): + # Iterate forward (1..L_max) and backward placebo (-1..-placebo) + # horizons in a single loop. Backward horizons emit "placebo + # predict_het" rows matching R's did_multiplegt_dyn(..., predict_het, + # placebo) per-by_level dispatcher (R's placebo block at the + # `effect = matrix(-i, ...)` rbind site). + horizons_to_compute = list(range(1, L_max + 1)) + list(range(-1, -placebo - 1, -1)) + for l_h in horizons_to_compute: + # Per-iteration survey gate (defensive backstop). Forward + # iterations under survey work; backward iterations raise + # because the Binder TSL cell-period allocator's REGISTRY + # justification is tied to post-period attribution. fit() gates + # this upstream by passing `placebo=0` when survey is active + # (with a UserWarning when the user requested placebo het), so + # the raise is reachable only via direct callers. + if use_survey and l_h < 0: + raise NotImplementedError( + "survey_design with backward-horizon (placebo) " + "predict_het is not yet supported. The Binder TSL " + "cell-period allocator is derived for post-period " + "attribution (REGISTRY.md ChaisemartinDHaultfoeuille " + "survey IF expansion Note); pre-period (l < 0) " + "attribution requires a separate methodology " + "derivation. Forward-horizon predict_het + " + "survey_design is supported." + ) # Eligible switchers at this horizon (same logic as multi-horizon DID) eligible = [] dep_var = [] @@ -5051,6 +5138,14 @@ def _compute_heterogeneity_test( continue if ref_idx < 0: continue + if out_idx < 0: + # Backward horizons can push out_idx below 0 when + # F_g - 1 + l_h < 0 (e.g., F_g=2, l_h=-2). numpy + # negative indexing on N_mat[g, out_idx] would silently + # wrap to a tail period rather than skip — guard + # explicitly so the eligibility filter is correct for + # any l_h sign. + continue if N_mat[g, ref_idx] <= 0 or N_mat[g, out_idx] <= 0: continue if T_g[g] < out_idx: @@ -5109,7 +5204,15 @@ def _compute_heterogeneity_test( continue if not use_survey: - # Plain OLS path (unchanged): standard inference per Lemma 7. + # Plain OLS path: standard inference per Lemma 7. df is the + # pre-drop column count (n_obs - n_params); matches R's + # did_multiplegt_dyn(predict_het=...) which uses the + # t-distribution with df = n - k from the OLS regression + # (DIDmultiplegtDYN:::did_multiplegt_main `t_stat <- qt(0.975, + # df.residual(model))` site). Under near-rank-deficient + # designs that solve_ols retains rather than NaN-out, n_params + # may exceed actual rank; see TODO row for the deferred + # rank-tracking follow-up. coefs, _residuals, vcov = solve_ols( design, dep_arr, @@ -5120,7 +5223,7 @@ def _compute_heterogeneity_test( se_het = float("nan") if vcov is not None and np.isfinite(vcov[1, 1]) and vcov[1, 1] > 0: se_het = float(np.sqrt(vcov[1, 1])) - t_stat, p_val, ci = safe_inference(beta_het, se_het, alpha=alpha, df=None) + t_stat, p_val, ci = safe_inference(beta_het, se_het, alpha=alpha, df=n_obs - n_params) else: # Survey-aware path: WLS with per-group weights + TSL IF variance. W_elig = W_g_all[eligible] @@ -6510,6 +6613,7 @@ def _compute_path_heterogeneity_test( by_path: Optional[int], paths_of_interest: Optional[List[Tuple[int, ...]]], D_mat: np.ndarray, + placebo: int = 0, alpha: float = 0.05, rank_deficient_action: str = "warn", group_ids_order: Optional[np.ndarray] = None, @@ -6528,6 +6632,13 @@ def _compute_path_heterogeneity_test( predict_het=...)`` on each path-restricted subsample, which is exactly what this helper does in Python. + When ``placebo > 0``, ``_compute_heterogeneity_test`` also emits + per-path heterogeneity on backward (placebo) horizons. Inner-dict keys + are negative ints ``-1..-placebo`` to match the global + ``heterogeneity_effects`` convention and the existing per-path + placebo allocator at ``_compute_path_placebos`` (negative keys for + unified ``{**positive, **negative}`` dict merging downstream). + The ``_enumerate_treatment_paths`` call here re-derives the path enumeration (already computed elsewhere in fit() for ``path_effects``). The call is wrapped in ``warnings.catch_warnings()`` to suppress @@ -6567,6 +6678,7 @@ def _compute_path_heterogeneity_test( T_g=T_g, X_het=X_het, L_max=L_max, + placebo=placebo, alpha=alpha, rank_deficient_action=rank_deficient_action, group_ids_order=group_ids_order, diff --git a/diff_diff/chaisemartin_dhaultfoeuille_results.py b/diff_diff/chaisemartin_dhaultfoeuille_results.py index b19a3bfe..b4c48ea9 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille_results.py +++ b/diff_diff/chaisemartin_dhaultfoeuille_results.py @@ -1279,7 +1279,11 @@ def _render_heterogeneity_section(self, lines: List[str], width: int, thin: str) lines.extend( [ thin, - "Note: Post-treatment regressions only (no placebo/joint test).", + "Note: Per-horizon regressions only (no joint F-test). " + "Negative l = placebo (backward) horizon when " + "placebo=True. Under survey_design, only forward " + "horizons are computed (backward-horizon survey " + "heterogeneity is deferred — see REGISTRY note).", "", ] ) @@ -1550,11 +1554,20 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: (always-present, NaN-when-None — same convention as ``cband_*``). The ``het_*`` columns surface the per-path heterogeneity coefficient (Web Appendix Section 1.5, - Lemma 7) when ``heterogeneity=""`` is also set; - populated for positive-horizon rows and NaN for placebo - rows / non-heterogeneity fits / the requested-but-empty - fallback DataFrame (always-present, NaN-when-None — same - convention as ``cband_*`` and ``cumulated_*``). + Lemma 7) when ``heterogeneity=""`` is also set. + Populated for positive-horizon (forward) rows whenever + heterogeneity is requested, AND for negative-horizon + (placebo) rows when ``placebo=True`` is also set + (post-2026-05-15: per-path placebo predict_het R-parity + against ``did_multiplegt_dyn(by_path, predict_het, placebo)``). + NaN for non-heterogeneity fits / the requested-but-empty + fallback DataFrame, AND for placebo rows under + ``survey_design`` (forward-only fallback — backward-horizon + survey predict_het is deferred until the pre-period cell + allocator is derived; a ``UserWarning`` fires at fit-time + when ``survey_design + placebo + heterogeneity`` are + co-set). Always-present, NaN-when-None — same convention + as ``cband_*`` and ``cumulated_*``. Returns ------- @@ -1872,6 +1885,16 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: # path placebo cumulation surface (placebo under # trends_lin returns RAW per-horizon values per R). ph_cband = ph_entry.get("cband_conf_int", (np.nan, np.nan)) + # Per-path placebo heterogeneity (TODO #422). R- + # verified: did_multiplegt_dyn(..., by_path, + # predict_het, placebo) emits per-path predict_het + # rows on backward (negative) horizons. Negative + # `lag_key` indexes into `path_het` to look up the + # placebo het entry; absent key (placebo > 0 but + # this (path, lag) is rank-deficient or has < 3 + # eligible groups) -> NaN columns. + ph_het_entry = path_het.get(lag_key, {}) if path_het else {} + ph_het_ci = ph_het_entry.get("conf_int", (np.nan, np.nan)) rows.append( { "path": path, @@ -1889,19 +1912,12 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "cband_upper": ph_cband[1] if ph_cband else np.nan, "cumulated_effect": np.nan, "cumulated_se": np.nan, - # Heterogeneity is forward-only in this release. - # Per-path placebo heterogeneity is not exposed - # yet; R may emit placebo het rows under - # did_multiplegt_dyn(..., by_path, predict_het) - # but R-parity for that surface has not been - # validated, so we emit NaN on placebo rows - # rather than claim parity. See REGISTRY note. - "het_beta": np.nan, - "het_se": np.nan, - "het_t_stat": np.nan, - "het_p_value": np.nan, - "het_conf_int_lower": np.nan, - "het_conf_int_upper": np.nan, + "het_beta": ph_het_entry.get("beta", np.nan), + "het_se": ph_het_entry.get("se", np.nan), + "het_t_stat": ph_het_entry.get("t_stat", np.nan), + "het_p_value": ph_het_entry.get("p_value", np.nan), + "het_conf_int_lower": ph_het_ci[0] if ph_het_ci else np.nan, + "het_conf_int_upper": ph_het_ci[1] if ph_het_ci else np.nan, } ) for l_h in sorted(horizons.keys()): diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index dcf1d5f4..5bbe74c2 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -242,8 +242,8 @@ ChaisemartinDHaultfoeuille( placebo: bool = True, # Auto-compute single-lag placebo twfe_diagnostic: bool = True, # Auto-compute Theorem 1 TWFE decomposition drop_larger_lower: bool = True, # Drop multi-switch groups (matches R DIDmultiplegtDYN) - by_path: int | None = None, # Top-k per-path event study; requires drop_larger_lower=False, L_max>=1; supports binary or integer-coded discrete D (D in Z); composes with survey_design (analytical TSL + replicate-weight; multiplier bootstrap n_bootstrap>0 still gated under survey) and heterogeneity (per-path predict_het, mirrors R did_multiplegt_dyn(..., by_path, predict_het)); mutex with paths_of_interest - paths_of_interest: list[tuple[int, ...]] | None = None, # User-specified path subset, alternative to by_path=k (Python-only API; mutex with by_path; composes with survey_design and heterogeneity same as by_path=k) + by_path: int | None = None, # Top-k per-path event study; requires drop_larger_lower=False, L_max>=1; supports binary or integer-coded discrete D (D in Z); composes with survey_design (analytical TSL + replicate-weight; multiplier bootstrap n_bootstrap>0 still gated under survey) and heterogeneity (per-path predict_het, mirrors R did_multiplegt_dyn(..., by_path, predict_het); composes with placebo=True for per-path placebo predict_het on backward horizons — survey_design + placebo + heterogeneity warns and falls back to forward-horizon-only heterogeneity until the pre-period cell allocator is derived); mutex with paths_of_interest + paths_of_interest: list[tuple[int, ...]] | None = None, # User-specified path subset, alternative to by_path=k (Python-only API; mutex with by_path; composes with survey_design, heterogeneity, and placebo same as by_path=k) rank_deficient_action: str = "warn", # Used by TWFE diagnostic OLS ) ``` diff --git a/docs/api/chaisemartin_dhaultfoeuille.rst b/docs/api/chaisemartin_dhaultfoeuille.rst index 10c3533d..cf7f13d4 100644 --- a/docs/api/chaisemartin_dhaultfoeuille.rst +++ b/docs/api/chaisemartin_dhaultfoeuille.rst @@ -30,7 +30,16 @@ survey + by_path remains gated; no R parity since R and ``paths_of_interest`` also compose with ``heterogeneity=""``: per-path heterogeneity coefficient surfaces on ``results.path_heterogeneity_effects`` (mirrors R -``did_multiplegt_dyn(..., by_path, predict_het)`` per-by_level). +``did_multiplegt_dyn(..., by_path, predict_het)`` per-by_level). When +combined with ``placebo=True``, heterogeneity is also computed on +backward (placebo) horizons and surfaced under negative-int keys — +both globally on ``results.heterogeneity_effects[-l]`` and per-path +on ``results.path_heterogeneity_effects[path][-l]``; +``to_dataframe(level="by_path")`` placebo rows have populated ``het_*`` +columns. ``survey_design + placebo + heterogeneity`` emits a +``UserWarning`` at fit-time and falls back to forward-horizon-only +heterogeneity until the pre-period cell allocator is derived; forward- +horizon ``predict_het + survey_design`` continues to work unchanged. The estimator: diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index ba7d4374..099b8059 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -634,13 +634,13 @@ The guard is fired by `_survey_se_from_group_if` (analytical and replicate) and - **Note (Phase 3 state-set trends):** Implements state-set-specific trends from Web Appendix Section 1.4 (Assumptions 13-14). Restricts the control pool for each switcher to groups in the same set (e.g., same state in county-level data). The restriction applies in all four DID/IF paths: `_compute_multi_horizon_dids()`, `_compute_per_group_if_multi_horizon()`, `_compute_multi_horizon_placebos()`, and `_compute_per_group_if_placebo_horizon()`. Cohort structure stays as `(D_{g,1}, F_g, S_g)` triples (does not incorporate set membership). Set membership must be time-invariant per group. **Note on Assumption 14 (common support):** The paper requires a common last-untreated period across sets (`T_u^s` equal for all `s`). This implementation does NOT enforce Assumption 14 up front. Instead, when within-set controls are exhausted at a given horizon (because a set has shorter untreated support than others), the affected switcher/horizon pairs are silently excluded via the existing empty-control-pool mechanism. This means `N_l` may be smaller under `trends_nonparam` than without it, and the effective estimand is trimmed to the within-set support at each horizon. The existing multi-horizon A11 warning fires when exclusions occur. Activated via `trends_nonparam="state_column"` in `fit()`. -- **Note (Phase 3 heterogeneity testing - partial implementation):** Partial implementation of the heterogeneity test from Web Appendix Section 1.5 (Assumption 15, Lemma 7). Computes post-treatment saturated OLS regressions of `S_g * (Y_{g, F_g-1+l} - Y_{g, F_g-1})` on a time-invariant covariate `X_g` plus cohort indicator dummies. Standard OLS inference is valid (paper shows no DID error correction needed). **Deviation from R `predict_het`:** R's full `predict_het` option additionally computes placebo regressions and a joint null test, and disallows combination with `controls`. This implementation provides only post-treatment regressions. **Rejected combinations:** `controls` (matching R), `trends_linear` (heterogeneity test uses raw level changes, incompatible with second-differenced outcomes), and `trends_nonparam` (heterogeneity test does not thread state-set control-pool restrictions). Results stored in `results.heterogeneity_effects`. Activated via `heterogeneity="covariate_column"` in `fit()`. **Note (survey support):** Under `survey_design`, heterogeneity uses WLS with per-group weights `W_g = sum of obs-level survey weights in group g`, and the group-level WLS coefficient influence function is `ψ_g[X] = inv(X'WX)[1,:] @ x_g * W_g * r_g`. The group-level IF is then attributed to observation level via **one of two allocators, chosen by variance helper** so each path preserves byte-identity for its aggregation rule: (1) **Binder TSL** (`compute_survey_if_variance`) uses the **cell-period single-cell allocator** — at each horizon `l_h`, `ψ_g` is assigned in full to the post-period cell `(g, out_idx)` with `out_idx = first_switch_idx[g] - 1 + l_h` and expanded as `ψ_i = ψ_g * (w_i / W_{g, out_idx})` for obs in that cell, zero elsewhere (matches the DID_l post-period convention in the Survey IF expansion Note below). Under PSU=group per-observation distribution differs from the legacy `ψ_i = ψ_g * (w_i / W_g)`, but PSU-level aggregates telescope to the same `ψ_g` — so Binder TSL variance is byte-identical to the pre-cell-period release under PSU=group. Under within-group-varying PSU mass lands in the post-period PSU of the transition, which is what Binder TSL needs. An **empty post-period cell under zero-weight obs** (all obs at `(g, out_idx)` have `w_i = 0` despite `N > 0`) drops the group's contribution, matching the ATT cell allocator's convention; the pre-cell-period path diverged here by redistributing mass to other cells of the group. (2) **Rao-Wu replicate** (`compute_replicate_if_variance`) uses the **legacy group-level allocator** `ψ_i = ψ_g * (w_i / W_g)`. Replicate variance computes `θ_r = sum_i ratio_ir * ψ_i` at the observation level, so moving `ψ_g` mass onto the post-period cell only would silently change the replicate SE whenever a replicate column's ratios vary within group (the library accepts arbitrary per-row replicate matrices, not just PSU-aligned ones). Keeping the legacy allocator on this branch preserves byte-identity of replicate SE across every previously-supported fit; replicate + within-group-varying PSU is unreachable by construction (`SurveyDesign` rejects `replicate_weights` combined with explicit `strata/psu/fpc`). Inference uses the t-distribution with `df_survey` when provided. Under rank deficiency (any regression coefficient dropped by `solve_ols`'s R-style drop), all inference fields return NaN (conservative, matches the NaN-consistent contract). **Library extension (replicate weights):** Under a replicate-weight design (BRR/Fay/JK1/JKn/SDR), the heterogeneity regression dispatches to `compute_replicate_if_variance` (Rao-Wu weight-ratio rescaling) instead of the Binder TSL formula. The effective df is the shared `min(resolved_survey.df_survey, min(n_valid_across_sites) - 1)` used by the rest of the dCDH surfaces; if the base `df_survey` is undefined (QR-rank ≤ 1), heterogeneity inference is NaN regardless of the local `n_valid_het` (matching the dCDH top-level contract — per-site `n_valid` cannot rescue a rank-deficient design). **Library extension:** R `DIDmultiplegtDYN::predict_het` does not natively support survey weights. **Scope note (bootstrap):** Heterogeneity inference is analytical (no bootstrap path). When `n_bootstrap > 0` is combined with `heterogeneity=`, the main ATT surfaces receive bootstrap SE/CI (via the cell-level wild PSU bootstrap described in the survey + bootstrap contract Note below) while `heterogeneity_effects` continues to use the Binder TSL / Rao-Wu analytical SE described above. No gate; the two inference paths are independent. +- **Note (Phase 3 heterogeneity testing - partial implementation):** Partial implementation of the heterogeneity test from Web Appendix Section 1.5 (Assumption 15, Lemma 7). Computes post-treatment saturated OLS regressions of `S_g * (Y_{g, F_g-1+l} - Y_{g, F_g-1})` on a time-invariant covariate `X_g` plus cohort indicator dummies. Standard OLS inference is valid (paper shows no DID error correction needed). **Deviation from R `predict_het`:** Python now matches R on per-horizon placebo regressions when the user sets `placebo=True` together with `heterogeneity=` (post-2026-05-15; see "Placebo predict_het" sub-note below for the full contract). The remaining gap is the joint null F-test that R aggregates across all `predict_het` rows — Python emits per-horizon `t_stat` / `p_value` / `conf_int` only and does NOT compute a joint Wald test across forward + placebo coefficients (tracked at REGISTRY note's "Per-horizon regressions only (no joint F-test)" rendering line). R also disallows combination with `controls`, which Python continues to enforce as an explicit `ValueError`. **Rejected combinations:** `controls` (matching R), `trends_linear` (heterogeneity test uses raw level changes, incompatible with second-differenced outcomes), and `trends_nonparam` (heterogeneity test does not thread state-set control-pool restrictions). Results stored in `results.heterogeneity_effects`. Activated via `heterogeneity="covariate_column"` in `fit()`. **Note (survey support):** Under `survey_design`, heterogeneity uses WLS with per-group weights `W_g = sum of obs-level survey weights in group g`, and the group-level WLS coefficient influence function is `ψ_g[X] = inv(X'WX)[1,:] @ x_g * W_g * r_g`. The group-level IF is then attributed to observation level via **one of two allocators, chosen by variance helper** so each path preserves byte-identity for its aggregation rule: (1) **Binder TSL** (`compute_survey_if_variance`) uses the **cell-period single-cell allocator** — at each horizon `l_h`, `ψ_g` is assigned in full to the post-period cell `(g, out_idx)` with `out_idx = first_switch_idx[g] - 1 + l_h` and expanded as `ψ_i = ψ_g * (w_i / W_{g, out_idx})` for obs in that cell, zero elsewhere (matches the DID_l post-period convention in the Survey IF expansion Note below). Under PSU=group per-observation distribution differs from the legacy `ψ_i = ψ_g * (w_i / W_g)`, but PSU-level aggregates telescope to the same `ψ_g` — so Binder TSL variance is byte-identical to the pre-cell-period release under PSU=group. Under within-group-varying PSU mass lands in the post-period PSU of the transition, which is what Binder TSL needs. An **empty post-period cell under zero-weight obs** (all obs at `(g, out_idx)` have `w_i = 0` despite `N > 0`) drops the group's contribution, matching the ATT cell allocator's convention; the pre-cell-period path diverged here by redistributing mass to other cells of the group. (2) **Rao-Wu replicate** (`compute_replicate_if_variance`) uses the **legacy group-level allocator** `ψ_i = ψ_g * (w_i / W_g)`. Replicate variance computes `θ_r = sum_i ratio_ir * ψ_i` at the observation level, so moving `ψ_g` mass onto the post-period cell only would silently change the replicate SE whenever a replicate column's ratios vary within group (the library accepts arbitrary per-row replicate matrices, not just PSU-aligned ones). Keeping the legacy allocator on this branch preserves byte-identity of replicate SE across every previously-supported fit; replicate + within-group-varying PSU is unreachable by construction (`SurveyDesign` rejects `replicate_weights` combined with explicit `strata/psu/fpc`). Inference uses the t-distribution with `df_survey` when provided. Under rank deficiency (any regression coefficient dropped by `solve_ols`'s R-style drop), all inference fields return NaN (conservative, matches the NaN-consistent contract). **Library extension (replicate weights):** Under a replicate-weight design (BRR/Fay/JK1/JKn/SDR), the heterogeneity regression dispatches to `compute_replicate_if_variance` (Rao-Wu weight-ratio rescaling) instead of the Binder TSL formula. The effective df is the shared `min(resolved_survey.df_survey, min(n_valid_across_sites) - 1)` used by the rest of the dCDH surfaces; if the base `df_survey` is undefined (QR-rank ≤ 1), heterogeneity inference is NaN regardless of the local `n_valid_het` (matching the dCDH top-level contract — per-site `n_valid` cannot rescue a rank-deficient design). **Library extension:** R `DIDmultiplegtDYN::predict_het` does not natively support survey weights. **Scope note (bootstrap):** Heterogeneity inference is analytical (no bootstrap path). When `n_bootstrap > 0` is combined with `heterogeneity=`, the main ATT surfaces receive bootstrap SE/CI (via the cell-level wild PSU bootstrap described in the survey + bootstrap contract Note below) while `heterogeneity_effects` continues to use the Binder TSL / Rao-Wu analytical SE described above. No gate; the two inference paths are independent. - **Note (HonestDiD integration):** HonestDiD sensitivity analysis (Rambachan & Roth 2023) is available on the placebo + event study surface via `honest_did=True` in `fit()` or `compute_honest_did(results)` post-hoc. **Library extension:** dCDH HonestDiD uses `DID^{pl}_l` placebo estimates as pre-period coefficients rather than standard event-study pre-treatment coefficients. The Rambachan-Roth restrictions bound violations of the parallel trends assumption underlying the dCDH placebo estimand; interpretation differs from canonical event-study HonestDiD. A `UserWarning` is emitted at runtime. Uses diagonal variance (no full VCV available for dCDH). Relative magnitudes (DeltaRM) with Mbar=1.0 is the default when called from `fit()`, targeting the equal-weight average over all post-treatment horizons (`l_vec=None`). R's HonestDiD defaults to the first post/on-impact effect; use `compute_honest_did(results, ...)` with a custom `l_vec` to match that behavior. When `trends_linear=True`, bounds apply to the second-differenced estimand (parallel trends in first differences). Requires `L_max >= 1` for multi-horizon placebos. Gaps in the horizon grid from `trends_nonparam` support-trimming are handled by filtering to the largest consecutive block and warning. - **Note (Phase 3 Design-2 switch-in/switch-out):** Convenience wrapper for Web Appendix Section 1.6 (Assumption 16). Identifies groups with exactly 2 treatment changes (join then leave), reports switch-in and switch-out mean effects. This is a descriptive summary, not a full re-estimation with specialized control pools as described in the paper. **Always uses raw (unadjusted) outcomes** regardless of active `controls`, `trends_linear`, or `trends_nonparam` options - those adjustments apply to the main estimator surface but not to the Design-2 descriptive block. For full adjusted Design-2 estimation with proper control pools, the paper recommends "running the command on a restricted subsample and using `trends_nonparam` for the entry-timing grouping." Activated via `design2=True` in `fit()`, requires `drop_larger_lower=False` to retain 2-switch groups. -- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** combinations with `design2` and `honest_did` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs); `heterogeneity` is supported per-path — see the **Per-path heterogeneity testing** paragraph below. `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `survey_design` is supported under analytical Binder TSL and replicate-weight bootstrap — see the **Per-path survey-design SE** paragraph below; multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` remains gated. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper, cumulated_effect, cumulated_se, het_beta, het_se, het_t_stat, het_p_value, het_conf_int_lower, het_conf_int_upper]` (the `cband_*` columns are added by the joint sup-t Note below, populated for positive-horizon rows of paths with a finite sup-t crit and NaN otherwise; the `cumulated_*` columns are added by the per-path linear-trends Note below, populated for positive-horizon rows when `trends_linear=True` is set and NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). **Per-path linear-trends DID^{fd}:** when `trends_linear=True` is set with `by_path=k`, the first-differencing transform at `chaisemartin_dhaultfoeuille.py:1599-1630` runs once globally BEFORE path enumeration (replaces `Y_mat` with `Z_mat = Y_t - Y_{t-1}` and shrinks the time axis by one), so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path cumulated level effects `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)` per the existing parity test pivot at `tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409`) surface on the new `results.path_cumulated_event_study[path][l]` field — a per-group running sum of `DID^{fd}_{g, l'}` averaged over the path's switchers eligible at horizon `l`, mirroring the global `linear_trends_effects` cumulation logic at `chaisemartin_dhaultfoeuille.py:3340-3398`. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs from `path_effects[path]["horizons"][l]["se"]`, NaN-consistent: any non-finite component yields a NaN cumulated SE). **Post-bootstrap recomputation:** the cumulated layer is built AFTER the bootstrap propagation block at `chaisemartin_dhaultfoeuille.py:3034-3081` so it reads the FINAL post-bootstrap per-horizon SEs (mirrors the global `linear_trends_effects` placement). When `n_bootstrap > 0`, cumulated SE / t / p / CI are derived from bootstrap per-horizon SEs; when bootstrap produces non-finite SE (e.g., `n_bootstrap=1` degenerate distribution), the cumulated layer's full inference tuple is NaN per the library-wide NaN-on-invalid bootstrap contract. `to_dataframe(level="by_path")` exposes `cumulated_effect` and `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` always-present convention from PR #374). `summary()` renders a `Cumulated Level Effects (DID^{fd}, trends_linear)` sub-section under each per-path block. **Path enumeration uses the post-first-differenced `N_mat_fd`**: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning at line 1620 surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have exactly 2 pre-switch periods, which after first-differencing and the `time==1` filter leaves only 1 valid pre-window Z value. R's per-path full-pipeline call handles this single-pre-period regime differently from Python's global-then-disaggregate architecture, producing 30%+ relative divergence on point estimates for paths whose switchers include `F_g=3` (empirically observed on the parity fixture's earlier `F_g=3` variant). A separate `UserWarning` fires at fit-time when the panel includes any `F_g=3` switcher AND `by_path + trends_linear` is set, mirroring the `F_g < 3` exclusion warning. The shipped parity fixture (`single_baseline_multi_path_by_path_trends_lin`) restricts to `F_g >= 4` exclusively to avoid this regime; per-path R parity is asserted only there. **Placebo under `trends_linear` returns RAW per-horizon values** (no per-path placebo cumulation surface) — verified empirically against the existing `joiners_only_trends_lin` parity fixture: R's per-path Placebo_l matches Python's `path_placebo_event_study[path][-l]` (raw) bit-exactly under non-`by_path` trends_lin. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, trends_lin)` re-runs the full pipeline (including first-differencing) on each path's restricted subsample, so it operates on different switcher samples per path when switchers have different baseline values `D_{g,1}`. Python first-differences once globally before path enumeration. On single-baseline switcher panels the two architectures coincide; on multi-baseline switcher panels per-path point estimates can diverge — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R (mirroring the analogous `by_path + controls` warning). Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_lin=TRUE, placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` on the `single_baseline_multi_path_by_path_trends_lin` scenario (single-baseline + cohort-single-path + `F_g >= 4` DGP designed to eliminate the multi-baseline divergence, the cross-path cohort-sharing deviation, and the F_g=3 boundary case under R's per-path full-pipeline call). Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) on event horizons under those conditions; cumulated SE_RTOL is widened to `0.20` (vs `0.12` used for non-cumulated by_path parity) because the conservative upper-bound SE compounds the cross-path cohort-sharing deviation under summation. **Placebo parity is intentionally skipped for `trends_linear`**: R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces, producing a sign-and-magnitude divergence on paths whose switchers have minimal pre-window depth (e.g., `F_g=4` switchers). Placebo under `by_path + trends_linear` is exercised via internal regression in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` (finite values, bootstrap inheritance) but not pinned to R bit-by-bit. Cross-surface invariants (analytical + bootstrap + placebo + sup-t + `path_cumulated_event_study` + `to_dataframe` columns + `summary()` rendering) are regression-tested at `TestByPathTrendsLinear`. **Per-path state-set trends:** when `trends_nonparam="state_col"` is set with `by_path=k`, the set membership column is validated and stored once globally as `set_ids_arr` (time-invariance, NaN rejection, partition-coarseness checks unchanged from the non-by_path path). The `set_ids` parameter is threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. R does NOT first-difference and does NOT cumulate under `trends_nonparam` (unlike `trends_lin`); per-horizon `Effect_l` is a normal DID with set-restricted controls. Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_nonparam="state", placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam` on the `multi_path_reversible_by_path_trends_nonparam` scenario; per-path point estimates AND placebos match R bit-exactly (rtol ~1e-9), per-path SE matches within the Phase 2 envelope (~13% rtol observed). Cross-surface invariants are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsNonparam`. **Per-path non-binary treatment:** integer-coded discrete treatment (D in Z, e.g. ordinal {0, 1, 2}) is supported under `by_path=k` and `paths_of_interest`. Path tuples become integer-state tuples (`(0, 2, 2, 2)`) keyed bit-for-bit against R's comma-separated path strings (`"0,2,2,2"`) for D in {0..9}. Continuous D (e.g. `1.5`) raises `ValueError` at fit-time per the no-silent-failures contract — the existing `int(round(float(v)))` cast in `_enumerate_treatment_paths` is now defensive (no-op for integer-coded D). **Deviation from R for multi-character baseline states (D >= 10 or negative D):** R's `did_multiplegt_by_path` derives the per-path baseline via `path_index$baseline_XX <- substr(path_index$path, 1, 1)` (extracted 2026-05-03 via `Rscript -e 'cat(paste(deparse(DIDmultiplegtDYN:::did_multiplegt_by_path), collapse="\n"))'`), capturing only the first character of the comma-separated path string. For multi-character baselines this drops the rest of the value: for `path = "12,12,..."` it captures `"1"` instead of `"12"`; for `path = "-1,-1,..."` it captures `"-"` instead of `"-1"`. R's per-path control-pool subset is mis-allocated in both regimes. Python's tuple-key matching is correct in both — the per-path point estimates we compute are correct, R's per-path subset for the same path is buggy. The shipped R-parity scenarios stay in `D in {0, 1, 2}` to avoid the R bug; R-parity is asserted on that set at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathNonBinary` via the `multi_path_reversible_by_path_non_binary` scenario (78 switchers, 3 paths, single-baseline custom DGP, F_g >= 4). The string-encoding compatibility extends to all single-digit nonnegative D (`{0..9}`) since each value renders as a single character, but no R-parity scenario currently exercises D outside `{0, 1, 2}` — per-path point estimates match R bit-exactly (rtol ~1e-9 events; rtol+atol envelope for placebo near-zero values), SE inherits the documented cross-path cohort-sharing deviation (~5% rtol observed; SE_RTOL=0.15 envelope). Negative-integer treatment-state support (paths containing negative D values in non-baseline positions, e.g. `(0, -1, -1, -1)`) is regression-tested in Python only at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary::test_negative_integer_D_supported`; a dedicated regression for a negative-baseline path (e.g. `(-1, 0, 0, 0)`, the exact regime that would trigger R's `substr` bug) is deferred to a follow-up. Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary`. **Per-path survey-design SE** (analytical Binder TSL + replicate-weight bootstrap): under `by_path` / `paths_of_interest` + `survey_design`, the per-path per-horizon SE routes through `_survey_se_from_group_if` using the cell-period allocator. The per-path influence function `U_pp_l_path` is the per-period IF with non-path switcher-side contributions skipped — control contributions remain unchanged, matching the joiners/leavers IF convention from the **Per-path SE convention** paragraph above (the `switcher_subset_mask` zeroes the switcher row of the per-group IF, which trivially zeroes the corresponding row of the per-cell IF, preserving the row-sum identity `U_pp.sum(axis=1) == U`). The IF is cohort-recentered via `_cohort_recenter_per_period` and expanded to observations as `psi_i = U_pp[g_i, t_i] · (w_i / W_{g_i, t_i})`. Replicate-weight designs unconditionally route through the cell allocator (Class A contract, PR #323). Multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` raises `NotImplementedError` at fit-time — the survey-aware perturbation pivot for path-restricted IFs is methodologically underived and deferred to a future wave; the global non-by_path TSL multiplier bootstrap is unaffected and continues to ship. **Path-enumeration ranking is unweighted** under `survey_design`: top-k selection uses group cardinality (`path_to_count[p]` = number of groups), not population-weight mass — survey weights do not affect which paths are selected as "top-k". A weighted-ranking variant (sum of survey weights per path) is deferred until concrete demand. **`df_survey` propagation:** under replicate weights, every per-path per-horizon fit contributes an `n_valid` count to the shared `_replicate_n_valid_list` accumulator and the final `_effective_df_survey = min(...) - 1` reflects all per-path replicate fits. A post-call `_refresh_path_inference` helper re-runs `safe_inference` on every populated entry so `multi_horizon_inference`, `placebo_horizon_inference`, `path_effects`, and `path_placebos` all use the same final df after per-path appends complete. **Lonely-PSU policy is sample-wide, not per-path** — the `lonely_psu` policy (`remove`/`certainty`/`adjust`) operates on the full design-level PSU/strata structure, not on path-restricted subsamples. **Telescope invariant:** on a single-path panel where every switcher follows the same trajectory and `eligible_groups` matches between by_path and non-by_path, per-path SE equals the global non-by_path survey SE bit-exactly — pinned at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignTelescope::test_telescope_analytical_TSL`. **Deviation from R:** none — R `did_multiplegt_dyn` does not support survey weighting, so this is a Python-only methodology extension (no R parity available; no R parity test class). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignAnalytical` covering analytical SE, replicate-weight SE, the `n_bootstrap` gate, the global anti-regression, per-path placebos, `trends_linear` composition, and unobserved-path warnings under survey. **Per-path heterogeneity testing** (analytical OLS / WLS + survey-aware Binder TSL + replicate-weight): under `by_path` / `paths_of_interest` + `heterogeneity=""`, the per-path per-horizon coefficient `beta_X^path_l` is computed by re-running `_compute_heterogeneity_test` on the path-restricted switcher subsample. The path filter (`path_groups: Optional[Set[int]]`) restricts eligibility to switchers ON path `p` inside the inner regression; the variance machinery (HC1-robust OLS vcov for non-survey via `solve_ols(..., return_vcov=True)` (`vcov_type="hc1"` default), WLS-on-pweights with cell-period IF allocator for analytical Binder TSL, group-level allocator for Rao-Wu replicate) is unchanged from the global heterogeneity path. **Cohort dummies absorb baseline by construction** — the cohort key `(D_{g,1}, F_g, S_g)` includes baseline, so multi-baseline switcher panels do not produce R-divergence (unlike `controls` / `trends_linear`); no parallel `UserWarning` is emitted. **R parity:** matches `did_multiplegt_dyn(..., by_path, predict_het)` per-by_level on the `multi_path_reversible_by_path_predict_het` scenario for `beta`, `se`, `t_stat`, and `n_obs` (`BETA_RTOL = 1e-6` on `beta`, `SE_RTOL = 1e-5` on `se` / `t_stat`; the SE tolerance is one decade looser than `BETA_RTOL` to absorb the small OLS denominator-and-cohort-recentering numerical drift observed on this fixture; `n_obs` matches exactly). Inherits the same tolerances as the new global `multi_path_reversible_predict_het` scenario (`TestDCDHDynRParityHeterogeneity`) since the per-path R call is `did_multiplegt_main(..., predict_het=...)` per path-restricted subsample with no additional numerical loss. **Deviation from R (heterogeneity inference critical value):** `p_value` and `conf_int` are NOT pinned to R because Python's `safe_inference(..., df=None)` uses the normal Z critical value (`~1.96` at `alpha=0.05`) while R uses the t-distribution with df = n - k from the OLS regression. The structural deviation produces ~0.1-2% rtol gaps on CI bounds and p-values that exceed `SE_RTOL`; pinning would either require a separately documented inference-tolerance constant or a finite-df switch in the Python heterogeneity inference layer. The `t_stat = beta / se` field is distribution-invariant and pins normally. R's `dont_drop_larger_lower=TRUE` is set in both fixture scenarios to match the Python `drop_larger_lower=False` requirement. **Survey composition:** inherits from the **Per-path survey-design SE** paragraph above — analytical Binder TSL routes through `_survey_se_from_group_if`'s cell-period allocator on the post-period of the transition; replicate-weights route through the group-level allocator. Multiplier bootstrap (`n_bootstrap > 0`) under `by_path + heterogeneity + survey_design` inherits the existing per-path multiplier-bootstrap-survey gate. **`df_survey` propagation:** every per-(path, horizon) replicate-weight fit appends `n_valid` to the shared `_replicate_n_valid_list` accumulator; per-path heterogeneity inference is refreshed with the FINAL `_effective_df_survey(...)` in the R2 P1b refresh block (separate dedicated loop because the schema shape is `{path: {l: {...}}}` rather than `{path: {"horizons": {l: {...}}}}`). **Result schema:** `results.path_heterogeneity_effects: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]` keyed `{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}`. Empty-state contract mirrors `path_effects`: `None` when not requested, `{}` when requested but no path has eligible switchers. **DataFrame integration:** `to_dataframe(level="by_path")` adds always-present `het_*` columns (`het_beta`, `het_se`, `het_t_stat`, `het_p_value`, `het_conf_int_lower`, `het_conf_int_upper`), populated for positive-horizon rows when `heterogeneity` is set and NaN otherwise (mirrors the `cband_*` and `cumulated_*` always-present convention). Placebo rows (negative `horizon`) have NaN in `het_*` columns: per-path placebo heterogeneity is not exposed in this release. R's `did_multiplegt_dyn(..., by_path, predict_het)` forwards `predict_het` into each per-path `did_multiplegt_main` call along with `placebo`, so R may emit placebo heterogeneity rows we do not yet mirror - this is a Python-side deferral, NOT a verified R-parity. R parity for the per-path placebo heterogeneity surface is deferred to a follow-up (tracked in TODO.md). Regression test anchors: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathHeterogeneity` (gate dispatch, behavior, telescope-to-global on single-path panel, zero-signal anti-regression, multi-baseline UserWarning anti-regression, DataFrame integration, edge cases) + `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityHeterogeneity` (global anchor, FIRST `predict_het` parity baseline) + `::TestDCDHDynRParityByPathHeterogeneity` (per-path). +- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** combinations with `design2` and `honest_did` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs); `heterogeneity` is supported per-path — see the **Per-path heterogeneity testing** paragraph below. `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `survey_design` is supported under analytical Binder TSL and replicate-weight bootstrap — see the **Per-path survey-design SE** paragraph below; multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` remains gated. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper, cumulated_effect, cumulated_se, het_beta, het_se, het_t_stat, het_p_value, het_conf_int_lower, het_conf_int_upper]` (the `cband_*` columns are added by the joint sup-t Note below, populated for positive-horizon rows of paths with a finite sup-t crit and NaN otherwise; the `cumulated_*` columns are added by the per-path linear-trends Note below, populated for positive-horizon rows when `trends_linear=True` is set and NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). **Per-path linear-trends DID^{fd}:** when `trends_linear=True` is set with `by_path=k`, the first-differencing transform at `chaisemartin_dhaultfoeuille.py:1599-1630` runs once globally BEFORE path enumeration (replaces `Y_mat` with `Z_mat = Y_t - Y_{t-1}` and shrinks the time axis by one), so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path cumulated level effects `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)` per the existing parity test pivot at `tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409`) surface on the new `results.path_cumulated_event_study[path][l]` field — a per-group running sum of `DID^{fd}_{g, l'}` averaged over the path's switchers eligible at horizon `l`, mirroring the global `linear_trends_effects` cumulation logic at `chaisemartin_dhaultfoeuille.py:3340-3398`. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs from `path_effects[path]["horizons"][l]["se"]`, NaN-consistent: any non-finite component yields a NaN cumulated SE). **Post-bootstrap recomputation:** the cumulated layer is built AFTER the bootstrap propagation block at `chaisemartin_dhaultfoeuille.py:3034-3081` so it reads the FINAL post-bootstrap per-horizon SEs (mirrors the global `linear_trends_effects` placement). When `n_bootstrap > 0`, cumulated SE / t / p / CI are derived from bootstrap per-horizon SEs; when bootstrap produces non-finite SE (e.g., `n_bootstrap=1` degenerate distribution), the cumulated layer's full inference tuple is NaN per the library-wide NaN-on-invalid bootstrap contract. `to_dataframe(level="by_path")` exposes `cumulated_effect` and `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` always-present convention from PR #374). `summary()` renders a `Cumulated Level Effects (DID^{fd}, trends_linear)` sub-section under each per-path block. **Path enumeration uses the post-first-differenced `N_mat_fd`**: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning at line 1620 surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have exactly 2 pre-switch periods, which after first-differencing and the `time==1` filter leaves only 1 valid pre-window Z value. R's per-path full-pipeline call handles this single-pre-period regime differently from Python's global-then-disaggregate architecture, producing 30%+ relative divergence on point estimates for paths whose switchers include `F_g=3` (empirically observed on the parity fixture's earlier `F_g=3` variant). A separate `UserWarning` fires at fit-time when the panel includes any `F_g=3` switcher AND `by_path + trends_linear` is set, mirroring the `F_g < 3` exclusion warning. The shipped parity fixture (`single_baseline_multi_path_by_path_trends_lin`) restricts to `F_g >= 4` exclusively to avoid this regime; per-path R parity is asserted only there. **Placebo under `trends_linear` returns RAW per-horizon values** (no per-path placebo cumulation surface) — verified empirically against the existing `joiners_only_trends_lin` parity fixture: R's per-path Placebo_l matches Python's `path_placebo_event_study[path][-l]` (raw) bit-exactly under non-`by_path` trends_lin. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, trends_lin)` re-runs the full pipeline (including first-differencing) on each path's restricted subsample, so it operates on different switcher samples per path when switchers have different baseline values `D_{g,1}`. Python first-differences once globally before path enumeration. On single-baseline switcher panels the two architectures coincide; on multi-baseline switcher panels per-path point estimates can diverge — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R (mirroring the analogous `by_path + controls` warning). Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_lin=TRUE, placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` on the `single_baseline_multi_path_by_path_trends_lin` scenario (single-baseline + cohort-single-path + `F_g >= 4` DGP designed to eliminate the multi-baseline divergence, the cross-path cohort-sharing deviation, and the F_g=3 boundary case under R's per-path full-pipeline call). Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) on event horizons under those conditions; cumulated SE_RTOL is widened to `0.20` (vs `0.12` used for non-cumulated by_path parity) because the conservative upper-bound SE compounds the cross-path cohort-sharing deviation under summation. **Placebo parity is intentionally skipped for `trends_linear`**: R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces, producing a sign-and-magnitude divergence on paths whose switchers have minimal pre-window depth (e.g., `F_g=4` switchers). Placebo under `by_path + trends_linear` is exercised via internal regression in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` (finite values, bootstrap inheritance) but not pinned to R bit-by-bit. Cross-surface invariants (analytical + bootstrap + placebo + sup-t + `path_cumulated_event_study` + `to_dataframe` columns + `summary()` rendering) are regression-tested at `TestByPathTrendsLinear`. **Per-path state-set trends:** when `trends_nonparam="state_col"` is set with `by_path=k`, the set membership column is validated and stored once globally as `set_ids_arr` (time-invariance, NaN rejection, partition-coarseness checks unchanged from the non-by_path path). The `set_ids` parameter is threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. R does NOT first-difference and does NOT cumulate under `trends_nonparam` (unlike `trends_lin`); per-horizon `Effect_l` is a normal DID with set-restricted controls. Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_nonparam="state", placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam` on the `multi_path_reversible_by_path_trends_nonparam` scenario; per-path point estimates AND placebos match R bit-exactly (rtol ~1e-9), per-path SE matches within the Phase 2 envelope (~13% rtol observed). Cross-surface invariants are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsNonparam`. **Per-path non-binary treatment:** integer-coded discrete treatment (D in Z, e.g. ordinal {0, 1, 2}) is supported under `by_path=k` and `paths_of_interest`. Path tuples become integer-state tuples (`(0, 2, 2, 2)`) keyed bit-for-bit against R's comma-separated path strings (`"0,2,2,2"`) for D in {0..9}. Continuous D (e.g. `1.5`) raises `ValueError` at fit-time per the no-silent-failures contract — the existing `int(round(float(v)))` cast in `_enumerate_treatment_paths` is now defensive (no-op for integer-coded D). **Deviation from R for multi-character baseline states (D >= 10 or negative D):** R's `did_multiplegt_by_path` derives the per-path baseline via `path_index$baseline_XX <- substr(path_index$path, 1, 1)` (extracted 2026-05-03 via `Rscript -e 'cat(paste(deparse(DIDmultiplegtDYN:::did_multiplegt_by_path), collapse="\n"))'`), capturing only the first character of the comma-separated path string. For multi-character baselines this drops the rest of the value: for `path = "12,12,..."` it captures `"1"` instead of `"12"`; for `path = "-1,-1,..."` it captures `"-"` instead of `"-1"`. R's per-path control-pool subset is mis-allocated in both regimes. Python's tuple-key matching is correct in both — the per-path point estimates we compute are correct, R's per-path subset for the same path is buggy. The shipped R-parity scenarios stay in `D in {0, 1, 2}` to avoid the R bug; R-parity is asserted on that set at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathNonBinary` via the `multi_path_reversible_by_path_non_binary` scenario (78 switchers, 3 paths, single-baseline custom DGP, F_g >= 4). The string-encoding compatibility extends to all single-digit nonnegative D (`{0..9}`) since each value renders as a single character, but no R-parity scenario currently exercises D outside `{0, 1, 2}` — per-path point estimates match R bit-exactly (rtol ~1e-9 events; rtol+atol envelope for placebo near-zero values), SE inherits the documented cross-path cohort-sharing deviation (~5% rtol observed; SE_RTOL=0.15 envelope). Negative-integer treatment-state support (paths containing negative D values in non-baseline positions, e.g. `(0, -1, -1, -1)`) is regression-tested in Python only at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary::test_negative_integer_D_supported`; a dedicated regression for a negative-baseline path (e.g. `(-1, 0, 0, 0)`, the exact regime that would trigger R's `substr` bug) is deferred to a follow-up. Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathNonBinary`. **Per-path survey-design SE** (analytical Binder TSL + replicate-weight bootstrap): under `by_path` / `paths_of_interest` + `survey_design`, the per-path per-horizon SE routes through `_survey_se_from_group_if` using the cell-period allocator. The per-path influence function `U_pp_l_path` is the per-period IF with non-path switcher-side contributions skipped — control contributions remain unchanged, matching the joiners/leavers IF convention from the **Per-path SE convention** paragraph above (the `switcher_subset_mask` zeroes the switcher row of the per-group IF, which trivially zeroes the corresponding row of the per-cell IF, preserving the row-sum identity `U_pp.sum(axis=1) == U`). The IF is cohort-recentered via `_cohort_recenter_per_period` and expanded to observations as `psi_i = U_pp[g_i, t_i] · (w_i / W_{g_i, t_i})`. Replicate-weight designs unconditionally route through the cell allocator (Class A contract, PR #323). Multiplier bootstrap (`n_bootstrap > 0`) under `survey_design + by_path/paths_of_interest` raises `NotImplementedError` at fit-time — the survey-aware perturbation pivot for path-restricted IFs is methodologically underived and deferred to a future wave; the global non-by_path TSL multiplier bootstrap is unaffected and continues to ship. **Path-enumeration ranking is unweighted** under `survey_design`: top-k selection uses group cardinality (`path_to_count[p]` = number of groups), not population-weight mass — survey weights do not affect which paths are selected as "top-k". A weighted-ranking variant (sum of survey weights per path) is deferred until concrete demand. **`df_survey` propagation:** under replicate weights, every per-path per-horizon fit contributes an `n_valid` count to the shared `_replicate_n_valid_list` accumulator and the final `_effective_df_survey = min(...) - 1` reflects all per-path replicate fits. A post-call `_refresh_path_inference` helper re-runs `safe_inference` on every populated entry so `multi_horizon_inference`, `placebo_horizon_inference`, `path_effects`, and `path_placebos` all use the same final df after per-path appends complete. **Lonely-PSU policy is sample-wide, not per-path** — the `lonely_psu` policy (`remove`/`certainty`/`adjust`) operates on the full design-level PSU/strata structure, not on path-restricted subsamples. **Telescope invariant:** on a single-path panel where every switcher follows the same trajectory and `eligible_groups` matches between by_path and non-by_path, per-path SE equals the global non-by_path survey SE bit-exactly — pinned at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignTelescope::test_telescope_analytical_TSL`. **Deviation from R:** none — R `did_multiplegt_dyn` does not support survey weighting, so this is a Python-only methodology extension (no R parity available; no R parity test class). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSurveyDesignAnalytical` covering analytical SE, replicate-weight SE, the `n_bootstrap` gate, the global anti-regression, per-path placebos, `trends_linear` composition, and unobserved-path warnings under survey. **Per-path heterogeneity testing** (analytical OLS / WLS + survey-aware Binder TSL + replicate-weight): under `by_path` / `paths_of_interest` + `heterogeneity=""`, the per-path per-horizon coefficient `beta_X^path_l` is computed by re-running `_compute_heterogeneity_test` on the path-restricted switcher subsample. The path filter (`path_groups: Optional[Set[int]]`) restricts eligibility to switchers ON path `p` inside the inner regression; the variance machinery (HC1-robust OLS vcov for non-survey via `solve_ols(..., return_vcov=True)` (`vcov_type="hc1"` default), WLS-on-pweights with cell-period IF allocator for analytical Binder TSL, group-level allocator for Rao-Wu replicate) is unchanged from the global heterogeneity path. **Cohort dummies absorb baseline by construction** — the cohort key `(D_{g,1}, F_g, S_g)` includes baseline, so multi-baseline switcher panels do not produce R-divergence (unlike `controls` / `trends_linear`); no parallel `UserWarning` is emitted. **R parity:** matches `did_multiplegt_dyn(..., by_path, predict_het)` per-by_level on the `multi_path_reversible_by_path_predict_het` scenario for `beta`, `se`, `t_stat`, and `n_obs` (`BETA_RTOL = 1e-6` on `beta`, `SE_RTOL = 1e-5` on `se` / `t_stat`; the SE tolerance is one decade looser than `BETA_RTOL` to absorb the small OLS denominator-and-cohort-recentering numerical drift observed on this fixture; `n_obs` matches exactly). Inherits the same tolerances as the new global `multi_path_reversible_predict_het` scenario (`TestDCDHDynRParityHeterogeneity`) since the per-path R call is `did_multiplegt_main(..., predict_het=...)` per path-restricted subsample with no additional numerical loss. **R parity (heterogeneity inference, post-2026-05-15 df threading):** Python now passes `df = n_obs - n_params` to `safe_inference` on the non-survey OLS path at `chaisemartin_dhaultfoeuille.py`'s `_compute_heterogeneity_test`, matching R's t-distribution with df from the WLS regression (`DIDmultiplegtDYN:::did_multiplegt_main` `t_stat <- qt(0.975, df.residual(model))` site). Parity tolerance is `INFERENCE_RTOL = 1e-4` on `p_value` and `conf_int`; `beta` / `se` / `t_stat` continue to use `BETA_RTOL = 1e-6` / `SE_RTOL = 1e-5`. The `t_stat = beta / se` field is distribution-invariant. **Rank-deficient caveat:** `n_params = design.shape[1]` is the pre-drop column count; under near-rank-deficient designs that `solve_ols` retains rather than NaN-out, the actual rank may be lower than `n_params` (R's `df.residual` uses post-drop rank). Fully rank-deficient designs are NaN-filled by the rank-deficient short-circuit at `_compute_heterogeneity_test:5141-5150`, so the gap only affects edge cases. Tracked as a Low TODO follow-up (rank-from-`solve_ols` threading). R's `dont_drop_larger_lower=TRUE` is set in both fixture scenarios to match the Python `drop_larger_lower=False` requirement. **Survey composition:** inherits from the **Per-path survey-design SE** paragraph above — analytical Binder TSL routes through `_survey_se_from_group_if`'s cell-period allocator on the post-period of the transition; replicate-weights route through the group-level allocator. Multiplier bootstrap (`n_bootstrap > 0`) under `by_path + heterogeneity + survey_design` inherits the existing per-path multiplier-bootstrap-survey gate. **`df_survey` propagation:** every per-(path, horizon) replicate-weight fit appends `n_valid` to the shared `_replicate_n_valid_list` accumulator; per-path heterogeneity inference is refreshed with the FINAL `_effective_df_survey(...)` in the R2 P1b refresh block (separate dedicated loop because the schema shape is `{path: {l: {...}}}` rather than `{path: {"horizons": {l: {...}}}}`). **Result schema:** `results.path_heterogeneity_effects: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]` keyed `{path: {l: {beta, se, t_stat, p_value, conf_int, n_obs}}}`. Empty-state contract mirrors `path_effects`: `None` when not requested, `{}` when requested but no path has eligible switchers. **DataFrame integration:** `to_dataframe(level="by_path")` adds always-present `het_*` columns (`het_beta`, `het_se`, `het_t_stat`, `het_p_value`, `het_conf_int_lower`, `het_conf_int_upper`), populated for positive-horizon rows when `heterogeneity` is set and NaN otherwise (mirrors the `cband_*` and `cumulated_*` always-present convention). **Per-path placebo heterogeneity (`placebo + predict_het + by_path`, post-2026-05-15):** R-verified — `did_multiplegt_dyn(by_path, predict_het, placebo)` emits per-path heterogeneity OLS results on backward (placebo) horizons via R's per-by_level dispatcher (`DIDmultiplegtDYN:::did_multiplegt_main` placebo block at the `effect = matrix(-i, ...)` rbind site). R's predict_het syntax: passing `predict_het = list("X", c(-1))` with `placebo > 0` triggers "compute heterogeneity for ALL forward (1..effects) AND ALL placebo (1..placebo) positions"; forward rows have positive `effect` values, placebo rows negative. Python mirrors via `_compute_heterogeneity_test(..., placebo=L_max)` (set when `self.placebo` is truthy) — the function iterates forward (1..L_max) and backward (-1..-L_max) horizons in a single loop with an explicit `out_idx < 0` eligibility guard for backward horizons whose `F_g` is too small (would otherwise silently misread `N_mat` via numpy negative indexing). Placebo rows in `to_dataframe(level="by_path")` have non-NaN `het_*` columns when `placebo=True` and `heterogeneity=` are both set; `path_heterogeneity_effects` uses negative-int keys for backward horizons, mirroring the existing `path_placebo_event_study` convention. **Survey gate (warn + skip):** `survey_design + placebo + heterogeneity` emits a `UserWarning` at fit-time and falls back to forward-horizon-only heterogeneity (codex R1 P1 #1: the eager raise broke the previously-supported forward-horizon survey + predict_het path under the default `placebo=True`) — the Binder TSL cell-period allocator's justification (Survey IF expansion Note above) is tied to **post-period** attribution (`out_idx = first_switch_idx[g] - 1 + l_h` with `l_h > 0`); backward-horizon attribution puts ψ_g mass on a pre-period cell, which is a separate library-extension claim that needs its own derivation. Forward-horizon `predict_het + survey_design` continues to work unchanged on both global and per-path surfaces. The function-level `_compute_heterogeneity_test` keeps a per-iteration backstop that raises `NotImplementedError` if a direct caller bypasses fit() and passes `survey + placebo > 0` (regression-tested at `test_compute_heterogeneity_test_direct_call_raises_on_backward_survey`). Pre-period allocator derivation is deferred to a follow-up methodology PR. R parity confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathHeterogeneityWithPlacebo` on the `multi_path_reversible_predict_het_with_placebo` fixture (scenario 22, `placebo=2, effects=3, by_path=3, predict_het=list("het_x", c(-1))`) AND `::TestDCDHDynRParityHeterogeneityWithPlacebo` on the global anchor (`multi_path_reversible_predict_het_with_placebo_global`, scenario 23, same DGP without by_path) — both surfaces emit forward + backward heterogeneity rows in matching parity. Pinned at `BETA_RTOL=1e-6` / `SE_RTOL=1e-5` for `beta` / `se` / `t_stat` / `n_obs`; `INFERENCE_RTOL=1e-4` for `p_value` / `conf_int`. Cross-surface invariants regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPredictHetPlacebo`. Regression test anchors: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathHeterogeneity` (gate dispatch, behavior, telescope-to-global on single-path panel, zero-signal anti-regression, multi-baseline UserWarning anti-regression, DataFrame integration, edge cases) + `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityHeterogeneity` (global anchor, FIRST `predict_het` parity baseline) + `::TestDCDHDynRParityByPathHeterogeneity` (per-path). **Per-path user-specified path selection (`paths_of_interest`):** Python-only API extension — R's `did_multiplegt_dyn(..., by_path=k)` only accepts a positive int (top-k automatic ranking) or `-1` (all observed paths) and provides no list-based selection. Activated via `ChaisemartinDHaultfoeuille(paths_of_interest=[(0, 1, 1, 1), (0, 1, 0, 0)], drop_larger_lower=False)` as an alternative to `by_path=k`; the two are **mutually exclusive** (setting both raises `ValueError` at `__init__` and `set_params` time). Each path tuple must have length `L_max + 1`; the type / element / non-empty / length-uniformity checks fire at `__init__`, the length-vs-L_max check fires at fit-time. `bool` and `np.bool_` are explicitly rejected; `np.integer` is accepted and canonicalized to Python `int` for tuple-key consistency. Duplicates emit a `UserWarning` and are deduplicated; paths not observed in the panel emit a `UserWarning` and are omitted from `path_effects`. Paths appear in `results.path_effects` in the user-specified order, modulo deduplication and unobserved-path filtering. Composes with non-binary D and all downstream `by_path` surfaces (bootstrap, per-path placebos, per-path joint sup-t bands, `controls`, `trends_linear`, `trends_nonparam`) — mechanical filter on observed paths, no methodology change. Behavior + cross-feature regressions live at `tests/test_chaisemartin_dhaultfoeuille.py::TestPathsOfInterest`. diff --git a/tests/test_chaisemartin_dhaultfoeuille.py b/tests/test_chaisemartin_dhaultfoeuille.py index 56d6669a..ae7776d1 100644 --- a/tests/test_chaisemartin_dhaultfoeuille.py +++ b/tests/test_chaisemartin_dhaultfoeuille.py @@ -2867,18 +2867,21 @@ def test_heterogeneity_multi_horizon(self): assert 1 in r.heterogeneity_effects assert 2 in r.heterogeneity_effects - def test_heterogeneity_inference_matches_safe_inference(self): - """Local invariant: non-survey heterogeneity `t_stat` / `p_value` / - `conf_int` must equal ``safe_inference(beta, se, df=None)`` on every - populated horizon. R parity for these fields is intentionally - skipped (Python uses normal Z, R uses finite-df t — documented in - REGISTRY); without this local invariant a regression isolated to - the inference extraction or `_refresh_path_inference` ordering - could silently drop / mis-route the SE-derived fields while - beta / se still pass R parity. + def test_heterogeneity_inference_local_invariants(self): + """Local SE-derivation invariants for non-survey heterogeneity + inference. Post-2026-05-15 df threading: Python passes + ``df = n_obs - n_params`` to ``safe_inference`` (matching R's + t-distribution); R-parity is pinned in + ``tests/test_chaisemartin_dhaultfoeuille_parity.py``. This local + test verifies the SE-derived fields are wired correctly + without requiring back-derivation of ``n_params``: + ``t_stat = beta / se``; ``conf_int`` symmetric around ``beta`` + with positive half-width; ``p_value`` in ``[0, 1]``. + Without these checks a regression isolated to the inference + extraction or ``_refresh_path_inference`` ordering could + silently drop / mis-route the SE-derived fields while beta / se + still pass R parity. """ - from diff_diff.utils import safe_inference - df = self._make_panel_with_het() r = ChaisemartinDHaultfoeuille(seed=1).fit( df, "outcome", "group", "period", "treatment", @@ -2889,19 +2892,22 @@ def test_heterogeneity_inference_matches_safe_inference(self): for l_h, het in r.heterogeneity_effects.items(): if not (np.isfinite(het["beta"]) and np.isfinite(het["se"])): continue - expected_t, expected_p, expected_ci = safe_inference( - het["beta"], het["se"], df=None - ) + expected_t = het["beta"] / het["se"] assert het["t_stat"] == pytest.approx(expected_t, rel=1e-12), ( f"l={l_h} t_stat: stored={het['t_stat']} vs " - f"safe_inference={expected_t}" + f"beta/se={expected_t}" + ) + half_low = het["beta"] - het["conf_int"][0] + half_high = het["conf_int"][1] - het["beta"] + assert half_low > 0, f"l={l_h} conf_int_lower not below beta" + assert half_high > 0, f"l={l_h} conf_int_upper not above beta" + assert half_low == pytest.approx(half_high, rel=1e-12), ( + f"l={l_h} conf_int asymmetric: " + f"below={half_low} above={half_high}" ) - assert het["p_value"] == pytest.approx(expected_p, rel=1e-12), ( - f"l={l_h} p_value: stored={het['p_value']} vs " - f"safe_inference={expected_p}" + assert 0.0 <= het["p_value"] <= 1.0, ( + f"l={l_h} p_value out of [0, 1]: {het['p_value']}" ) - assert het["conf_int"][0] == pytest.approx(expected_ci[0], rel=1e-12) - assert het["conf_int"][1] == pytest.approx(expected_ci[1], rel=1e-12) checked += 1 assert checked >= 1, "Expected at least one populated heterogeneity horizon" @@ -10280,20 +10286,18 @@ def test_per_path_heterogeneity_finite_under_known_signal(self): f"(DGP: 5 + 3*het_x), got {horizons[1]['beta']}" ) - def test_per_path_heterogeneity_inference_matches_safe_inference(self): - """Local invariant: non-survey per-path heterogeneity `t_stat` / - `p_value` / `conf_int` must equal ``safe_inference(beta, se, - df=None)`` on every populated (path, horizon) entry. R parity - for these fields is intentionally skipped (Python uses normal Z, - R uses finite-df t — documented in REGISTRY); without this local - invariant a regression isolated to the inference extraction or - `_refresh_path_inference` ordering could silently drop or - mis-route the SE-derived fields while beta / se still pass R - parity. Mirrors the global heterogeneity test of the same name - in TestHeterogeneityTesting. + def test_per_path_heterogeneity_inference_local_invariants(self): + """Local SE-derivation invariants for non-survey per-path + heterogeneity inference. Post-2026-05-15 df threading: Python + passes ``df = n_obs - n_params`` to ``safe_inference``; R-parity + is pinned in + ``tests/test_chaisemartin_dhaultfoeuille_parity.py:: + TestDCDHDynRParityByPathHeterogeneity``. Verifies SE-derivation + wiring (``t_stat = beta/se``, symmetric ``conf_int`` around beta, + ``p_value`` in ``[0, 1]``) without back-deriving ``n_params``. + Mirrors + ``TestHeterogeneityTesting::test_heterogeneity_inference_local_invariants``. """ - from diff_diff.utils import safe_inference - df = _by_path_het_data() est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) with warnings.catch_warnings(): @@ -10308,22 +10312,24 @@ def test_per_path_heterogeneity_inference_matches_safe_inference(self): for l_h, het in horizons.items(): if not (np.isfinite(het["beta"]) and np.isfinite(het["se"])): continue - expected_t, expected_p, expected_ci = safe_inference( - het["beta"], het["se"], df=None - ) + expected_t = het["beta"] / het["se"] assert het["t_stat"] == pytest.approx(expected_t, rel=1e-12), ( f"path={path} l={l_h} t_stat: stored={het['t_stat']} vs " - f"safe_inference={expected_t}" + f"beta/se={expected_t}" ) - assert het["p_value"] == pytest.approx(expected_p, rel=1e-12), ( - f"path={path} l={l_h} p_value: stored={het['p_value']} vs " - f"safe_inference={expected_p}" + half_low = het["beta"] - het["conf_int"][0] + half_high = het["conf_int"][1] - het["beta"] + assert half_low > 0, ( + f"path={path} l={l_h} conf_int_lower not below beta" ) - assert het["conf_int"][0] == pytest.approx( - expected_ci[0], rel=1e-12 + assert half_high > 0, ( + f"path={path} l={l_h} conf_int_upper not above beta" ) - assert het["conf_int"][1] == pytest.approx( - expected_ci[1], rel=1e-12 + assert half_low == pytest.approx(half_high, rel=1e-12), ( + f"path={path} l={l_h} conf_int asymmetric" + ) + assert 0.0 <= het["p_value"] <= 1.0, ( + f"path={path} l={l_h} p_value out of [0, 1]" ) checked += 1 assert checked >= 1, ( @@ -10988,7 +10994,9 @@ def test_survey_design_plus_n_bootstrap_with_heterogeneity_still_raises( def test_to_dataframe_by_path_includes_heterogeneity_columns(self): """``to_dataframe(level='by_path')`` includes het_* columns; - populated for positive horizons and NaN for placebo rows.""" + populated for both forward and placebo horizons when + ``placebo=True`` and ``heterogeneity=`` are both set + (post-2026-05-15 #422).""" df = _by_path_het_data() est = ChaisemartinDHaultfoeuille( drop_larger_lower=False, by_path=2, placebo=True @@ -11006,13 +11014,18 @@ def test_to_dataframe_by_path_includes_heterogeneity_columns(self): assert "het_p_value" in out.columns assert "het_conf_int_lower" in out.columns assert "het_conf_int_upper" in out.columns - # Placebo rows: het_* must be NaN - if (out.horizon < 0).any(): - placebo_rows = out[out.horizon < 0] - assert placebo_rows["het_beta"].isna().all() # Positive horizons: at least some entries are populated positive_rows = out[out.horizon > 0] assert positive_rows["het_beta"].notna().any() + # Placebo rows: NOW also populated (closes TODO #422). Pre-PR + # contract was hardcoded NaN; new contract reads from + # path_heterogeneity_effects negative-int keys. + if (out.horizon < 0).any(): + placebo_rows = out[out.horizon < 0] + assert placebo_rows["het_beta"].notna().any(), ( + "Expected at least one placebo row with non-NaN het_beta " + "after #422 (per-path placebo predict_het R-parity)." + ) def test_per_path_heterogeneity_renders_in_summary(self): """``summary()`` includes per-path heterogeneity sub-block. @@ -11083,3 +11096,365 @@ def test_path_unobserved_under_heterogeneity_warns_omits(self): assert res.path_heterogeneity_effects is not None assert (1, 1, 1, 0) not in res.path_heterogeneity_effects assert (0, 1, 1, 1) in res.path_heterogeneity_effects + + +def _single_path_het_data(seed=44, n_switchers=30, n_controls=15, n_periods=10): + """Single-path multi-cohort panel with binary `het_x` for telescope tests. + + All 30 switchers follow path (0, 1, 1, 1) with F_g cycling in {3, 4, 5} + (10 groups per F_g). Mirrors `_by_path_het_data` shape but restricted + to a single observed path so `path_heterogeneity_effects[(0,1,1,1)]` + can be compared bit-exactly against global `heterogeneity_effects`. + """ + rng = np.random.RandomState(seed) + rows = [] + path = (0, 1, 1, 1) + for g in range(n_switchers): + F_g = 3 + ((g // 10) % 3) + het_x = 1 if g < n_switchers // 2 else 0 + effect = 5.0 + 3.0 * het_x + for t in range(n_periods): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + y = 0.5 * t + effect * d + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": y, "het_x": het_x, + }) + for k in range(n_controls): + het_x = 1 if k < n_controls // 2 else 0 + g = n_switchers + k + for t in range(n_periods): + y = 0.5 * t + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": 0, + "outcome": y, "het_x": het_x, + }) + return pd.DataFrame(rows) + + +class TestByPathPredictHetPlacebo: + """`predict_het` × `placebo` × `by_path` (closes TODO #422 + pilot-412). + + R-verified: `did_multiplegt_dyn(by_path, predict_het, placebo)` emits + per-path heterogeneity OLS results on backward (placebo) horizons via + R's per-by_level dispatcher. Python mirrors via + ``_compute_heterogeneity_test(..., placebo=L_max)`` when the user + sets ``placebo=True``. + + R-parity coverage in + ``tests/test_chaisemartin_dhaultfoeuille_parity.py:: + TestDCDHDynRParityByPathHeterogeneityWithPlacebo``. + """ + + def test_to_dataframe_by_path_emits_het_columns_on_placebo_rows(self): + """`to_dataframe(level="by_path")` placebo rows now have het_*.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + out = res.to_dataframe(level="by_path") + placebo_rows = out[out["horizon"] < 0] + assert len(placebo_rows) > 0, "expected at least one placebo row" + finite_het_count = placebo_rows["het_beta"].notna().sum() + assert finite_het_count > 0, ( + "Expected at least one placebo row with non-NaN het_beta. " + "Pre-#422 contract was hardcoded NaN; new contract populates " + "from path_heterogeneity_effects negative-key lookup." + ) + + def test_predict_het_placebo_survey_design_warns_and_skips_backward(self): + """survey_design + placebo + heterogeneity warns + emits forward-only. + + Per codex R1 P1 #1: the previous eager-at-function-entry gate + broke the previously-supported forward-horizon survey + predict_het + path under the default `placebo=True` setting. Replaced with a + per-iteration backstop in `_compute_heterogeneity_test` (raises + only when actually computing a backward iteration under survey) + plus fit-time warn+skip at the global and per-path call sites + that pass `placebo=0` when survey is active. User gets a + UserWarning and forward-horizon results, NOT an exception. + + The defensive direct-call gate is exercised separately by + `test_compute_heterogeneity_test_direct_call_raises_on_backward_survey`. + """ + from diff_diff.survey import SurveyDesign + + df = _by_path_het_data() + df["sw"] = 1.0 + df["stratum"] = df["group"] % 4 + df["psu_id"] = df["group"] + sd = SurveyDesign( + weights="sw", strata="stratum", psu="psu_id", + ) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, + heterogeneity="het_x", survey_design=sd, + ) + # Warning fired with the expected substring + het_warnings = [ + w for w in caught + if "backward-horizon (placebo) predict_het" in str(w.message) + ] + assert het_warnings, ( + "Expected UserWarning about backward-horizon survey gate. " + f"Got: {[str(w.message) for w in caught]}" + ) + # Forward-horizon heterogeneity ran successfully + assert res.heterogeneity_effects is not None + # Only positive-int keys (forward); no negative (placebo) keys + het_keys = sorted(res.heterogeneity_effects.keys()) + assert all(h > 0 for h in het_keys), ( + f"Expected only positive horizons under survey gate, got: {het_keys}" + ) + # Per-path heterogeneity also forward-only + assert res.path_heterogeneity_effects is not None + for path, horizons in res.path_heterogeneity_effects.items(): + path_keys = sorted(horizons.keys()) + assert all(h > 0 for h in path_keys), ( + f"path={path}: expected only positive horizons, got {path_keys}" + ) + + def test_compute_heterogeneity_test_direct_call_raises_on_backward_survey( + self, + ): + """Direct calls to `_compute_heterogeneity_test` with survey + + backward horizon raise NotImplementedError. + + Defensive backstop: fit() gates this case upstream, so the + function-level raise is unreachable via the normal flow. This + test exercises the per-iteration gate directly to lock the API + contract for any future internal call site. + """ + from diff_diff.chaisemartin_dhaultfoeuille import ( + _compute_heterogeneity_test, + ) + from diff_diff.survey import SurveyDesign + + df = _by_path_het_data() + df["sw"] = 1.0 + df["stratum"] = df["group"] % 4 + df["psu_id"] = df["group"] + sd = SurveyDesign( + weights="sw", strata="stratum", psu="psu_id", + ) + # Build a minimal valid obs_survey_info dict matching the + # function's contract. SurveyDesign.resolve() takes only the + # dataframe; group/time are inferred from the design context. + resolved = sd.resolve(df) + groups = sorted(df["group"].unique()) + periods = sorted(df["period"].unique()) + n_groups = len(groups) + n_periods = len(periods) + Y_mat = np.zeros((n_groups, n_periods)) + N_mat = np.ones((n_groups, n_periods)) + baselines = np.zeros(n_groups) + first_switch_idx = np.full(n_groups, 3, dtype=int) + switch_direction = np.ones(n_groups) + T_g = np.full(n_groups, n_periods - 1, dtype=int) + X_het = np.zeros(n_groups) + obs_survey_info = { + "group_ids": df["group"].to_numpy(), + "time_ids": df["period"].to_numpy(), + "weights": df["sw"].to_numpy(dtype=np.float64), + "resolved": resolved, + "periods": np.asarray(periods), + } + with pytest.raises( + NotImplementedError, + match=r"backward-horizon \(placebo\) predict_het", + ): + _compute_heterogeneity_test( + Y_mat=Y_mat, + N_mat=N_mat, + baselines=baselines, + first_switch_idx=first_switch_idx, + switch_direction=switch_direction, + T_g=T_g, + X_het=X_het, + L_max=2, + placebo=2, + group_ids_order=np.asarray(groups), + obs_survey_info=obs_survey_info, + ) + + def test_predict_het_placebo_survey_forward_only_still_works(self): + """survey + predict_het without placebo continues to work.""" + from diff_diff.survey import SurveyDesign + + df = _by_path_het_data() + df["sw"] = 1.0 + df["stratum"] = df["group"] % 4 + df["psu_id"] = df["group"] + sd = SurveyDesign( + weights="sw", strata="stratum", psu="psu_id", + ) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=False + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, + heterogeneity="het_x", survey_design=sd, + ) + assert res.path_heterogeneity_effects is not None + for path, horizons in res.path_heterogeneity_effects.items(): + for h in [1, 2, 3]: + if h in horizons: + assert np.isfinite(horizons[h]["beta"]), ( + f"path={path} h={h} beta non-finite under " + f"forward+survey path" + ) + + def test_predict_het_placebo_eligible_filter(self): + """`out_idx < 0` guard filters groups when F_g < |placebo|+1. + + Backward horizon `l_h = -k` requires `out_idx = F_g - 1 - k >= 0`, + i.e., `F_g >= k + 1`. Groups with smaller F_g are filtered out + rather than producing wrong-cell numpy reads via negative indexing. + """ + rng = np.random.RandomState(99) + rows = [] + path = (0, 1, 1, 1) + n_switchers = 60 + n_controls = 30 + n_periods = 10 + for g in range(n_switchers): + F_g = 2 # ALL switchers have F_g=2 + het_x = 1 if g < n_switchers // 2 else 0 + effect = 5.0 + 3.0 * het_x + for t in range(n_periods): + if F_g - 1 <= t < F_g - 1 + len(path): + d = path[t - (F_g - 1)] + elif t >= F_g - 1 + len(path): + d = path[-1] + else: + d = 0 + y = 0.5 * t + effect * d + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": d, + "outcome": y, "het_x": het_x, + }) + for k in range(n_controls): + het_x = 1 if k < n_controls // 2 else 0 + g = n_switchers + k + for t in range(n_periods): + y = 0.5 * t + rng.normal(0, 0.5) + rows.append({ + "group": g, "period": t, "treatment": 0, + "outcome": y, "het_x": het_x, + }) + df = pd.DataFrame(rows) + + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, paths_of_interest=[(0, 1, 1, 1)], + placebo=True, + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + + assert res.path_heterogeneity_effects is not None + path_het = res.path_heterogeneity_effects.get((0, 1, 1, 1), {}) + # -1 has out_idx=0 → eligible + # -2 has out_idx=-1 → all groups filtered, n_obs=0, NaN-consistent + # -3 has out_idx=-2 → all groups filtered, n_obs=0, NaN-consistent + if -2 in path_het: + assert path_het[-2]["n_obs"] == 0, ( + f"placebo -2 should be filtered (out_idx<0): " + f"got n_obs={path_het[-2]['n_obs']}" + ) + assert np.isnan(path_het[-2]["beta"]) + assert np.isnan(path_het[-2]["se"]) + if -3 in path_het: + assert path_het[-3]["n_obs"] == 0 + assert np.isnan(path_het[-3]["beta"]) + + def test_path_heterogeneity_telescopes_to_global_on_single_path_panel( + self, + ): + """Single-path panel: per-path het == global het bit-exactly. + + Cross-surface twin: when only one path is observed, + `path_heterogeneity_effects[(only_path,)]` should equal + `heterogeneity_effects` (forward + backward) because the + path-restricted regression has the same eligible group set as + the global regression. + """ + df = _single_path_het_data() + est_g = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res_g = est_g.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + est_p = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + paths_of_interest=[(0, 1, 1, 1)], + placebo=True, + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res_p = est_p.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + path_het = res_p.path_heterogeneity_effects[(0, 1, 1, 1)] + global_het = res_g.heterogeneity_effects + + for h in list(global_het.keys()): + assert h in path_het, f"horizon {h} missing in path_het" + g_h = global_het[h] + p_h = path_het[h] + if not np.isfinite(g_h["beta"]): + assert not np.isfinite(p_h["beta"]) + continue + np.testing.assert_allclose( + p_h["beta"], g_h["beta"], atol=1e-14, rtol=1e-14, + err_msg=f"horizon {h} beta telescope failed", + ) + np.testing.assert_allclose( + p_h["se"], g_h["se"], atol=1e-14, rtol=1e-14, + err_msg=f"horizon {h} se telescope failed", + ) + assert int(p_h["n_obs"]) == int(g_h["n_obs"]) + + def test_summary_renders_placebo_het_rows(self): + """`result.summary()` renders without error after #422.""" + df = _by_path_het_data() + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = est.fit( + df, outcome="outcome", group="group", time="period", + treatment="treatment", L_max=3, heterogeneity="het_x", + ) + s = res.summary() + assert isinstance(s, str) + assert len(s) > 0 diff --git a/tests/test_chaisemartin_dhaultfoeuille_parity.py b/tests/test_chaisemartin_dhaultfoeuille_parity.py index 6afe64ce..ee8b42b2 100644 --- a/tests/test_chaisemartin_dhaultfoeuille_parity.py +++ b/tests/test_chaisemartin_dhaultfoeuille_parity.py @@ -38,6 +38,23 @@ ) +def _as_dict(slot): + """Coerce a fixture slot to a dict for type-stable iteration. + + R's ``jsonlite::toJSON`` serializes empty named lists (``list()``) + as JSON arrays (``[]``); populated named lists serialize as JSON + objects (``{}``). The dCDH `predict_het` extractors return empty + lists for the no-rows case (e.g., scenarios without ``placebo`` set + have empty ``placebo_predict_het`` slots), so consumers iterating + ``.items()`` need to handle the array form. This helper coerces + empty list ``[]``, ``None`` (missing slot), or any non-dict value + to ``{}``; populated dicts pass through unchanged. Used by all + parity-test classes that read the optional ``placebo_predict_het`` + / ``placebo_horizons`` slots. + """ + return slot if isinstance(slot, dict) else {} + + @pytest.fixture(scope="module") def golden_values(): """ @@ -1341,6 +1358,7 @@ class TestDCDHDynRParityHeterogeneity: BETA_RTOL = 1e-6 BETA_ATOL = 1e-9 SE_RTOL = 1e-5 + INFERENCE_RTOL = 1e-4 # p_value, conf_int — t-distribution implementation tol def test_parity_multi_path_reversible_predict_het(self, golden_values): """Global heterogeneity coefficient parity per horizon.""" @@ -1391,14 +1409,27 @@ def test_parity_multi_path_reversible_predict_het(self, golden_values): assert int(py_h["n_obs"]) == int(r_h["n_obs"]), ( f"h={h} n_obs: py={py_h['n_obs']} vs r={r_h['n_obs']}" ) - # NOTE: `p_value` and `conf_int` are NOT pinned to R here. Python's - # `safe_inference(..., df=None)` uses the normal Z critical value - # (~1.96 at alpha=0.05); R `did_multiplegt_dyn(..., predict_het)` - # uses t-distribution with df = n - k from the OLS regression. - # That structural deviation produces ~0.1-2% rtol gaps on CI - # bounds and p-values - tracked separately rather than masked by - # a loose parity tolerance. See REGISTRY Phase 3 heterogeneity - # Note "Deviation from R (heterogeneity inference critical value)". + # `p_value` and `conf_int` parity (post-2026-05-15 df threading). + # `_compute_heterogeneity_test` now passes `df = n_obs - n_params` + # to `safe_inference`, matching R's t-distribution with WLS df. + # Pinned at INFERENCE_RTOL = 1e-4 because Wald-test critical + # values come from `scipy.stats.t.ppf` and `t.sf` which are + # implementation-aligned with R's `qt`/`pt` to ~6 sig figs. + assert py_h["p_value"] == pytest.approx( + r_h["p_value"], rel=self.INFERENCE_RTOL + ), f"h={h} p_value: py={py_h['p_value']:.6e} vs r={r_h['p_value']:.6e}" + assert py_h["conf_int"][0] == pytest.approx( + r_h["ci_lo"], rel=self.INFERENCE_RTOL + ), ( + f"h={h} ci_lo: py={py_h['conf_int'][0]:.6f} " + f"vs r={r_h['ci_lo']:.6f}" + ) + assert py_h["conf_int"][1] == pytest.approx( + r_h["ci_hi"], rel=self.INFERENCE_RTOL + ), ( + f"h={h} ci_hi: py={py_h['conf_int'][1]:.6f} " + f"vs r={r_h['ci_hi']:.6f}" + ) class TestDCDHDynRParityByPathHeterogeneity: @@ -1421,6 +1452,7 @@ class TestDCDHDynRParityByPathHeterogeneity: BETA_RTOL = 1e-6 BETA_ATOL = 1e-9 SE_RTOL = 1e-5 + INFERENCE_RTOL = 1e-4 def _path_key_from_r_label(self, r_label: str): return tuple(int(x) for x in r_label.split(",")) @@ -1489,10 +1521,7 @@ def test_parity_multi_path_reversible_by_path_predict_het( f"py={py_h['se']:.6f} vs r={r_h['se']:.6f}" ) # `t_stat = beta / se` is invariant to the Wald-test - # critical-value distribution; pin it at SE_RTOL so a - # regression in beta or se surfaces here too. p_value - # and conf_int are not pinned - see the global parity - # class for the Z-vs-t deviation note. + # critical-value distribution; pin at SE_RTOL. assert py_h["t_stat"] == pytest.approx( r_h["t"], rel=self.SE_RTOL ), ( @@ -1503,3 +1532,280 @@ def test_parity_multi_path_reversible_by_path_predict_het( f"path={path_key} h={h} n_obs: " f"py={py_h['n_obs']} vs r={r_h['n_obs']}" ) + # `p_value` and `conf_int` parity — same df-threading + # rationale as the global heterogeneity class. + assert py_h["p_value"] == pytest.approx( + r_h["p_value"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} p_value: " + f"py={py_h['p_value']:.6e} vs r={r_h['p_value']:.6e}" + ) + assert py_h["conf_int"][0] == pytest.approx( + r_h["ci_lo"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} ci_lo: " + f"py={py_h['conf_int'][0]:.6f} vs r={r_h['ci_lo']:.6f}" + ) + assert py_h["conf_int"][1] == pytest.approx( + r_h["ci_hi"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} ci_hi: " + f"py={py_h['conf_int'][1]:.6f} vs r={r_h['ci_hi']:.6f}" + ) + + +class TestDCDHDynRParityByPathHeterogeneityWithPlacebo: + """Parity tests for ``by_path`` + ``predict_het`` + ``placebo``. + + R-verified: ``did_multiplegt_dyn(by_path, predict_het, placebo)`` + emits per-path heterogeneity OLS results on backward (placebo) + horizons via R's per-by_level dispatcher + (``DIDmultiplegtDYN:::did_multiplegt_main`` placebo block at the + ``effect = matrix(-i, ...)`` rbind site). Python mirrors via + ``_compute_heterogeneity_test(..., placebo=N)`` which iterates + forward (1..L_max) and backward (-1..-N) horizons in a single loop. + + Fixture: scenario 22 (``multi_path_reversible_predict_het_with_placebo``) + — same DGP as scenario 21 (reuses ``d20``) so per-path tolerances + inherit from ``TestDCDHDynRParityByPathHeterogeneity``. + + R syntax note: scenarios 20/21 use ``predict_het = list("X", c(1,2,3))`` + which is rejected by R when ``placebo > 0`` (forward indices > placebo + count error). Scenario 22 uses the ``c(-1)`` sentinel that triggers + "compute het for ALL forward (1..effects) AND ALL placebo + (1..placebo) positions" — see the R script comment at + ``benchmarks/R/generate_dcdh_dynr_test_values.R`` scenario 22. + """ + + BETA_RTOL = 1e-6 + BETA_ATOL = 1e-9 + SE_RTOL = 1e-5 + INFERENCE_RTOL = 1e-4 + + def _path_key_from_r_label(self, r_label: str): + return tuple(int(x) for x in r_label.split(",")) + + def test_parity_multi_path_reversible_predict_het_with_placebo( + self, golden_values + ): + """Per-path heterogeneity on forward + backward horizons.""" + import warnings + + scenario = golden_values.get( + "multi_path_reversible_predict_het_with_placebo" + ) + if scenario is None: + pytest.skip( + "scenario 'multi_path_reversible_predict_het_with_placebo' " + "not in golden values" + ) + + df = _golden_to_df_with_extra(scenario["data"], extra_cols=["het_x"]) + # Python's `placebo=True` triggers all-backward-horizons (1..L_max) + # placebo computation; the parity check iterates only R's emitted + # subset (placebo=2 in scenario 22 -> backward horizons -1, -2). + # Python's extra backward horizons (e.g., -3 if eligible) are + # allowed and not parity-checked. + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + ) + + assert results.path_heterogeneity_effects is not None + r_by_path = scenario["results"]["by_path_predict_het"] + + py_keys = set(results.path_heterogeneity_effects.keys()) + r_keys = {self._path_key_from_r_label(e["path"]) for e in r_by_path} + assert py_keys == r_keys, ( + f"Path-set mismatch.\n" + f" Python only: {py_keys - r_keys}\n" + f" R only: {r_keys - py_keys}" + ) + + for r_path_entry in r_by_path: + path_key = self._path_key_from_r_label(r_path_entry["path"]) + py_horizons = results.path_heterogeneity_effects[path_key] + + # Forward horizons (positive int keys). + for h_str, r_h in r_path_entry["horizons"].items(): + h = int(h_str) + assert h > 0 + assert h in py_horizons, ( + f"path={path_key}: forward horizon {h} missing" + ) + py_h = py_horizons[h] + self._assert_horizon_parity(path_key, h, py_h, r_h) + + # Placebo horizons (negative int keys). R-verified: scenario 22 + # has 2 placebo rows per path (-1, -2); Python mirrors with + # negative-int keys in path_heterogeneity_effects. + for h_str, r_h in _as_dict( + r_path_entry.get("placebo_horizons") + ).items(): + h = int(h_str) + assert h < 0 + assert h in py_horizons, ( + f"path={path_key}: placebo horizon {h} missing " + f"from Python path_heterogeneity_effects" + ) + py_h = py_horizons[h] + self._assert_horizon_parity(path_key, h, py_h, r_h) + + def _assert_horizon_parity(self, path_key, h, py_h, r_h): + """Pin all 6 inference fields against R.""" + assert py_h["beta"] == pytest.approx( + r_h["beta"], rel=self.BETA_RTOL, abs=self.BETA_ATOL + ), ( + f"path={path_key} h={h} beta: " + f"py={py_h['beta']:.6f} vs r={r_h['beta']:.6f}" + ) + assert py_h["se"] == pytest.approx(r_h["se"], rel=self.SE_RTOL), ( + f"path={path_key} h={h} se: " + f"py={py_h['se']:.6f} vs r={r_h['se']:.6f}" + ) + assert py_h["t_stat"] == pytest.approx( + r_h["t"], rel=self.SE_RTOL + ), ( + f"path={path_key} h={h} t_stat: " + f"py={py_h['t_stat']:.6f} vs r={r_h['t']:.6f}" + ) + assert int(py_h["n_obs"]) == int(r_h["n_obs"]), ( + f"path={path_key} h={h} n_obs: " + f"py={py_h['n_obs']} vs r={r_h['n_obs']}" + ) + assert py_h["p_value"] == pytest.approx( + r_h["p_value"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} p_value: " + f"py={py_h['p_value']:.6e} vs r={r_h['p_value']:.6e}" + ) + assert py_h["conf_int"][0] == pytest.approx( + r_h["ci_lo"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} ci_lo: " + f"py={py_h['conf_int'][0]:.6f} vs r={r_h['ci_lo']:.6f}" + ) + assert py_h["conf_int"][1] == pytest.approx( + r_h["ci_hi"], rel=self.INFERENCE_RTOL + ), ( + f"path={path_key} h={h} ci_hi: " + f"py={py_h['conf_int'][1]:.6f} vs r={r_h['ci_hi']:.6f}" + ) + + +class TestDCDHDynRParityHeterogeneityWithPlacebo: + """Parity tests for global ``predict_het`` + ``placebo`` (no by_path). + + Per codex R1 P1 #2: the Phase 1A change extended the global + `_compute_heterogeneity_test` loop to cover backward horizons, so + `results.heterogeneity_effects` now includes negative-int keys when + ``placebo=True`` and ``heterogeneity=`` are co-set. This class pins + the global surface independently of the per-path version + (``TestDCDHDynRParityByPathHeterogeneityWithPlacebo``). + + Fixture: scenario 23 (``multi_path_reversible_predict_het_with_placebo_global``) + — same DGP as scenarios 20-22 (reuses ``d20``) so per-horizon + tolerances inherit from ``TestDCDHDynRParityHeterogeneity``. + + R syntax note: ``predict_het = list("X", c(-1))`` triggers + "compute heterogeneity for ALL forward (1..effects) AND ALL placebo + (1..placebo) positions" — same sentinel as scenario 22. + """ + + BETA_RTOL = 1e-6 + BETA_ATOL = 1e-9 + SE_RTOL = 1e-5 + INFERENCE_RTOL = 1e-4 + + def test_parity_multi_path_reversible_predict_het_with_placebo_global( + self, golden_values + ): + """Global heterogeneity on forward + backward horizons.""" + import warnings + + scenario = golden_values.get( + "multi_path_reversible_predict_het_with_placebo_global" + ) + if scenario is None: + pytest.skip( + "scenario 'multi_path_reversible_predict_het_with_placebo_global' " + "not in golden values" + ) + + df = _golden_to_df_with_extra(scenario["data"], extra_cols=["het_x"]) + # Python's `placebo=True` triggers all-backward-horizons + # (1..L_max) placebo computation; the parity check iterates only + # R's emitted subset (placebo=2 in scenario 23 -> backward + # horizons -1, -2). Python's extra backward horizons (e.g., -3 + # if eligible) are allowed and not parity-checked. + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + heterogeneity="het_x", + ) + + assert results.heterogeneity_effects is not None + py_het = results.heterogeneity_effects + + # Forward horizons (positive int keys). + r_predict_het = scenario["results"]["predict_het"] + for h_str, r_h in r_predict_het.items(): + h = int(h_str) + assert h > 0 + assert h in py_het, f"forward horizon {h} missing" + self._assert_horizon_parity(h, py_het[h], r_h) + + # Placebo horizons (negative int keys). + r_placebo_het = _as_dict(scenario["results"].get("placebo_predict_het")) + for h_str, r_h in r_placebo_het.items(): + h = int(h_str) + assert h < 0 + assert h in py_het, ( + f"placebo horizon {h} missing from Python heterogeneity_effects" + ) + self._assert_horizon_parity(h, py_het[h], r_h) + + def _assert_horizon_parity(self, h, py_h, r_h): + """Pin all 6 inference fields against R.""" + assert py_h["beta"] == pytest.approx( + r_h["beta"], rel=self.BETA_RTOL, abs=self.BETA_ATOL + ), f"h={h} beta: py={py_h['beta']:.6f} vs r={r_h['beta']:.6f}" + assert py_h["se"] == pytest.approx(r_h["se"], rel=self.SE_RTOL), ( + f"h={h} se: py={py_h['se']:.6f} vs r={r_h['se']:.6f}" + ) + assert py_h["t_stat"] == pytest.approx(r_h["t"], rel=self.SE_RTOL), ( + f"h={h} t_stat: py={py_h['t_stat']:.6f} vs r={r_h['t']:.6f}" + ) + assert int(py_h["n_obs"]) == int(r_h["n_obs"]), ( + f"h={h} n_obs: py={py_h['n_obs']} vs r={r_h['n_obs']}" + ) + assert py_h["p_value"] == pytest.approx( + r_h["p_value"], rel=self.INFERENCE_RTOL + ), ( + f"h={h} p_value: py={py_h['p_value']:.6e} vs r={r_h['p_value']:.6e}" + ) + assert py_h["conf_int"][0] == pytest.approx( + r_h["ci_lo"], rel=self.INFERENCE_RTOL + ), f"h={h} ci_lo: py={py_h['conf_int'][0]:.6f} vs r={r_h['ci_lo']:.6f}" + assert py_h["conf_int"][1] == pytest.approx( + r_h["ci_hi"], rel=self.INFERENCE_RTOL + ), f"h={h} ci_hi: py={py_h['conf_int'][1]:.6f} vs r={r_h['ci_hi']:.6f}"