Skip to content
Published studies are shown for demonstration purposes, not because they are good or bad research. Demonstrator · items marked Example are invented · what exists today
Search

Templates

Policy evaluation (difference-in-differences)

By e2er authors

What a policy changed, by difference-in-differences: the design (treatment timing, comparison group, estimator) is checked against the data before estimation, and the event study, pre-trends test and placebo or sensitivity analysis after it.

In placeIn e2er 0.16.1empiricaldifference-in-differences
No evaluations yet
Adapt this templateRun it
Anyone can contribute. Improve this template Contribute a template
Input
A question about what a policy changed, and a panel with the treatment timing
Output
Difference-in-differences estimates with the event study, a pre-trends test and a placebo or sensitivity analysis, as a draft paper with a dossier
At the end
When any run ends, e2er builds the PDF, writes the audit trail and the structured export, and pushes to GitHub if you set that up.

Sources

Methods

Brantly Callaway and Pedro H. C. Sant'Anna (2021). Difference-in-Differences with multiple time periods. Journal of Econometrics 225(2), 200-230.

Andrew Goodman-Bacon (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics 225(2), 254-277.

Liyang Sun and Sarah Abraham (2021). Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Journal of Econometrics 225(2), 175-199.

Clément de Chaisemartin and Xavier D'Haultfœuille (2020). Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects. American Economic Review 110(9), 2964-2996.

Kirill Borusyak, Xavier Jaravel and Jann Spiess (2024). Revisiting Event-Study Designs: Robust and Efficient Estimation. Review of Economic Studies 91(6), 3253-3285.

Jonathan Roth, Pedro H. C. Sant'Anna, Alyssa Bilinski and John Poe (2023). What's trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics 235(2), 2218-2244.

Jonathan Roth (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights 4(3), 305-322.

Ashesh Rambachan and Jonathan Roth (2023). A More Credible Approach to Parallel Trends. Review of Economic Studies 90(5), 2555-2591.

Steps

Greyed steps do not run in the selected mode.

  1. 1
    Difference-in-differences design check
    Before estimation. The identification strategist writes did_design.json (outcome, panel, treatment timing, comparison group, estimator; schema in skills/files/econometrics/did-practice.md). The check reads the panel in data.db and stops the run when the outcome or timing is not in the data, when there is no comparison group, when timing is staggered and the estimator is two-way fixed effects (it asks for a heterogeneity-robust estimator), or when no cohort has `min_pre_periods` periods before treatment. A failed check sends the identification strategist back once with the reasons, then stops the run for the researcher..
    check
  2. 2
    Pre-registration
    The design, frozen before any estimation (approve it to go on; edit it first to change it)..
    your review
  3. 3
    Pre-trends and event-study check
    After estimation, before drafting. The results must hold the event-study path with at least `min_pre_periods` pre-treatment periods, a joint pre-trends test (a rejection at `pretrend_alpha` stops the run unless a Rambachan-Roth sensitivity analysis is reported), a placebo test or a sensitivity analysis, and the declared estimator under `main`. The event-study plot is drawn from the results. A failed check sends the econometrics specialist back once, then stops the run..
    check
  4. 4 specialists
  5. 5
    Improve until it stops getting better
    specialists
  6. 6
    Estimation check
    check
  7. 7
    Self-critique
    Self-critique
    specialists
  8. 8 specialists, parallel
  9. 9 reviewers, parallel
  10. 10 specialists
  11. 11
    Replication package
    Replication package
    specialists
Technical details

Template id policy-evaluation · file pipelines/policy-evaluation.toml, checked against pipeline.schema.json.

StepId in the template fileSpecialists
1. Difference-in-differences design checkdid_design_gate
2. Pre-registrationpreregister
3. Pre-trends and event-study checkdid_results_gate
4. Design, data, estimation and draftinitialidea_developer, literature_scanner, data_architect, identification_strategist, econometrics_specialist, data_analyst, paper_drafter
5. Improve until it stops getting betteriterative
6. Estimation checkestimation_gate
7. Self-critiqueself_attackself_attacker
8. Polishpolishpolish_formula, polish_numerics, polish_bibliography
9. Review panelreviewidentification_reviewer, data_reviewer, methods_reviewer, writing_reviewer, technical_reviewer
10. Revisionrevisionrevisor, patch_revisor
11. Replication packagereplicationreplication_packager

At the end (finalize): compile, audit_export, github_push, structured_export. Modes: single_pass, iterative.

More

Description

Data model
Discipline
Economics
Method family
Empirical (quantitative)
Design
Difference-in-differences
Research stage
ReviewRevision and editingReplication
Authors
e2er authors (Software, Methodology)
Usage
not used in published research yet
Source
e2er repository · pipelines/policy-evaluation.toml · @d8fd8ab
Record
template:policy-evaluation · JSON

Solid tags are declared by the source or mapped from its terms; dashed tags are inferred by a published rule. Hover a tag for its provenance.

Versions and evaluations

One version, unreleased

One version so far: pipelines/policy-evaluation.toml in e2er 0.16.1. No evaluation records exist yet.

Evaluate this template Report a failure
Cite this template
Persistent identifiers planned 
@software{e2erauthors2026e2erpolicyevaluationdiff,
  title   = {e2er policy evaluation (difference-in-differences) template},
  author  = {e2er authors},
  year    = {2026},
  note    = {e2er template. Persistent identifier planned},
  url     = {https://e2er.org/templates/policy-evaluation}
}

Connectors this template can use

What a run of this template can reach outside your machine, and what each sends out.

FREDLeaves your machine: Queries only Yahoo Finance (yfinance)Leaves your machine: Queries only Global Macro Database (GMD)Leaves your machine: Queries only USGS Earthquake CatalogLeaves your machine: Queries only NASA Exoplanet ArchiveLeaves your machine: Queries only ESA Gaia ArchiveLeaves your machine: Queries only NASA POWERLeaves your machine: Queries only NOAA Climate Data OnlineLeaves your machine: Queries only World Bank Open DataLeaves your machine: Queries only EurostatLeaves your machine: Queries only Our World in DataLeaves your machine: Queries only WHO Global Health ObservatoryLeaves your machine: Queries only Project GutenbergLeaves your machine: Queries only GitHub repository metadataLeaves your machine: Queries only Eurostat GISCO boundariesLeaves your machine: Queries only Natural EarthLeaves your machine: Queries only

Checks on this template

  • ✓Output contracts · Every step produced the artifact it declared, in the declared form
    Required of every template.
  • ✓Execution reliability · Analysis code ran and wrote a parseable, non-empty result
    Required of every template.
  • ○Numbers · Every number in tables and text matches the output of executed code
    Proposed. Proposed for the floor; runs log it but may not block on it.
  • ○Citations · Every reference resolves to an existing publication via OpenAlex, Semantic Scholar or Crossref
    Proposed. Proposed for the floor; runs log it but may not block on it.
  • ○Verbatim passages · Every claim about a source points to a passage found word for word in that source
    Proposed. Proposed for the floor; runs log it but may not block on it.

Runs of this template

No runs yet.

Use it

Install e2er as described on the Install page, then start a run:

e2er run "<your research question>" --template policy-evaluation

From e2er 0.16.1. Get started goes from install to publish.

On e2er.org

Loading…

Report a failurePropose a new version

Discussion

Loading…