Review guide

What you need for each task

A study package is one .e2er file. It holds the paper, the code, the data whose source allows passing them on, the results and the list of packages the code needs, each file with its fingerprint. e2er.org keeps no copy: the authors keep the file on GitHub or Zenodo, and the study page shows where.

Install e2er

The installer uv brings its own Python, so you do not need to install Python first.

macOS and Linux, in a terminal

Windows, in PowerShell

Then open a new window and check the installation with e2er --version. With pipx, run pipx install e2er; with your own Python, install e2er into a virtual environment (install shows how).

Download the package

fetch takes the address of a study (the latest version; --version 2 picks another) or of a dossier. It downloads the file from GitHub or Zenodo and keeps it only when its size and fingerprint match what e2er.org records. It then prints the next three commands with the file’s name filled in. The study page offers the same file under Download the study package.

For a double-blind review the journal gives you the file itself and an address of the form https://e2er.org/r/<code>. Skip this step and start with the next one.

See what it holds and check every file

inspect unpacks nothing. It shows the title, the authors, which data the package includes and which it leaves out, the Python version, the pinned packages, the size and the exact commands a rerun would run.

verify checks every file against the package’s list of files, then checks the study itself: every file against its fingerprint, the numbers in the paper against the results, the tables against the estimates and the citations against the bibliography. It works offline. --against also compares the package with the one e2er.org records for that version, reading e2er.org without changing anything. Without --against, verify checks the package against itself only.

Run the code again

Before anything runs, reproduce prints every command the study lists, in order, with the packages it will install, and asks once. A step that loads data from the web asks again.

The authors’ code runs in a Docker container with no network, a read-only system, no folder of yours except a new run folder, and no API key or token. e2er compares every result value with the published one, renders the paper’s tables again and lists each value that differs.

The first run of a study takes longest, because e2er downloads the Python image once and installs the study’s packages. On GitHub’s Linux machines a study that needs no packages beyond Python takes a little over a minute; installing pandas, SciPy, statsmodels and matplotlib adds about a minute on a fast connection. The study’s own scripts then take as long as they took for the authors. A second run with the same packages reuses the environment.

Without Docker

Without Docker, reproduce runs nothing and says so. e2er reproduce --no-sandbox runs the steps on your computer in a new environment, with an empty home folder and no API key, but the code can read every file you can. Use it only for a study whose code you have read or whose authors you trust.

Notes for Windows

Docker Desktop has to run Linux containers, its default with the WSL 2 backend. In Windows-containers mode reproduce refuses and names the switch in Docker Desktop’s menu.

Download the package with fetch when you can. Git for Windows usually changes the line endings of text files when it checks out a repository, and verify then fails and names the files whose line endings changed. To check a study from its repository, clone it with git clone -c core.autocrlf=false.

Windows opens paths of at most 259 characters unless you turn on long paths. If e2er reports a path that is too long, unpack into a short folder such as C:\e2er\study.