Skip to content
Published studies are shown for demonstration purposes, not because they are good or bad research. Demonstrator · items marked Example are invented · what exists today
Search

Templates

Text analysis

By e2er authors

A corpus in, counts and models of its language out: term frequencies, key terms (log-likelihood with the log ratio), collocations, dictionary and sentiment measures with a hand-checked sample, topic models with their stability across seeds, change over time. The corpus design and preprocessing are fixed before the corpus is built; e2er recounts every number from the corpus.

In placeIn e2er 0.16.1empiricaltext analysis
No evaluations yet
Adapt this templateRun it
Anyone can contribute. Improve this template Contribute a template
Input
A question about language and a corpus: books from Project Gutenberg or your own texts
Output
Term frequencies, key terms, collocations, dictionary measures with a hand-checked sample and topic models, every number recounted from the corpus, as a draft paper with a dossier
At the end
When any run ends, e2er builds the PDF, writes the audit trail and the structured export, and pushes to GitHub if you set that up.

Sources

Methods

Ted Dunning (1993). Accurate Methods for the Statistics of Surprise and Coincidence. Computational Linguistics 19(1), 61-74.

Paul Rayson and Roger Garside (2000). Comparing corpora using frequency profiling. Proceedings of the Workshop on Comparing Corpora (ACL 2000), 1-6.

Andrew Hardie (2014). Log Ratio - an informal introduction. ESRC Centre for Corpus Approaches to Social Science (CASS), Lancaster University.

Kenneth Ward Church and Patrick Hanks (1990). Word Association Norms, Mutual Information, and Lexicography. Computational Linguistics 16(1), 22-29.

David M. Blei, Andrew Y. Ng and Michael I. Jordan (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993-1022.

Daniel D. Lee and H. Sebastian Seung (1999). Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788-791.

Derek Greene, Derek O'Callaghan and Pádraig Cunningham (2014). How Many Topics? Stability Analysis for Topic Models. Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2014), Lecture Notes in Computer Science, 498-513.

Justin Grimmer and Brandon M. Stewart (2013). Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis 21(3), 267-297.

Matthew Gentzkow, Bryan Kelly and Matt Taddy (2019). Text as Data. Journal of Economic Literature 57(3), 535-574.

Steps

Greyed steps do not run in the selected mode.

  1. 1
    Corpus design and preprocessing check
    After the data architect: corpus_design.json states the question, the corpus and how it was sampled, the metadata with their sources, the preprocessing (token pattern, lower-casing, stop words, lemmas, n-grams), the groups, the analyses and the sampling bias; dictionaries and topic-model seeds are fixed here, before the analysis.
    check
  2. 2
    Design review
    The researcher reads the plan and the corpus design before the corpus is built.
    your review
  3. 3
    Corpus check: sources, hashes, counts
    After the data analyst: every document with its source, licence and SHA-256; token counts and corpus statistics recomputed with the recorded preprocessing.
    check
  4. 4
    Counts, keyness and topics recomputed
    After the analysis: term counts, keyness (G2, log ratio), collocations, dictionary hits recounted; topic models with seeds, k-selection evidence, stability across seeds recomputed, example documents.
    check
  5. 5
    Hand-checked sample of dictionary hits
    After the text validator: the sample of every dictionary or sentiment measure, every passage found in its document and judged, the precision recomputed (both checks of the results pass before the paper drafter is dispatched; a text validator the plan left out is sent out first: `before`).
    check
  6. 6 specialists
  7. 7
    Improve until it stops getting better
    specialists
  8. 8
    Estimation check
    The results held to the text results contract.
    check
  9. 9
    Figures re-read from the data
    Every figure re-read from the results it names.
    check
  10. 10
    Draft review
    The researcher reads the draft before the reviewers.
    your review
  11. 11
    Self-critique
    Self-critique
    specialists
  12. 12 specialists, parallel
  13. 13 reviewers, parallel
  14. 14 specialists
  15. 15
    Replication package
    Replication package
    specialists
Technical details

Template id text-analysis · file pipelines/text-analysis.toml, checked against pipeline.schema.json.

StepId in the template fileSpecialists
1. Corpus design and preprocessing checkcorpus_design_gate
2. Design reviewreview_design
3. Corpus check: sources, hashes, countscorpus_gate
4. Counts, keyness and topics recomputedtext_results_gate
5. Hand-checked sample of dictionary hitstext_validation_gate
6. Design, data, estimation and draftinitialidea_developer, literature_scanner, data_architect, identification_strategist, econometrics_specialist, data_analyst, paper_drafter
7. Improve until it stops getting betteriterative
8. Estimation checkestimation_gate
9. Figures re-read from the datafigure_gate
10. Draft reviewreview_draft
11. Self-critiqueself_attackself_attacker
12. Polishpolishpolish_numerics, polish_bibliography
13. Review panelreviewmethods_reviewer, data_reviewer, plausibility_reviewer, writing_reviewer, technical_reviewer
14. Revisionrevisionrevisor, patch_revisor
15. Replication packagereplicationreplication_packager

At the end (finalize): compile, audit_export, github_push, structured_export. Modes: single_pass, iterative.

More

Description

Data model
Discipline
Humanities
Method family
Empirical (quantitative)
Design
not specified
Research stage
ReviewRevision and editingReplication
Authors
e2er authors (Software, Methodology)
Usage
not used in published research yet
Source
e2er repository · pipelines/text-analysis.toml · @d8fd8ab
Record
template:text-analysis · JSON

Solid tags are declared by the source or mapped from its terms; dashed tags are inferred by a published rule. Hover a tag for its provenance.

Versions and evaluations

One version, unreleased

One version so far: pipelines/text-analysis.toml in e2er 0.16.1. No evaluation records exist yet.

Evaluate this template Report a failure
Cite this template
Persistent identifiers planned 
@software{e2erauthors2026e2ertextanalysistemplate,
  title   = {e2er text analysis template},
  author  = {e2er authors},
  year    = {2026},
  note    = {e2er template. Persistent identifier planned},
  url     = {https://e2er.org/templates/text-analysis}
}

Connectors this template can use

What a run of this template can reach outside your machine, and what each sends out.

FREDLeaves your machine: Queries only Yahoo Finance (yfinance)Leaves your machine: Queries only Global Macro Database (GMD)Leaves your machine: Queries only USGS Earthquake CatalogLeaves your machine: Queries only NASA Exoplanet ArchiveLeaves your machine: Queries only ESA Gaia ArchiveLeaves your machine: Queries only NASA POWERLeaves your machine: Queries only NOAA Climate Data OnlineLeaves your machine: Queries only World Bank Open DataLeaves your machine: Queries only EurostatLeaves your machine: Queries only Our World in DataLeaves your machine: Queries only WHO Global Health ObservatoryLeaves your machine: Queries only Project GutenbergLeaves your machine: Queries only GitHub repository metadataLeaves your machine: Queries only Eurostat GISCO boundariesLeaves your machine: Queries only Natural EarthLeaves your machine: Queries only

Checks on this template

  • ✓Output contracts · Every step produced the artifact it declared, in the declared form
    Required of every template.
  • ✓Execution reliability · Analysis code ran and wrote a parseable, non-empty result
    Required of every template.
  • ○Numbers · Every number in tables and text matches the output of executed code
    Proposed. Proposed for the floor; runs log it but may not block on it.
  • ○Citations · Every reference resolves to an existing publication via OpenAlex, Semantic Scholar or Crossref
    Proposed. Proposed for the floor; runs log it but may not block on it.
  • ○Verbatim passages · Every claim about a source points to a passage found word for word in that source
    Proposed. Proposed for the floor; runs log it but may not block on it.

Runs of this template

No runs yet.

Use it

Install e2er as described on the Install page, then start a run:

e2er run "<your research question>" --template text-analysis

From e2er 0.16.1. Get started goes from install to publish.

On e2er.org

Loading…

Report a failurePropose a new version

Discussion

Loading…