Replication Data for: Sign-Congruence, External Validity, and Replication (doi:10.7910/DVN/BDCJBZ)

View:

Part 1: Document Description
Part 2: Study Description
Part 5: Other Study-Related Materials
Entire Codebook

Replication Data for: Sign-Congruence, External Validity, and Replication

doi:10.7910/DVN/BDCJBZ

Harvard Dataverse

2024-10-07

1

Slough, Tara; Tyson, Scott, 2024, "Replication Data for: Sign-Congruence, External Validity, and Replication", https://doi.org/10.7910/DVN/BDCJBZ, Harvard Dataverse, V1

Replication Data for: Sign-Congruence, External Validity, and Replication

doi:10.7910/DVN/BDCJBZ

Slough, Tara (New York University)

Tyson, Scott (University of Rochester)

Harvard Dataverse

Slough, Tara

Code Ocean

https://doi.org/10.7910/DVN/BDCJBZ

Social Sciences

We develop a formal framework for accumulating evidence across studies and apply it to develop theoretical foundations for replication. Our primary contribution is to characterize the relationship between replication and distinct formulations of external validity. Whereas conventional wisdom holds that replication facilitates learning about external validity, we show that this is not, in general, the case. Our results show how comparisons of the magnitude or sign of empirical findings link to distinct concepts of external validity. However, without careful attention to the research design of constituent studies, replication can mislead efforts to assess external validity. We show that two studies must have essentially the same research designs, i.e., be harmonized, in order for their estimates to provide information about any kind of external validity. This result shows that even minor differences in research design between a study and its replication can introduce a discrepancy that is typically overlooked, a problem that becomes more pronounced as the number of studies increases. We conclude by outlining a design-driven approach to replication, which responds to the issues our framework identifies and details how a research agenda can manage them productively.

Label:

-1.gitignore

Text:

application/octet-stream

Label:

.gitignore

Text:

application/octet-stream

Label:

REPRODUCING-1.md

Text:

text/markdown

Label:

REPRODUCING.md

Text:

text/markdown

Label:

01_code-1.R

Text:

type/x-r-syntax

Label:

01_code.R

Text:

type/x-r-syntax

Label:

codebook-1.pdf

Text:

application/pdf

Label:

codebook.pdf

Text:

application/pdf

Label:

LICENSE

Text:

text/plain; charset=US-ASCII

Label:

LICENSE-1

Text:

text/plain; charset=US-ASCII

Label:

readme-1.pdf

Text:

application/pdf

Label:

readme.pdf

Text:

application/pdf

Label:

run

Text:

text/plain; charset=US-ASCII

Label:

run-1

Text:

text/plain; charset=US-ASCII

Label:

LICENSE

Text:

text/plain; charset=US-ASCII

Label:

LICENSE-1

Text:

text/plain; charset=US-ASCII

Label:

meta-1.bib

Text:

application/octet-stream

Label:

meta.bib

Text:

application/octet-stream

Label:

data_power_to_people-1.xlsx

Text:

application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

Label:

data_power_to_people.xlsx

Text:

application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

Label:

Dockerfile

Text:

application/x-docker-file

Label:

Dockerfile-1

Text:

application/x-docker-file

Label:

metadata-1.yml

Text:

application/octet-stream

Label:

metadata.yml

Text:

application/octet-stream

Label:

Figure_2-1.pdf

Text:

application/pdf

Label:

Figure_2.pdf

Text:

application/pdf

Label:

output

Text:

text/plain; charset=UTF-8

Label:

output-1

Text:

text/plain; charset=UTF-8

Label:

Table_A2.tex

Text:

application/x-tex

Label:

Table_E2.tex

Text:

application/x-tex