Replication data for: Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance (doi:10.7910/DVN/26555)

View:

Part 1: Document Description
Part 2: Study Description
Part 3: Data Files Description
Part 4: Variable Description
Part 5: Other Study-Related Materials
Entire Codebook

(external link)

Document Description

Citation

Title:

Replication data for: Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance

Identification Number:

doi:10.7910/DVN/26555

Distributor:

Harvard Dataverse

Date of Distribution:

2014-06-23

Version:

2

Bibliographic Citation:

Gross, Justin, 2014, "Replication data for: Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance", https://doi.org/10.7910/DVN/26555, Harvard Dataverse, V2, UNF:5:Ir/KmbFq2NouqaZ1Z4J/Dw== [fileUNF]

Study Description

Citation

Title:

Replication data for: Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance

Identification Number:

doi:10.7910/DVN/26555

Authoring Entity:

Gross, Justin (University of North Carolina at Chapel Hill)

Producer:

Justin H. Gross

Distributor:

Harvard Dataverse

Access Authority:

Justin H. Gross

Depositor:

JHGross

Date of Deposit:

2014-06-22

Date of Distribution:

2013

Holdings Information:

https://doi.org/10.7910/DVN/26555

Study Scope

Keywords:

Social Sciences, Methodology, Hypothesis testing, Empirical methods, NHST, Statistical inference, Statistical significance testing

Abstract:

For over a half-century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation of research results. A long list of criticisms has fueled the so-called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (scientific, substantive, social, political, etc.) significance. I introduce a simple, intuitive approach that grounds testing in subject-area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to focus upon real-world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation.

Time Period:

1990-1990

Date of Collection:

1990-11-07-1991-01-21

Country:

United States

Geographic Coverage:

30 U.S. states; see http://www.electionstudies.org/studypages/1990post/1990post.htm

Geographic Unit(s):

see ANES for more information

Methodology and Processing

Sources Statement

Data Access

Notes:

<a href="http://creativecommons.org/publicdomain/zero/1.0">CC0 1.0</a>

Other Study Description Materials

Related Publications

Citation

Title:

Gross, Justin H. 2015. “Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance.” <i>American Journal of Political Science</i> 59 (3): 775-788.

Identification Number:

10.1111/ajps.12149

Bibliographic Citation:

Gross, Justin H. 2015. “Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance.” <i>American Journal of Political Science</i> 59 (3): 775-788.

File Description--f2475008

File: ANES-1990_subset-variables_Testing-What-Matters.tab

  • Number of cases: 1980

  • No. of variables per record: 34

  • Type of File: text/tab-separated-values

Notes:

UNF:5:Ir/KmbFq2NouqaZ1Z4J/Dw==

Subset of variables from ANES 1990 Time Series Study, for reanalysis of Iyengar, S., & Simon, A. (1993). News Coverage of the Gulf Crisis and Public Opinion A Study of Agenda-Setting, Priming, and Framing. Communication research, 20(3), 365-383.

Variable Description

List of Variables:
  • V900072 - #DAYS PST WK WATCH TV
  • V900320 - R'S PARTY ID: SUMMARY
  • V900357 - US FORCES P-GULF RIGHT?
  • V900358 - WHAT DO NOW IN P-GULF?
  • V900395 - DOES R KNOW: DAN QUAYLE
  • V900396 - DOES R KNOW:GEO MITCHELL
  • V900397 - DOES R KNOW:WM REHNQUIST
  • V900398 - DOES R KNOW:ML GORBACHEV
  • V900399 - DOES R KNOW:MT THATCHER
  • V900400 - DOES R KNOW:NEL MANDELA
  • V900401 - DOES R KNOW: TOM FOLEY
  • V900439 - DEFENSE SPND SCALE: R
  • V900547 - R'S GENDER(GIVEN BY INF)
  • V900549 - R'S RACE
  • V900557 - SUMMARY: R'S EDUCATION
  • USforce - RECODE of V900357 (US FORCES P-GULF RIGHT?)
  • doNow - RECODE of V900358 (WHAT DO NOW IN P-GULF?)
  • dv
  • TVnews - RECODE of V900072 (#DAYS PST WK WATCH TV)
  • quayle - RECODE of V900395 (DOES R KNOW: DAN QUAYLE)
  • mitchell - RECODE of V900396 (DOES R KNOW:GEO MITCHELL)
  • rehnquist - RECODE of V900397 (DOES R KNOW:WM REHNQUIST)
  • gorbachev - RECODE of V900398 (DOES R KNOW:ML GORBACHEV)
  • thatcher - RECODE of V900399 (DOES R KNOW:MT THATCHER)
  • mandela - RECODE of V900400 (DOES R KNOW:NEL MANDELA)
  • foley - RECODE of V900401 (DOES R KNOW: TOM FOLEY)
  • knowledge
  • partyID - RECODE of V900320 (R'S PARTY ID: SUMMARY)
  • nonwhite - RECODE of V900549 (R'S RACE)
  • male - RECODE of V900547 (R'S GENDER(GIVEN BY INF))
  • educ - RECODE of V900557 (SUMMARY: R'S EDUCATION)
  • defense - RECODE of V900439 (DEFENSE SPND SCALE: R)
  • male_knowl
  • nonwhite_knowl

Variables

#DAYS PST WK WATCH TV

f2475008 Location:

Value

Label

Frequency

Text

7.

7. EVERY DAY

1071

8.

8. DK

1

2.

2. TWO

149

9.

9. NA

4

4.

4. FOUR

128

0.

0. NONE

188

6.

6. SIX

44

5.

5. FIVE

131

1.

1. ONE

90

3.

3. THREE

174

Variable Format: numeric

Notes: UNF:5:JgqzlvlE1LCKVGTVqs39nw==

R'S PARTY ID: SUMMARY

f2475008 Location:

Value

Label

Frequency

Text

8.

8. APOLITICAL

31

5.

5. WEAK REP

293

4.

4. INDEPENDENT-REP

233

3.

3. INDEPENDENT-INDEPENDENT

205

1.

1. WEAK DEMOCRAT

377

9.

9. NA

9

2.

2. INDEPENDENT-DEMOCRAT

245

7.

7. OTHER-MINOR PARTY

5

6.

6. STRONG REP

189

0.

0. STRONG DEMOCRAT

393

Variable Format: numeric

Notes: UNF:5:VtOvAGPzcqTNBKvziFqLyg==

US FORCES P-GULF RIGHT?

f2475008 Location:

Value

Label

Frequency

Text

8.

8. DK

97

1.

1. DID THE RIGHT THING

1087

5.

5. SHOULD HAVE STAYED OUT

715

3.

3. DEPENDS (VOL.)

46

7.

7. OTHER...(Make Card)

22

9.

9. NA

13

Variable Format: numeric

Notes: UNF:5:XPCE3tDKSnKXOr0drUR0DQ==

WHAT DO NOW IN P-GULF?

f2475008 Location:

Value

Label

Frequency

Text

270.

270. 2 and 7

0

137.

137. 1, 3 and 7

0

237.

237. 2, 3 and 7

0

170.

170. 1 and 7

0

2.

2. TRY HARDER TO FIND A DIPLOMATIC SOLUTI

0

230.

230. 2 and 3

0

370.

370. 3 and 7

0

234.

234. 2, 3 and 4

0

500.

500. 1, 2, 3, 4 and 7

0

4.

4. TAKE TOUGHER MILITARY ACTION

0

240.

240. 2 and 4

0

124.

124. 1, 2 and 4

0

998.

998. DK

0

470.

470. 4 and 7

0

7.

7. OTHER (SPECIFY)

0

130.

130. 1 and 3

0

134.

134. 1, 3 and 4

0

123.

123. 1, 2 and 3

0

340.

340. 3 and 4

0

147.

147. 1, 4 and 7

0

247.

247. 2, 4 and 7

0

120.

120. 1 and 2

0

403.

403. 1, 2, 4 and 7

0

1.

1. PULL OUT U.S. FORCES ENTIRELY

0

401.

401. 1, 2, 3 and 4

0

402.

402. 1, 2, 3 and 7

0

347.

347. 3, 4 and 7

0

3.

3. TIGHTEN THE ECONOMIC EMBARGO

0

405.

405. 2, 3, 4 and 7

0

127.

127. 1, 2 and 7

0

404.

404. 1, 3, 4 and 7

0

140.

140. 1 and 4

0

999.

999. NA

0

Variable Format: numeric

Notes: UNF:5:/PPh0Tu8YWPlGvDg0Xm9tg==

DOES R KNOW: DAN QUAYLE

f2475008 Location:

Value

Label

Frequency

Text

8.

8. DK

339

9.

9. NA

5

5.

5. Incomplete or wrong

36

1.

1. Correctly identifies

1600

Variable Format: numeric

Notes: UNF:5:Zgmhqu0jORCFzC2Q8+RdsA==

DOES R KNOW:GEO MITCHELL

f2475008 Location:

Value

Label

Frequency

Text

9.

9. NA

7

8.

8. DK

1680

1.

1. Correctly identifies

46

5.

5. Incomplete or wrong

247

Variable Format: numeric

Notes: UNF:5:xBe+dygqPYGyZmm5Tc2pow==

DOES R KNOW:WM REHNQUIST

f2475008 Location:

Value

Label

Frequency

Text

1.

1. Correctly identifies

84

8.

8. DK

1548

9.

9. NA

8

5.

5. Incomplete or wrong

340

Variable Format: numeric

Notes: UNF:5:g2t5lgxmxDWsVtVsKbWYmA==

DOES R KNOW:ML GORBACHEV

f2475008 Location:

Value

Label

Frequency

Text

1.

1. Correctly identifies

1367

8.

8. DK

347

9.

9. NA

6

5.

5. Incomplete or wrong

260

Variable Format: numeric

Notes: UNF:5:erkxV4lS5cRKxnTEudXaQA==

DOES R KNOW:MT THATCHER

f2475008 Location:

Value

Label

Frequency

Text

8.

8. DK;

427

2.

2. Thatcher as former Prime Minister

355

5.

5. Incomplete or wrong

560

1.

1. Thatcher as Prime Minister

632

9.

9. NA

6

Variable Format: numeric

Notes: UNF:5:UHsYEunCqfQztIUJmGx2Pg==

DOES R KNOW:NEL MANDELA

f2475008 Location:

Value

Label

Frequency

Text

5.

5. Incomplete or wrong

927

1.

1. Correctly identifies

301

8.

8. DK

744

9.

9. NA

8

Variable Format: numeric

Notes: UNF:5:3QF9HA1uil/YLqRBn3pMjQ==

DOES R KNOW: TOM FOLEY

f2475008 Location:

Value

Label

Frequency

Text

8.

8. DK

1639

1.

1. Correctly identifies

175

5.

5. Incomplete or wrong

160

9.

9. NA

6

Variable Format: numeric

Notes: UNF:5:99PIozeVLvlqeZB/2ubk9w==

DEFENSE SPND SCALE: R

f2475008 Location:

Value

Label

Frequency

Text

0.

0. HAVEN'T THOUGHT MUCH

259

8.

8. DK

17

7.

7. GREATLY INCREASE DEFENSE SPENDING

80

9.

9. NA

14

1.

1. GREATLY DECREASE DEFENSE SPENDING

217

Variable Format: numeric

Notes: UNF:5:VanCU+mq+Sq6QehnXfBNOg==

R'S GENDER(GIVEN BY INF)

f2475008 Location:

Value

Label

Frequency

Text

1.

1. Male

895

2.

2. Female

1085

9.

9. NA

0

Variable Format: numeric

Notes: UNF:5:s4WTh/+J0L7+IUKVKuOW8A==

R'S RACE

f2475008 Location:

Value

Label

Frequency

Text

4.

4. ASIAN OR PACIFIC ISLANDER

31

9.

9. NA

3

3.

3. AMERICAN INDIAN OR ALASKAN NATIVE

25

2.

2. BLACK

257

1.

1. WHITE

1664

Variable Format: numeric

Notes: UNF:5:aXWOwcNt1ggdBSK3qHV+ow==

SUMMARY: R'S EDUCATION

f2475008 Location:

Value

Label

Frequency

Text

7.

7. Advanced degree

0

2.

2. 9-11 grades, no further schooling

0

99.

99. NA

0

3.

3. High school diploma

0

1.

1. 8 grades or less

0

5.

5. Junior or community college

0

98.

98. DK

0

4.

4. More than 12 years

0

6.

6. BA level degrees

0

Variable Format: numeric

Notes: UNF:5:LlJ6Rw8v2yv6fJvGn4N03A==

RECODE of V900357 (US FORCES P-GULF RIGHT?)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:6h1awhgMYki3vxqLS69wyQ==

RECODE of V900358 (WHAT DO NOW IN P-GULF?)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:s4Gkf8oIj2Sa2cQInDx6dQ==

dv

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:K8klT/009sTamU1JY9rFjg==

RECODE of V900072 (#DAYS PST WK WATCH TV)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:dzREZ8paeIR5XUSlGLt/Lg==

RECODE of V900395 (DOES R KNOW: DAN QUAYLE)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:98x9WAV95y4I1wxmX7Nfsg==

RECODE of V900396 (DOES R KNOW:GEO MITCHELL)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:TEqiTRRKrxzX3otRK+VArA==

RECODE of V900397 (DOES R KNOW:WM REHNQUIST)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:EOe5KA/EOe4R38DqjmCIMA==

RECODE of V900398 (DOES R KNOW:ML GORBACHEV)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:fLqWdgwrtg5wIN0aAnsUZg==

RECODE of V900399 (DOES R KNOW:MT THATCHER)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:tIsRdoGX4J+5Fgpyh00ymA==

RECODE of V900400 (DOES R KNOW:NEL MANDELA)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:CStIdJocZlUidDsWXFVERA==

RECODE of V900401 (DOES R KNOW: TOM FOLEY)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:JvyC1rEv9qcsFy1225dJWQ==

knowledge

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:sZnq44a9B6TMVARoGp5CEQ==

RECODE of V900320 (R'S PARTY ID: SUMMARY)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:1FGuTct2Idd8VUcFhgEEvg==

RECODE of V900549 (R'S RACE)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:raiMu+AYuwehXyXODzF+tw==

RECODE of V900547 (R'S GENDER(GIVEN BY INF))

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:DUxT281PVbvNDTSyHsL5PQ==

RECODE of V900557 (SUMMARY: R'S EDUCATION)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:jO0G3U9X8ujHXzeBTuV+0Q==

RECODE of V900439 (DEFENSE SPND SCALE: R)

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:zmlDtDqp1IxRJAlLUzWAFw==

male_knowl

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:Q62WEtB+2j2sL+tkvc2Pyg==

nonwhite_knowl

f2475008 Location:

Variable Format: numeric

Notes: UNF:5:s0IQIXQHOeMTAMDvlPV9bg==

Other Study-Related Materials

Label:

ANES-1990_recoding-and-analysis-Stata-do-file_JHGross_Testing-What-Matters.txt

Text:

Recoding of original ANES 1990 data to create the .dta file provided, as well as command for regression

Notes:

text/plain; charset=US-ASCII