Replication data for: Explaining Systematic Bias and Nontransparency in US Social Security Administration Forecasts (doi:10.7910/DVN/28323)

View:

Part 1: Document Description
Part 2: Study Description
Part 3: Data Files Description
Part 4: Variable Description
Part 5: Other Study-Related Materials
Entire Codebook

Document Description

Citation

Title:

Replication data for: Explaining Systematic Bias and Nontransparency in US Social Security Administration Forecasts

Identification Number:

doi:10.7910/DVN/28323

Distributor:

Harvard Dataverse

Date of Distribution:

2015-05-08

Version:

Bibliographic Citation:

Kashin, Konstantin; King, Gary; Soneji, Samir, 2015, "Replication data for: Explaining Systematic Bias and Nontransparency in US Social Security Administration Forecasts", https://doi.org/10.7910/DVN/28323, Harvard Dataverse, V1, UNF:6:967llFHgiywsHWWp1cVg9A== [fileUNF]

Study Description

Citation

Title:

Replication data for: Explaining Systematic Bias and Nontransparency in US Social Security Administration Forecasts

Identification Number:

doi:10.7910/DVN/28323

Authoring Entity:

Kashin, Konstantin

King, Gary

Soneji, Samir

Distributor:

Harvard Dataverse

Distributor:

Harvard Dataverse

Date of Deposit:

2014-12-18

Holdings Information:

https://doi.org/10.7910/DVN/28323

Study Scope

Keywords:

Social Sciences

Abstract:

The accuracy of U.S. Social Security Administration (SSA) demographic and financial forecasts is crucial for the solvency of its Trust Funds, other government programs, industry decision making, and the evidence base of many scholarly articles. Because SSA makes public little replication information and uses qualitative and antiquated statistical forecasting methods, fully independent alternative forecasts (and the ability to score policy proposals to change the system) are nonexistent. Yet, no systematic evaluation of SSA forecasts has ever been published by SSA or anyone else --- until a companion paper to this one (King, Kashin, and Soneji, 2015a). We show that SSA's forecasting errors were approximately unbiased until about 2000, but then began to grow quickly, with increasingly overconfident uncertainty intervals. Moreover, the errors are all in the same potentially dangerous direction, making the Social Security Trust Funds look healthier than they actually are. We extend and then attempt to explain these findings with evidence from a large number of interviews we conducted with participants at every level of the forecasting and policy processes. We show that SSA's forecasting procedures meet all the conditions the modern social-psychology and statistical literatures demonstrate make bias likely. When those conditions mixed with potent new political forces trying to change Social Security, SSA's actuaries hunkered down trying hard to insulate their forecasts from strong political pressures. Unfortunately, this otherwise laudable resistance to undue influence, along with their ad hoc qualitative forecasting models, led the actuaries to miss important changes in the input data. Retirees began living longer lives and drawing benefits longer than predicted by simple extrapolations. We also show that the solution to this problem involves SSA or Congress implementing in government two of the central projects of political science over the last quarter century: [1] promoting transparency in data and methods and [2] replacing with formal statistical models large numbers of qualitative decisions too complex for unaided humans to make optimally.

Methodology and Processing

Sources Statement

Data Access

Notes:

This dataset is made available without information on how it can be used. You should communicate with the Contact(s) specified before use.

Other Study Description Materials

Related Publications

Citation

Title:

Kashin, Konstantin; King, Gary; and Samir Soneji. "Explaining Systematic Bias and Nontransparency in US Social Security Administration Forecasts." Political Analysis, 23 (4). <a href="http://j.mp/1H2Dy0f" target="_blank">Link to article</a>

Bibliographic Citation:

File Description--f2526935

File: hmd_fltper_1x1.tab

Number of cases: 8658
No. of variables per record: 10
Type of File: text/tab-separated-values

Notes:

UNF:5:V5SPWleyqd+FvWZUGTyDtA==

Female life table from Human Mortality Database.

File Description--f2526932

File: hmd_mltper_1x1.tab

Number of cases: 8658
No. of variables per record: 10
Type of File: text/tab-separated-values

Notes:

UNF:5:8aS38ziyhsrIMqIHBzYlHw==

Male life table from Human Mortality Database.

File Description--f2548639

File: lt.tab

Number of cases: 377225
No. of variables per record: 12
Type of File: text/tab-separated-values

Notes:

UNF:5:p6rCNLXMC/UFwAq6+EBFoQ==

Life tables for 39 different countries from Human Mortality Database.

File Description--f2548641

File: proposal_scoring.tab

Number of cases: 110
No. of variables per record: 8
Type of File: text/tab-separated-values

Notes:

UNF:5:Lzan4z5/6mY9GnAEZKWACA==

Policy proposal scoring.

File Description--f2548640

File: ssa_balance.tab

Number of cases: 1647
No. of variables per record: 25
Type of File: text/tab-separated-values

Notes:

UNF:5:fmIX4xUFfPObC8chtCX8Jw==

SSA's forecasts of Trust Fund balance and cost rate.

File Description--f2526934

File: ssa_ex.tab

Number of cases: 5152
No. of variables per record: 16
Type of File: text/tab-separated-values

Notes:

UNF:5:VobccIZ318EVz/rno8AGEg==

SSA's forecasts of life expectancy.

File Description--f2526931

File: ssa_mx_2009.tab

Number of cases: 14700
No. of variables per record: 5
Type of File: text/tab-separated-values

Notes:

UNF:5:PcATK+yQysyd3e0tFkNdRw==

Observed and forecast mortality by cause of death from SSA.

File Description--f2526933

File: urod.tab

Number of cases: 20
No. of variables per record: 3
Type of File: text/tab-separated-values

Notes:

UNF:5:8ovwhIDM2Z+CJk9xiZbrzA==

Ultimate rates of decline of mortality.

File Description--f2526930

File: urod_arrows.tab

Number of cases: 19
No. of variables per record: 5
Type of File: text/tab-separated-values

Notes:

UNF:5:5tWNMfPjt9qzsRWjcur9lg==

Helper file used to correctly position arrows in ultimate rate of decline figure.

Variable Description

List of Variables:

Year - Year
Age - Age
mx - mx
qx - qx
ax - ax
lx - lx
dx - dx
Lx - Lx
Tx - Tx
ex - ex
Year - Year
Age - Age
mx - mx
qx - qx
ax - ax
lx - lx
dx - dx
Lx - Lx
Tx - Tx
ex - ex
Year - Year
Age - Age
mx - mx
qx - qx
ax - ax
lx - lx
dx - dx
Lx - Lx
Tx - Tx
ex - ex
country - country
log_qx - log.qx
month - month
day - day
year - year
sponsor - sponsor
variable - variable
change_10 - change.10
change_max_10 - change.max.10
change_summarized_rate - change.summarized.rate
TR - TR
forecast_year - forecast.year
income_rate - income.rate
cost_rate - cost.rate
balance - balance
income_lower - income.lower
income_upper - income.upper
income_I - income.I
income_III - income.III
income_IIA - income.IIA
income_truth - income.truth
cost_lower - cost.lower
cost_upper - cost.upper
cost_I - cost.I
cost_III - cost.III
cost_IIA - cost.IIA
cost_truth - cost.truth
balance_lower - balance.lower
balance_upper - balance.upper
balance_I - balance.I
balance_III - balance.III
balance_IIA - balance.IIA
balance_truth - balance.truth
cost_residual - cost.residual
balance_residual - balance.residual
TR - TR
forecast.year - forecast.year
sex - sex
age - age
forecast - forecast
high - high
low - low
hmd_observed - hmd_observed
ssa_observed - ssa_observed
hmd_residual - hmd_residual
ssa_residual - ssa_residual
lower - lower
upper - upper
ci_width - ci_width
hmd_coverage - hmd_coverage
ssa_coverage - ssa_coverage
age - age
year - year
sex - sex
cause - cause
mx - mx
year - year
urod - urod
type - type
fromx - fromx
fromy - fromy
tox - tox
toy - toy
type - type

Variables

Year

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:WlL1Dd16LSc6FmT8l9e+xg==

Age

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:YOnwescg7BYC46CwVCCmbw==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:ZEifycLWEiSfvAOzleDM9Q==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:1JmT8Bc8Tm1CzmwHe9bNXg==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:wBYYemvDEeUUX/ofJJ5X3Q==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:d9TtSuH4twNfumEMOoXFNQ==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:EoGzSznZo+YlThP22Q2Q1Q==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:bAsRUY6GKFH96HBeRzLV4g==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:dIevgMo9ztTtp4YScQEtBw==

f2526935 Location:

Variable Format: numeric

Notes: UNF:5:vVE8WaRD1nJud1qcf9IPig==

Year

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:WlL1Dd16LSc6FmT8l9e+xg==

Age

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:YOnwescg7BYC46CwVCCmbw==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:fmId64I5wE9hZ4sVovXmPw==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:Z4264LJtO2INje7JUdoslg==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:UiQSZPFHpSO52Ms6CUD0oA==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:twiCn0uXLAfZATfQFAdozA==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:3l9/i+Vx0NLQSzjLndBbdA==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:pP+jTJHdJIAsYHidWphbZA==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:YqGPDG3Gh9N+LWMzR9Jh+g==

f2526932 Location:

Variable Format: numeric

Notes: UNF:5:ZmW4IaNsfb79NEPv1OjBkw==

Year

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:q8cyrZ6gZizaejf+wg0dbw==

Age

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:hzxC35mmYTj/x7XE+QVOiQ==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:NwyBz8JZ38DgLJybazPJ3w==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:K29Pz0m0QczP3nryuMoT8A==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:nvitmRnZudpCbfuH1aeTRQ==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:co6vY83Oimldmj5hSQWLzA==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:8YukwhWvujVI52WZF02aBg==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:0unFpWmejtX5Bn35fjBBvQ==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:Dys8ptgU/MT9z0axX5XHFA==

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:3o2HRph+Bwbju/fGk6g34Q==

country

f2548639 Location:

Variable Format: character

Notes: UNF:5:Yx8w10fsBoZLPSQu/dtjjA==

log.qx

f2548639 Location:

Variable Format: numeric

Notes: UNF:5:A7V3CzvlgrVQ8woKXQLLyQ==

month

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:Iqa65Ql/D8f2dxvjsqS5Xg==

day

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:/25cbf5clcQXy2kToUBnNQ==

year

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:1i1ln22106cOtz+6P5x2TA==

sponsor

f2548641 Location:

Variable Format: character

Notes: UNF:5:ags1cOYUoMKdZiq4BPx/rA==

variable

f2548641 Location:

Variable Format: character

Notes: UNF:5:WyrB4BYQ3DQK3TrKpmPifQ==

change.10

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:JgTjLCbOkkoDvX6PJN07vg==

change.max.10

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:MUg6UQWokElfmuzWhVAhZA==

change.summarized.rate

f2548641 Location:

Variable Format: numeric

Notes: UNF:5:XS0D7LIDw3IPF8PUYA5CAg==

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:PGusHaQ/QCB6wdaEcsH+NQ==

forecast.year

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:dNqhteNINwdYdaXqQF4cag==

income.rate

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:tWxcQgstqDMFFa7hnN1SMw==

cost.rate

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:6qPAWxXNNF9gibaDStCHkw==

balance

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:FYKz9hqZvGWfJwLfZwYErA==

income.lower

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:8B310cEeX3+xsye54OptUg==

income.upper

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:wdDXd0v15JW7A+ESo5bduA==

income.I

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:AalZYY08kj00S4C3BqhJHA==

income.III

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:32GHUi1LSa+DRVASWuo/HA==

income.IIA

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:94UxjVPhAXOkcVYESddebg==

income.truth

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:82mR6eMff2b4Jxf8lhz0JQ==

cost.lower

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:G0U/ZWyRhDZLJjIHmr40qQ==

cost.upper

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:WP5NgXm7zFCXT3lYM1g1vQ==

cost.I

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:iny6U/0uEUki0JghuB5C4g==

cost.III

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:m2KcaG5TJwFJ7M62QBBNug==

cost.IIA

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:oMCxmV+3II0aDuLvdpwaOg==

cost.truth

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:ftVBwZHDARli4AqfMLlteg==

balance.lower

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:/wtAy79jthKJGz+1f5eSXA==

balance.upper

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:wKiWpXwafO3W/hisJ/fGzw==

balance.I

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:EvfonWxKCDvFKv9GKUMkZg==

balance.III

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:CuBLzZIB+idf2f/3gWCTcQ==

balance.IIA

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:1A5ngD9ihBDeQs6GtSX2qQ==

balance.truth

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:JRmZymF43653ebWjhfvaJQ==

cost.residual

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:KJvIY1DvLjhtuiVWeoNqBQ==

balance.residual

f2548640 Location:

Variable Format: numeric

Notes: UNF:5:7ue+hdQLCB3onzK/riuy8Q==

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:ab33kbFRKlsrfufWh6tdpA==

forecast.year

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:0sKnGSxpjj5/b/6MfNp7dg==

sex

f2526934 Location:

Variable Format: character

Notes: UNF:5:HV+6LPMWdUGdAu2JWfinHQ==

age

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:mGZ18Ui5QFPz2zm2Lxmlvg==

forecast

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:AC9Ukkxrl0Rxct1R+RPyLg==

high

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:BOUTr2MfjxyxuQTuKfNvdg==

low

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:7JS3z0dSEwLPNoKtBmmkLg==

hmd_observed

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:RtudmmRNHb11mv2UB/RVHg==

ssa_observed

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:RtudmmRNHb11mv2UB/RVHg==

hmd_residual

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:RtudmmRNHb11mv2UB/RVHg==

ssa_residual

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:RtudmmRNHb11mv2UB/RVHg==

lower

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:bR/7G3bZ3nffYzxr2KJ3LA==

upper

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:liVxAe9vBgM+ktaBEvXAFA==

ci_width

f2526934 Location:

Variable Format: numeric

Notes: UNF:5:szGvjNMCv5KHA5apx81trg==

hmd_coverage

f2526934 Location:

Value	Label	Frequency	Text
1.	TRUE	222
0.	FALSE	394

Variable Format: numeric

Notes: UNF:5:+RMQY9sOTXM4vqeHavxdQw==

ssa_coverage

f2526934 Location:

Value	Label	Frequency	Text
0.	FALSE	321
1.	TRUE	295

Variable Format: numeric

Notes: UNF:5:Bd/z+mtS3KOmeeQdRhbdcQ==

age

f2526931 Location:

Variable Format: numeric

Notes: UNF:5:zc39SFuzlQw68/H0/kx1Lg==

year

f2526931 Location:

Variable Format: numeric

Notes: UNF:5:7uRAGyqk5n7OoFaQPCEZaQ==

sex

f2526931 Location:

Value	Label	Frequency	Text
1.	female	7350
2.	male	7350

Variable Format: numeric

Notes: UNF:5:qKrKxoB9jlMEcyWDgzyB4Q==

cause

f2526931 Location:

Value	Label	Frequency
7.	violence	2100
2.	diabetes	2100
3.	heart	2100
4.	other	2100
5.	resp	2100
6.	vascular	2100
1.	cancer	2100

Variable Format: numeric

Notes: UNF:5:iavHXt3fQ/2cXNY2tp/K6g==

f2526931 Location:

Variable Format: numeric

Notes: UNF:5:hrCJVn1SuWOtvMOyjLyhqw==

year

f2526933 Location:

Variable Format: numeric

Notes: UNF:5:evDyK7A+C+o8YeXrEcbmfA==

urod

f2526933 Location:

Variable Format: numeric

Notes: UNF:5:rtJV+cKdVIMj59ioeOmL2Q==

type

f2526933 Location:

Value	Label	Frequency	Text
2.	TR	16
1.	TAR	4

Variable Format: numeric

Notes: UNF:5:CX3Qls3apYlqE+Eg/smKBA==

fromx

f2526930 Location:

Variable Format: numeric

Notes: UNF:5:5YJAmEPDPUUwdKodQWWaOA==

fromy

f2526930 Location:

Variable Format: numeric

Notes: UNF:5:19CzgBbQNmPMD2UaeECRQw==

tox

f2526930 Location:

Variable Format: numeric

Notes: UNF:5:qpBUUinQjSSbfVgmPAQd7A==

toy

f2526930 Location:

Variable Format: numeric

Notes: UNF:5:K7X4/JyaURQVTYblO6sEFA==

type

f2526930 Location:

Value	Label	Frequency
2.	ssa-tech	4
3.	tech-ssa	4
1.	ssa	11

Variable Format: numeric

Notes: UNF:5:oSF0s3VmCpc/BgmJJXPGAg==

Other Study-Related Materials

Label:

analysis.pdf

Text:

All figures (color) presented in paper.

Notes:

application/pdf

Other Study-Related Materials

Label:

analysis.Rnw

Text:

Sweave file that can be compiled using the knitr package in R. knitr runs the R code, outputs the figures into a figures subdirectory, and creates the analysis.tex file.

Notes:

text/plain; charset=US-ASCII

Other Study-Related Materials

Label:

analysis_grayscale.pdf

Text:

All figures (grayscale) presented in paper.

Notes:

application/pdf

Other Study-Related Materials

Label:

analysis_grayscale.Rnw

Text:

Sweave file that can be compiled using the knitr package in R. knitr runs the R code, outputs the figures into a figures subdirectory, and creates the analysis_grayscale.tex file.

Notes:

text/plain; charset=US-ASCII

Other Study-Related Materials

Label:

geom_segment_plus.R

Text:

Helper function used to draw ultimate rate of decline plot.

Notes:

text/plain; charset=US-ASCII

Other Study-Related Materials

Label:

KKS_PA2015.zip

Text:

Full replication file. Download this to run Makefile.

Notes:

application/zip

Other Study-Related Materials

Label:

Makefile

Text:

Makefile that controls workflow for analysis.

Notes:

text/plain; charset=US-ASCII

Other Study-Related Materials

Label:

README.pdf

Text:

Readme file.

Notes:

application/pdf