2023 Consumer Spending by US Census Block Group

Version 1.1

Bryan, Michael, 2025, "2023 Consumer Spending by US Census Block Group", https://doi.org/10.7910/DVN/SNUUGO, Harvard Dataverse, V1, UNF:6:knKH4yMTu4hNBraKDOs4hQ== [fileUNF]

Learn about Data Citation Standards.

Contact Owner

Dataset Metrics

10 Downloads

Description	blockgroupspending Opportunity US Consumers express their behavior in a number of ways, but critically in their spending decisions. The US Bureau of Labor Statistics is charged with publishing spending activity and provides its Consumer Expenditure Survey (CEX) annually with US totals, with selected states (40) and cities (23). Limited to aggregates, the survey only needs 10s of thousands of observations in the original collection. While this is sufficient for macroeconomic use, the volume gives a weak basis for estimating lower levels of geography. In addition, the CEX includes demographic measurements that are similar, but not directly related, to Census variables. So, the CEX does not integtate well with the American Commuity Survey or other Census publications. This *blockgroupspending* publication by Open Environments attempts to address this problem by using the BLS' Public Microdata (PUMD) sample to allocate CEX spending categories across 220,000 US Census block group geographies. For each block group, the effort applies two models to estimate: total consumer spending (regression) distribution of spending across spending categories (penetration) including Food, Transportation, Housing and Health costs. Ultimately, these project spending on block groups that can be joined to US Census publications for additional demographics. Understanding the results requires awareness of the BLS' CEX data structures. This is available in the markdown file named oe_bls_cex_EDA.md The publication is made together with the source python code and notebooks used for repeatability. The materials are maintained under version control at https://github.com/OpenEnvironments/blockgroupspending. All feedback and development requests are welcome. Model details -- The CEX publication includes many files reflecting detailed 'diary' surveys capturing spend on thousands of items every two weeks family 'interviews' collecting household spending over the previous 3 months The models are trained upon the latter, 'FMLI' files. The regression model uses extreme gradient boosting, or XGBoost methods that apply many decision trees to iteratively correct prediction error. The subcategory models also use tree based methods, trained upon a the family interview details. The spending variables are named, following the BLS' CEX convention: \|Variable\|Definition\|2023\|pct\| \|---\|---\|---\|---\| \|TOTEXP\|Average annual expenditures\|77280\|\| \|FOOD\|Food\|9985\|0.129\| \|ALCBEV\|Alcoholic beverages\|637\|0.008\| \|HOUS\|Housing\|25436\|0.329\| \|APPAR\|Apparel and services\|2041\|0.026\| \|TRANS\|Transportation\|13174\|0.17\| \|HEALTH\|Healthcare\|6159\|0.08\| \|ENTERT\|Entertainment\|3635\|0.047\| \|PERSCA\|Personal care products and services\|950\|0.012\| \|READ\|Reading\|117\|0.002\| \|EDUCA\|Education\|1656\|0.021\| \|TOBACC\|Tobacco products and smoking supplies\|370\|0.005\| \|MISC\|Miscellaneous\|1184\|0.015\| \|CASHCO\|Cash contributions\|2378\|0.031\| \|RETPEN\|Personal insurance and pensions\|9556\|0.124\| During the exploratory phase of this effort, ensemble modelling was evaluated finding that different groupings of income did not appreciably change model estimates while racial and ethnic categories did. As a result, the models are case for major races (White, African American, Asian, Other) and Hispanic. The ACS is collected by API at the block group level. Block group geographies are the lowest level of Census ACS detail and consolidate into Census tracts which in turn consolidate into counties. The FMLI responses are recorded in nominal dollars throughout the year, while total expenditure and ACS data represent year end states. As a result, the models' prediction for total expenditure is cast up using monthly inflation, weighted by monthly expenditure. Additional Caveats It is import to note, analytically, that the results are a stretch for credibility. CEX Consumer Units (people sharing financial decisions) are not exactly Census households (people in a housing unit) CEX demographics are not exactly Census demographics, with the CEX imputing incomes differenly than the Census medians. The CEX applies population weightings to the microdata while the Census primarily aggregates from respondents. The CEX observations are from 1 household (race is a 0/1 indicator) while Census demographics are many households (races are proportions) Models are trained upon repeated measures from a Consumer unit but not revised for ANOVA. Several of the CEX subcategories are very small, as spending has changed over the years. Reading, Alcohol and Tobacco use are still top level subcategories, for example as those have declined significantly since the CEX was first designed. So, this model is limited to the major subcategories of food, housing, transportation, health and retirement spending.* The model apply machine learning to large datasets so significance is not a consideration. However, in practice, those very small subcategories should be avoided. Difference in spending across racial categories also have different magnitudes and varieties. As a result, the R squared metrics on each submodel vary from 25%-45%. I find 40% R square results to be significant as they predict human behaviors rather than more deterministic relationships. The resulting dataset is available from the Open Environments at dataverse.harvard.edu. The supporting code is shared on github.com/OpenEnvironments under MIT open source licensing. The final data files are: YYYYblockgroupspending.csv where YYYY is the four digit year Get_FMLI.ipynb to collect the files Get_ACS.ipynb uses the Census API to download select ACS variables at the block group level modelvars.csv configures the requested variables and the Census definition of each. blockgroupspending_regression.ipynb trains the regression model. blockgroupspending_prediction.ipynb uses the two models to predict total expenditure and it's subcategories in each blockgroup. Other files are include for posterity, including functions to download and process other CEX PUMD data files. All questions or feedback is most welcome by email at support@openenvironments.com Citations U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Summary Tables. https://www.bls.gov/cex/tables/top-line-means.htm. 2023 U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Public Use Microdata files. https://www.bls.gov/cex/pumd.htm U.S. Census Bureau, “American Community Survey 5-Year Estimates: Comparison Profiles 5-Year,” 2022, http://api.census.gov/data/2022/acs/acs5 Python Package Index - PyPI. Python Software Foundation. "A simple wrapper for the United States Census Bureau’s API.". Retrieved from https://pypi.org/project/census/ The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. XGBoost. https://anaconda.org/conda-forge/xgboost The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. Tensorflow. https://anaconda.org/conda-forge/tensorflow Controls Pre-COVID estimates use means from 2020. https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-quintiles-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-deciles-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-region-1-year-average-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-population-area-size-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-area-type-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-composition-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-education-highest-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-housing-tenure-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-earners-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-size-2020.pdf
Subject	Social Sciences
Keyword	Census Block Groups
Related Publication	Cites: U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Summary Tables. https://www.bls.gov/cex/tables/top-line-means.htm. 2023https://www.bls.gov/cex/tables/top-line-means.htm
License/Data Use Agreement	CC0 1.0

	1 File
	2023blockgroupspending.tab Tabular Data - 32.3 MB Published Feb 7, 2025 10 Downloads 8 Variables, 222699 Observations UNF:6:knKH4yMTu4hNBraKDOs4hQ==	Preview "2023blockgroupspending.tab" Preview "2023blockgroupspending.tab" Access File File Access Public Download Options Comma Separated Values (Original File Format) Tab-Delimited RData Download Metadata DDI Codebook v2 Data File Citation Download EndNote XML Download RIS Download BibTeX Explore Options Data Explorer v2

Citation Metadata

Persistent Identifier	doi:10.7910/DVN/SNUUGO
Publication Date	2025-02-07
Title	2023 Consumer Spending by US Census Block Group
Author	Open Environmentshttps://orcid.org/0000-0002-7476-2820
Point of Contact	Use email button above to contact. Bryan, Michael (Open Environments)
Description	blockgroupspending Opportunity US Consumers express their behavior in a number of ways, but critically in their spending decisions. The US Bureau of Labor Statistics is charged with publishing spending activity and provides its Consumer Expenditure Survey (CEX) annually with US totals, with selected states (40) and cities (23). Limited to aggregates, the survey only needs 10s of thousands of observations in the original collection. While this is sufficient for macroeconomic use, the volume gives a weak basis for estimating lower levels of geography. In addition, the CEX includes demographic measurements that are similar, but not directly related, to Census variables. So, the CEX does not integtate well with the American Commuity Survey or other Census publications. This *blockgroupspending* publication by Open Environments attempts to address this problem by using the BLS' Public Microdata (PUMD) sample to allocate CEX spending categories across 220,000 US Census block group geographies. For each block group, the effort applies two models to estimate: total consumer spending (regression) distribution of spending across spending categories (penetration) including Food, Transportation, Housing and Health costs. Ultimately, these project spending on block groups that can be joined to US Census publications for additional demographics. Understanding the results requires awareness of the BLS' CEX data structures. This is available in the markdown file named oe_bls_cex_EDA.md The publication is made together with the source python code and notebooks used for repeatability. The materials are maintained under version control at https://github.com/OpenEnvironments/blockgroupspending. All feedback and development requests are welcome. Model details -- The CEX publication includes many files reflecting detailed 'diary' surveys capturing spend on thousands of items every two weeks family 'interviews' collecting household spending over the previous 3 months The models are trained upon the latter, 'FMLI' files. The regression model uses extreme gradient boosting, or XGBoost methods that apply many decision trees to iteratively correct prediction error. The subcategory models also use tree based methods, trained upon a the family interview details. The spending variables are named, following the BLS' CEX convention: \|Variable\|Definition\|2023\|pct\| \|---\|---\|---\|---\| \|TOTEXP\|Average annual expenditures\|77280\|\| \|FOOD\|Food\|9985\|0.129\| \|ALCBEV\|Alcoholic beverages\|637\|0.008\| \|HOUS\|Housing\|25436\|0.329\| \|APPAR\|Apparel and services\|2041\|0.026\| \|TRANS\|Transportation\|13174\|0.17\| \|HEALTH\|Healthcare\|6159\|0.08\| \|ENTERT\|Entertainment\|3635\|0.047\| \|PERSCA\|Personal care products and services\|950\|0.012\| \|READ\|Reading\|117\|0.002\| \|EDUCA\|Education\|1656\|0.021\| \|TOBACC\|Tobacco products and smoking supplies\|370\|0.005\| \|MISC\|Miscellaneous\|1184\|0.015\| \|CASHCO\|Cash contributions\|2378\|0.031\| \|RETPEN\|Personal insurance and pensions\|9556\|0.124\| During the exploratory phase of this effort, ensemble modelling was evaluated finding that different groupings of income did not appreciably change model estimates while racial and ethnic categories did. As a result, the models are case for major races (White, African American, Asian, Other) and Hispanic. The ACS is collected by API at the block group level. Block group geographies are the lowest level of Census ACS detail and consolidate into Census tracts which in turn consolidate into counties. The FMLI responses are recorded in nominal dollars throughout the year, while total expenditure and ACS data represent year end states. As a result, the models' prediction for total expenditure is cast up using monthly inflation, weighted by monthly expenditure. Additional Caveats It is import to note, analytically, that the results are a stretch for credibility. CEX Consumer Units (people sharing financial decisions) are not exactly Census households (people in a housing unit) CEX demographics are not exactly Census demographics, with the CEX imputing incomes differenly than the Census medians. The CEX applies population weightings to the microdata while the Census primarily aggregates from respondents. The CEX observations are from 1 household (race is a 0/1 indicator) while Census demographics are many households (races are proportions) Models are trained upon repeated measures from a Consumer unit but not revised for ANOVA. Several of the CEX subcategories are very small, as spending has changed over the years. Reading, Alcohol and Tobacco use are still top level subcategories, for example as those have declined significantly since the CEX was first designed. So, this model is limited to the major subcategories of food, housing, transportation, health and retirement spending.* The model apply machine learning to large datasets so significance is not a consideration. However, in practice, those very small subcategories should be avoided. Difference in spending across racial categories also have different magnitudes and varieties. As a result, the R squared metrics on each submodel vary from 25%-45%. I find 40% R square results to be significant as they predict human behaviors rather than more deterministic relationships. The resulting dataset is available from the Open Environments at dataverse.harvard.edu. The supporting code is shared on github.com/OpenEnvironments under MIT open source licensing. The final data files are: YYYYblockgroupspending.csv where YYYY is the four digit year Get_FMLI.ipynb to collect the files Get_ACS.ipynb uses the Census API to download select ACS variables at the block group level modelvars.csv configures the requested variables and the Census definition of each. blockgroupspending_regression.ipynb trains the regression model. blockgroupspending_prediction.ipynb uses the two models to predict total expenditure and it's subcategories in each blockgroup. Other files are include for posterity, including functions to download and process other CEX PUMD data files. All questions or feedback is most welcome by email at support@openenvironments.com Citations U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Summary Tables. https://www.bls.gov/cex/tables/top-line-means.htm. 2023 U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Public Use Microdata files. https://www.bls.gov/cex/pumd.htm U.S. Census Bureau, “American Community Survey 5-Year Estimates: Comparison Profiles 5-Year,” 2022, http://api.census.gov/data/2022/acs/acs5 Python Package Index - PyPI. Python Software Foundation. "A simple wrapper for the United States Census Bureau’s API.". Retrieved from https://pypi.org/project/census/ The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. XGBoost. https://anaconda.org/conda-forge/xgboost The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. Tensorflow. https://anaconda.org/conda-forge/tensorflow Controls Pre-COVID estimates use means from 2020. https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-quintiles-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-deciles-before-taxes-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-region-1-year-average-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-population-area-size-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-area-type-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-composition-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-education-highest-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-housing-tenure-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-earners-2020.pdf https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-size-2020.pdf
Subject	Social Sciences
Keyword	Census Block Groups https://www.census.gov/programs-surveys/geography/about/glossary.html#par_textimage_4 (Consumer Expenditure Survey) https://www.bls.gov/cex/
Related Publication	Cites: U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Summary Tables. https://www.bls.gov/cex/tables/top-line-means.htm. 2023 url https://www.bls.gov/cex/tables/top-line-means.htm Cites: U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Public Use Microdata files. https://www.bls.gov/cex/pumd.htm. 2023 url https://www.bls.gov/cex/pumd.htm Cites: U.S. Census Bureau, “American Community Survey 5-Year Estimates: Comparison Profiles 5-Year,” http://api.census.gov/data/2023/acs/acs5. 2023 url http://api.census.gov/data/2023/acs/acs5 Cites: Python Package Index - PyPI. Python Software Foundation. "A simple wrapper for the United States Census Bureau’s API.". Retrieved from https://pypi.org/project/census/ url https://pypi.org/project/census/ Cites: The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. XGBoost. https://anaconda.org/conda-forge/xgboost. 2023 url https://anaconda.org/conda-forge/xgboost
Depositor	Bryan, Michael
Deposit Date	2025-01-27

Dataset Terms

License/Data Use Agreement

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation shown on the dataset page.

Creative Commons CC0 1.0 Universal Public Domain Dedication. CC0 1.0

Dataset Version	Summary	Contributors	Published on
No records found.

Edit File

This file has already been deleted (or replaced) in the current version. It may not be edited.

Restrict Access

Restricting limits access to published files. People who want to use the restricted files can request access by default. If you disable request access, you must add information about access to the Terms of Access field.

Learn about restricting files and dataset access in the User Guide.

Request Access

Enable access request

You must enable request access or add terms of access to restrict file access.

Terms of Access for Restricted Files

Save Changes

Edit Embargo

The selected file or files have already been published. Contact an administrator to change the embargo date or reason of the file or files.

Edit Retention Period

The selected file or files have already been published. Contact an administrator to change the retention period date or reason of the file or files.

Delete Files

The file will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Select File(s)

Please select one or more files.

Share Dataset

Share this dataset on your favorite social media networks.

Continue

Dataset Citations

Citations for this dataset are retrieved from Crossref via DataCite using Make Data Count standards. For more information about dataset metrics, please refer to the User Guide.

Sorry, no citations were found.

Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired or the files can only be transferred via Globus.

You may request access to any restricted file(s) by clicking the Request Access button.

Ineligible Files Selected

The selected file(s) may not be transferred because you have not been granted access or the file(s) have a retention period that has expired or the files are not Globus accessible.

You may request access to any restricted file(s) by clicking the Request Access button.

Download Options

The files selected are too large to download as a ZIP.

You can select individual files that are below the 15.0 GB download limit from the files table, or use the Data Access API for programmatic access to the files.

Select File(s)

Please select a file or files to be downloaded.

Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired.

Click Continue to download the files you have access to download.

Ineligible Files Selected

Some file(s) cannot be transferred. (They are restricted, embargoed, with an expired retention period, or not Globus accessible.)

Click Continue to transfer the elligible files.

Delete Dataset

Are you sure you want to delete this dataset and all of its files? You cannot undelete this dataset.

Delete Draft Version

Are you sure you want to delete this draft version? Files will be reverted to the most recently published version. You cannot undelete this draft.

Unpublished Dataset Preview URL

Preview URL can only be used with unpublished versions of datasets.

Unpublished Dataset Preview URL

Are you sure you want to disable the Preview URL? If you have shared the Preview URL with others they will no longer be able to use it to access your unpublished dataset.

Delete Files

The file(s) will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Compute

This dataset contains restricted files you may not compute on because you have not been granted access.

Deaccession Dataset

Are you sure you want to deaccession? This is permanent and the selected version(s) will no longer be viewable by the public.

Deaccession Dataset

Are you sure you want to deaccession this dataset? This is permanent an it will no longer be viewable by the public.

Version Differences Details

Please select two versions to view the differences.

Version Differences Details

Version:
Last Updated:

Select File(s)

Please select a file or files for access request.

Select File(s)

Embargoed files cannot be accessed. Please select an unembargoed file or files for your access request.

Edit Tags

Select existing file tags or create new tags to describe your files. Each file can have more than one tag.

Request Access

You need to Sign Up or Log In to request access.

Dataset Terms

Please confirm and/or complete the information needed below in order to request access to files in this dataset.

This dataset is made available under the following terms. Please confirm and/or complete the information needed below in order to continue.

License/Data Use Agreement

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation shown on the dataset page.

Creative Commons CC0 1.0 Universal Public Domain Dedication. CC0 1.0

Preview Guestbook

Upon downloading files the guestbook asks for the following information.

Guestbook Name

Collected Data

Account Information

Package File Download

Use the Download URL in a Wget command or a download manager to download this package file. Download via web browser is not recommended. User Guide - Downloading a Dataverse Package via URL

Download URL

https://qa.dataverse.org/api/access/datafile/

Compute Batch

Clear Batch

Dataset	Persistent Identifier	Change Compute Batch

Compute Batch

Submit for Review

You will not be able to make changes to this dataset while it is in review.

Publish Dataset

Are you sure you want to republish this dataset?

By default datasets are published with the CC0-“Public Domain Dedication” waiver. Learn more about the CC0 waiver here.

To publish with custom Terms of Use, click the Cancel button and go to the Terms tab for this dataset.

Select if this is a minor or major version update.

Minor Release (1.2)

Major Release (2.0)

Publish Dataset

This dataset cannot be published until Block Group Datasets is published by its administrator.

Publish Dataset

This dataset cannot be published until Block Group Datasets and Harvard Dataverse are published.

Return to Author

Return this dataset to contributor for modification. The reason for return entered below will be sent by email to the author.

Curation Status History

Status	Date	Assigner
No records found.

Add/Edit a Version Note

Styled Citation

2023 Consumer Spending by US Census Block Group

blockgroupspending

Opportunity

Model details

Additional Caveats

Citations

Controls