The GeoStats Python package - geostats

The geostats package is a python module that provides an interface to compute spatial statistics based on a shapefile for various datasets. The package uses a user-provided shapefile or GeoDataFrame to compute statistics for each polygon.

Statistics

Default statistics computed are 'min', 'max', 'median', 'mean', 'majority', 'sum','std','count'. Additional statistics can be computed by defining a function to compute it and passing the function to the package using the addstats keyword. E.g., the package includes the additional statistic gini defined by

```python def gini(self): ''' gini(geo)

Return computed Gini coefficient.
'''
if self.count()>1:
    xsort = sorted(self.data[self.mask==False].flatten()) # increasing order
    y = np.cumsum(xsort)
    B = sum(y) / (y[-1] * len(xsort))
    return 1 +  1./len(xsort) - 2*B
else:
    return 1

`` which is then passed usingaddstats={'gini':gini}` keyword.

Usage example

```python

Import packages

import geopandas import geostats import requests import io

Download countries shapefile from NaturalEarth

headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8'}

url = 'https://naturalearth.s3.amazonaws.com/10mcultural/ne10madmin0countries.zip' r = requests.get(url, headers=headers) countries = gp.readfile(io.BytesIO(r.content))

Compute statistics for all measures

Stats = geostats.geostats(countries, stats=['mean','max','min','std', 'sum']) Stats.geostats() print(Stats.df)

Compute statistics for only one or a few measures

mymeasures = ['CSI', 'Sea100'] Stats = geostats.geostats(countries, stats=['mean','max','min','std', 'sum'], measures=mymeasures) Stats.geostats() print(Stats.df)

```

Stats.df is a GeoDataFrame that contains the information from the original GeoDataFrame or shapefile and all the requested statistics for each measure. Naming conventions are such that statistic s for measure m appears as column ms. Below are the names for each measure in the database.

Datasets

The package computes spatial statistics using the following datasets:

| Variable Type | Source/Citation | License | geostats name | Location | |--------|--------|-------|--------------|---------| | Geography, Climate, and Agricultural Data | | | | | | Mobility| Özak, Ömer. “Distance to the pre-industrial technological frontier and economic development.” Journal of Economic Growth 23.2 (2018): 175-221. Özak, Ömer. “The voyage of homo-economicus: Some economic measures of distance.” Department of Economics, Brown University (2010). (Data) | This data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.| HMI, HMI10, HMISea, HMISea10| /geostats-data/HMI/tifs/| |GAEZ Crop Yield| IIASA/FAO, 2012. Global Agro‐ecological Zones (GAEZ v3.0). IIASA, Laxenburg, Austria and FAO, Rome, Italy. Fischer, G., Nachtergaele, F.O., Prieler, S. , Teixeira, E. , Toth, G., van Velthuizen, H., Verelst, L., & Wiberg, D. (2012). Global Agro-ecological Zones (GAEZ v3.0)- Model Documentation. IIASA, Laxenburg, Austria and FAO, Rome, Italy. , Laxenburg, Austria; Rome, Italy. | License states "Reproduction and dissemination of material contained in GAEZ v3.0 or educational, research, personal or other noncommercial purposes are authorized without any prior written permission from the copyright holders, provided FAO and IIASA are fully acknowledged." | Crop name followed by input level given by hi, med or lo| /geostats-data/CropsYield/tifs/| |Crop Caloric Suitability| Oded Galor and Ömer Özak, 2016. “The Agricultural Origins of Time Preference,” American Economic Review, 2016, 106(10): 3064–3103. Oded Galor and Ömer Özak, 2015. “Land Productivity and Economic Development: Caloric Suitability vs. Agricultural Suitability,” Brown University Working Paper. Özak, Ö. (2015). Caloric Suitability Index - Data (v1.0) [Data set]. Zenodo. |This data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.| CSI followed by crop name and input level given by hi, med or lo| /geostats-data/CSICrops/tifs/| |Caloric Suitability Index| Oded Galor and Ömer Özak, 2016. “The Agricultural Origins of Time Preference,” American Economic Review, 2016, 106(10): 3064–3103. Oded Galor and Ömer Özak, 2015. “Land Productivity and Economic Development: Caloric Suitability vs. Agricultural Suitability,” Brown University Working Paper. Özak, Ö. (2015). Caloric Suitability Index - Data (v1.0) [Data set]. Zenodo. |This data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.| PeriodMeasure where Period is pre1500: Pre-1500 crop-based data; pre15002: Pre-1500 crop-based data excluding Asian crops in Africa; post1500: Post-1500 crop-based data; dif: Difference between post-1500 and pre-1500; dif2: Difference between post-1500 and pre-15002; Measure is Average, Maximum, Total, or other statistic of caloric suitability across crops | /geostats-data/CSI/tifs/| |Caloric Plow Suitability| Galor, Oded, Ömer Özak and Assaf Sarid, “Geographical Origins and Economic Consequences of Language Structures” Brown University Working Paper, 2016. Oded Galor and Ömer Özak, 2016. “The Agricultural Origins of Time Preference,” American Economic Review, 2016, 106(10): 3064–3103. Oded Galor and Ömer Özak, 2015. “Land Productivity and Economic Development: Caloric Suitability vs. Agricultural Suitability,” Brown University Working Paper. Özak, Ö. (2015). Caloric Suitability Index - Data (v1.0) [Data set]. Zenodo. |This data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.| CropTypePeriodMeasure where CropType is plowpos (plow positive), plowneg (plow negative), plowpot (plow potential: difference between positive and negative); Period is pre1500: Pre-1500 crop-based data; pre15002: Pre-1500 crop-based data excluding Asian crops in Africa; post1500: Post-1500 crop-based data; dif: Difference between post-1500 and pre-1500; dif2: Difference between post-1500 and pre-15002; Measure is en, enmean, entot, or other statistic of caloric suitability across crops | /geostats-data/CSIPlow/tifs/| |Caloric Suitability and Growth Cycle of best crop| Oded Galor and Ömer Özak, 2016. “The Agricultural Origins of Time Preference,” American Economic Review, 2016, 106(10): 3064–3103. Oded Galor and Ömer Özak, 2015. “Land Productivity and Economic Development: Caloric Suitability vs. Agricultural Suitability,” Brown University Working Paper. Özak, Ö. (2015). Caloric Suitability Index - Data (v1.0) [Data set]. Zenodo. |This data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.| PeriodMethodMeasure where Period is pre1500: Pre-1500 crop-based data; pre15002: Pre-1500 crop-based data excluding Asian crops in Africa; post1500: Post-1500 crop-based data; dif: Difference between post-1500 and pre-1500; dif2: Difference between post-1500 and pre-15002; Methods is how the best crop was chosen: en (calories), inv (return), cyc (cycle), hvst (harvests), winter (unproductive period), etc. followed by statistic.| /geostats-data/CSICycle/tifs/| |Elevation|NOAA National Centers for Environmental Information. 2022: ETOPO 2022 15 Arc-Second Global Relief Model. NOAA National Centers for Environmental Information. DOI: 10.25921/fd45-gt74. Accessed January 16, 2025. Documentation| Creative Commons Zero 1.0 Universal Public Domain Dedication (CC0-1.0). according to their website| etopo, etopocyl| /geostats-data/ETOPO/| |Elevation|GLOBE Task Team and others (Hastings, David A., Paula K. Dunbar, Gerald M. Elphingstone, Mark Bootz, Hiroshi Murakami, Hiroshi Maruyama, Hiroshi Masaharu, Peter Holland, John Payne, Nevin A. Bryant, Thomas L. Logan, J.-P. Muller, Gunter Schreier, and John S. MacDonald), eds., 1999. Global land one-kilometer base elevation (GLOBE) Digital Elevation Model, Version 1.0. National Oceanic and Atmospheric Administration, National Geophysical Data Center, 325 Broadway, Boulder, Colorado 80303, U.S.A. Digital data base on the World Wide Web (URL: http://www.ngdc.noaa.gov/mgg/topo/globe.html) and CDROMs. | Can be freely downloaded from the GLOBE website and redistributed according to documentation|globe, globecyl|/geostats-data/GLOBE/| |Ruggedness| Based on GLOBE data following methodology in Riley, S. J., DeGloria, S. D., & Elliot, R. (1999). Index that quantifies topographic heterogeneity. intermountain Journal of sciences, 5(1-4), 23-27.| Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License | rix, rixcyl| /geostats-data/RIX/| |Uban area, crop area, gras area| Klein Goldewijk, K., Beusen A., van Drecht, G., de Vos, M., 2011. The HYDE 3.1 spatially explicit database of human induced land use change over the past 12,000 years, Global Ecology and Biogeography 20(1): 73-86. Klein Goldewijk, K., Beusen, A., Janssen, P., 2010. Long term dynamic modeling of global population and built-up area in a spatially explicit way, HYDE 3 .1. The Holocene, 20(4), 565-573. | Freely available on the web here. License is Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License|uopp_XXXXX, grasXXXX, cropXXXX, where XXXX=10000BC, 9000BC, ..., 1000AD, ..., 2005AD|/geostats-data/HYDE/tifs/| | Climate (1901-2012) | Harris, I. P. D. J., Jones, P., Osborn, T., & Lister, D. (2014). Updated high-resolution grids of monthly climatic observations-the CRU TS3. 10 Dataset. International journal of climatology, 34, 623-642. | Contains public sector information licensed under the Open Government Licence v3.0. (License)| Rasters have been aggregated to show the mean, volatility, and spatial correlation (mean and min) for each cell across all years. The names of the different climatic variables are pre - Precipitation (mm/month), tmp - Mean Temperature (°C), tmn - Minimum Temperature (°C), tmx - Maximum Temperature (°C), dtr - Diurnal Temperature Range (°C), reh - Relative Humidity (%), pet - Potential Evapotranspiration (mm/month), cld - Cloud Cover (%), frs - Frequency of Frost Days (days/month), vap - Vapor Pressure (hPa), wet - Wet Days (days/month). So, e.g., for tmp the measure names are tmpmean, tmpspatcorr, tmpvolatility followed by the statistic name and cyl if it is uses the cylindrival equal area projection. |/geostats-data/CRU/tif/| |Malaria Ecology|Kiszewski, A., Mellinger, A., Spielman, A., Malaney, P., Sachs, S. E., & Sachs, J. (2004). A global index representing the stability of malaria transmission. American Journal of tropical medicine and hygiene, 70(5), 486-498. https://doi.org/10.4269/ajtmh.2004.70.486| Freely available on the web here. | stxv5, stxv5cyl|/geostats-data/Malaria/malaria_ecology/| | Tse-Tse (Africa, presence probability)| Wint, W. & Rogers, D. (2000) Predicted Distributions of Tsetse in Africa. Consultancy Report for the Animal Health Service of the Animal Production and Health Division of the Food and Agriculture Organization of the United Nations. FAO, Rome. [Accessed on Jan. 14, 2015] (Data)| FAO shares data under various versions of the Creative Commons Attribution-NonCommercial-ShareAlike. See, e.g., OPEN DATA LICENSING FOR STATISTICAL DATABASES or this related data. | Fuscacyl, Morsitanscyl, Palpaliscyl, tsetsecyl | /geostats-data/tsetse/ | |Ecological Diversity (based on 827 ecological zones around year 2000)| Olson, D. M., Dinerstein, E., Wikramanayake, E. D., Burgess, N. D., Powell, G. V., Underwood, E. C., D’amico, J. A., Itoua, I., Strand, H. E., Morrison, J. C. et al. (2001). Terrestrial ecoregions of the world: A new map of life on earth a new global map of terrestrial ecoregions provides an innovative tool for conserving biodiversity, BioScience 51(11): 933–938. (Old website) (Data)| PENDING CLARIFICATION: If it cannot be shared, then either user will have to download this file and put in right place, or the script will have to download and unzip this file| ecodiversitywwf, ecopolarizationwwf| /geostats-data/Ecological_Zones/| |Ecological Diversity (based on 16 biomes around year 2000)| Aggregation of ecological zones in Olson et al. (2001) | PENDING CLARIFICATION: If it cannot be shared, then either user will have to download this file and put in right place, or the script will have to download and unzip this file and then aggregate biomes. | ecodiversity, ecopolarization| /geostats-data/Ecodiversity/| |Ecological Diversity (during Last Glacial Maximum LGM)|Ray, Nicolas, Adams, Jonathan. A GIS-based vegetation map of the world at the Last Glacial Maximum (25,000-15,000 BP). In: Internet archaeology, 2001, vol. 11. (Accessed on Jun. 5, 2017) (Paper) (Data)| NOOA Website says "Use Constraints: Please cite original publication, online resource, dataset and publication DOIs (where available), and date accessed when using downloaded data. If there is no publication information, please cite investigator, title, online resource, and date accessed." Internet Archaeology is Open-Access and it states that "Except where otherwise noted, content from this work may be used under the terms of the Creative Commons Attribution 3.0 (CC BY) Unported licence, which permits unrestricted use, distribution, and reproduction in any medium, provided that attribution to the author(s), the title of the work, the Internet Archaeology journal and the relevant URL/DOI are given."| ecodiversityLGM, ecopolarizationLGM| /geostats-data/EcoDiversityLGM/ |Coast Length|Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com.|Creative Commons Public Domain License.| coastlen |/geostats-data/Waters-Coasts/| |Land within 100kms of Sea|Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com.|Creative Commons Public Domain License.| sea100, sea100pct |/geostats-data/Waters-Coasts/| | Agricultural Suitability| Ramankutty, N., J.A. Foley , J. Norman, and K. McSweeney. The global distribution of cultivable lands: current patterns and sensitivity to possible climate change. Submitted to Global Ecology and Biogeography, March 2001 | Originally data freely available at http://atlas.sage.wisc.edu/. Updated website (with limited data) https://sage.nelson.wisc.edu/data-and-models/atlas-of-the-biosphere/mapping-the-biosphere/land-use/suitability-for-agriculture/|climfac, climsuit, soilfac, suit, climfaccyl, climsuitcyl, soilfaccyl, suitcyl | /geostats-data/Ramankutty/ | | Lights, Population, and Other | | | | | |Lights| F. C. Hsu, K. Baugh, T. Ghosh, M. Zhizhin, and C. Elvidge, “DMSP-OLS Radiance Calibrated Nighttime Lights Time Series with Intercalibration,” Remote Sensing, vol. 7, pp. 1855–1876, 2015. Image and Data processing by NOAA's National Geophysical Data Center. DMSP data collected by the US Air Force Weather Agency. | Original data can be downloaded from this website. Data now housed at the Earth Observation Group. License is creative commons 4, and can be redistributed. | FXXYEAR, FXXYEARcyl where XX=10,...,18 and YEAR=1992,...,2012 | /geostats-data/Lights/tifs/| |Lights| Li, X., Zhou, Y., Zhao, M., & Zhao, X. (2020). A harmonized global nighttime light dataset 1992–2018. Scientific data, 7(1), 168. | Original data can be downloaded from this website. License is creative commons 4, and can be redistributed. | HLDYEAR where YEAR=1992,...,2020 | /geostats-data/HLD/tifs/| |Population|Center For International Earth Science Information Network-CIESIN-Columbia University, United Nations Food And Agriculture Programme-FAO, & Centro Internacional De Agricultura Tropical-CIAT. (2005). Gridded Population of the World, Version 3 (GPWv3): Population Count Grid (Version 3.00) [Data set]. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4639MPP (Accessed Mar. 25, 2015)| Users are prohibited from any commercial, non-free resale, or redistribution. | glupXXag, glupXXg, glpXXag, glpXXg, where XX=90, 95, 00 is the year | /geostats-data/GPW/v3/population/| |Population|Center For International Earth Science Information Network-CIESIN-Columbia University. (2018). Gridded Population of the World, Version 4 (GPWv4): Population Count, Revision 11 (Version 4.11) [Data set]. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4JW8BX5 (Accessed Jan. 17, 2025)| This work is licensed under the Creative Commons Attribution 4.0 International License. Users are free to use, copy, distribute, transmit, and adapt the work for commercial and non-commercial purposes, without restriction, as long as clear attribution of the source is provided. | gpwv4popcXXXX where XXXX=2000, 2005, 2010, 2015, 2020 is the year | /geostats-data/GPW/v4/population/| |Population|Center For International Earth Science Information Network-CIESIN-Columbia University. (2018). Gridded Population of the World, Version 4 (GPWv4): Population Count Adjusted to Match 2015 Revision of UN WPP Country Totals, Revision 11 (Version 4.11) [Data set]. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4PN93PB (Accessed Jan. 17, 2025)|This work is licensed under the Creative Commons Attribution 4.0 International License. Users are free to use, copy, distribute, transmit, and adapt the work for commercial and non-commercial purposes, without restriction, as long as clear attribution of the source is provided. | gpwv4popcadjXXXX where XXXX=2000, 2005, 2010, 2015, 2020 is the year | /geostats-data/GPW/v4/population/| |Population Density|Center For International Earth Science Information Network-CIESIN-Columbia University, & Centro Internacional De Agricultura Tropical-CIAT. (2005). Gridded Population of the World, Version 3 (GPWv3): Population Density Grid (Version 3.00) [Data set]. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4XK8CG2 (Accessed Mar. 25, 2015)| Users are prohibited from any commercial, non-free resale, or redistribution. | gludsXXag, gludsXXg, gldsXXag, gldsXXg, where XX=90, 95, 00 is the year | /geostats-data/GPW/v3/popdens/| |Population Density|Center For International Earth Science Information Network-CIESIN-Columbia University. (2017). Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11 (Version 4.11) [Data set]. Palisades, NY: Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H49C6VHW (Accessed Jan. 17, 2025)|This work is licensed under the Creative Commons Attribution 4.0 International License. Users are free to use, copy, distribute, transmit, and adapt the work for commercial and non-commercial purposes, without restriction, as long as clear attribution of the source is provided. | gpwv4popdXXXX where XXXX=2000, 2005, 2010, 2015, 2020 is the year | /geostats-data/GPW/v4/popdens/| |Population Density|Center For International Earth Science Information Network-CIESIN-Columbia University. (2018). Gridded Population of the World, Version 4 (GPWv4): Population Density Adjusted to Match 2015 Revision UN WPP Country Totals, Revision 11 (Version 4.11) [Data set]. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4F47M65 (Accessed Jan. 17, 2025)|This work is licensed under the Creative Commons Attribution 4.0 International License. Users are free to use, copy, distribute, transmit, and adapt the work for commercial and non-commercial purposes, without restriction, as long as clear attribution of the source is provided. | gpwv4popdadjXXXX where XXXX=2000, 2005, 2010, 2015, 2020 is the year | /geostats-data/GPW/v4/popdens/| |Population (counts, rural/ruban, density)| Klein Goldewijk, K., Beusen A., van Drecht, G., de Vos, M., 2011. The HYDE 3.1 spatially explicit database of human induced land use change over the past 12,000 years, Global Ecology and Biogeography 20(1): 73-86. Klein Goldewijk, K., Beusen, A., Janssen, P., 2010. Long term dynamic modeling of global population and built-up area in a spatially explicit way, HYDE 3 .1. The Holocene, 20(4), 565-573. | Freely available on the web here. License is Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License|popc_XXXXX, urbc_XXXX, rurc_XXXX, popd_XXXX, where XXXX=10000BC, 9000BC, ..., 1000AD, ..., 2005AD|/geostats-data/HYDE/tifs/| |Population Density (Africa) | UNESCO (1987) through UNEP/GRID-Sioux Falls (Accessed Jan. 22, 2015)| Most of the UNEP/GRID data sets are freely available for download by users via the Internet. UNEP/GRID does not place any restrictions on the use of this data, but does request that users cite UNEP/GRID. See license. | afpopdXX, where XX=60, 70, 80, 90, 00 is the year. | /geostats-data/UNEP/| |Population (counts, 2000-2023)| Bright, E. and P. R. Coleman, 2001. Landscan 2000-2023, Oak Ridge, TN: Oak Ridge National Laboratory. (Accessed Jan. 17, 2025) (Data)| Creative Commons Attribution 4.0 International License.| lsXXXX where XXXX=2000,...,20023 |/geostats-data/Landscan/|

FAO has updated its license policies for all its data, which now is CC-BY-4.0.

Install

Given the requirements of the package, the best way to install is to first create a mamba/conda environment as follows (this creates an python-3.11 environment and adds some basic packages that are needed).

bash mamba create --name GeoStats --override-channels -c conda-forge python=3.11 pip geopandas georasters jupyterlab jupyter seaborn geoplot pysal p7zip

activate the environment

bash mamba activate GeoStats

then

bash pip install geostats or

bash pip install git+git://github.com/ozak/geostats.git

Example Usage: GeoStats

``` python # Import packages import geopandas as gpd import geostats

# Read polygon data for which you want to compute statistics mygdf = gpd.read_file('myshapefile.shp')

# Setup geostats object mystats = = geostats.geostats(mygdf)

# Compute mystats.geostats()

# See results mystats.df

# Export data out = mystats.df.copy() out = out.drop(columns=['geometry']) out.to_csv(pathout + 'My-Stats.csv', encoding='utf-8', index=False) ```

Citation

If you use the package please cite:

Özak, Ömer. 2014. "The GeoStats Python package - `geostats`".

Also, make sure to cite all data sources above. This BibTeX file should contain all the (required) citations.

Disclaimer

GeoStats is provided for informational and research purposes only. The software and its outputs are distributed "as is" without any warranties, express or implied, including but not limited to the accuracy, completeness, reliability, or suitability for any particular purpose.

We do not guarantee that the computations, analyses, or geospatial statistics produced by GeoStats are free from errors or fit for any specific application. Users assume full responsibility for the interpretation and application of results obtained from this package.

We disclaim any liability for any loss, damage, or consequences arising from the use of GeoStats, including reliance on its results for decision-making. By using GeoStats, you acknowledge and accept these terms.

Issues

Find a bug? Report it via GitHub issues by providing

• a link to download the smallest possible raster and vector dataset necessary to reproduce the error
• python code or command to reproduce the error
• information on your environment: versions of python, gdal and numpy and system memory

Want a Data Source Added?

Any free and redistributable source can be added to the package, and we are constantly looking for additions. If you want some data source to be included, suggest it via GitHub issues by providing

• a link to the source
• a link to the license (ensuring it is free and redistributable)
• (if possible) create a pull request where you add the source as a new measure. This requires adding it in pathmeasures, main_measures, and possibly in main_measures in the main.py file.

Copyright

© Ömer Özak (2014)

This code and data is provided under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License and GPLv3.