# ---------------------------------------------------------------------- # Code and Data availability README # ---------------------------------------------------------------------- # Sebastian Sippel # 05.11.2019 This README describes the code and data to reproduce the key results of "Climate change now detectable from any single day of weather at global scale", published as Sippel et al. in Nat. Clim. Change. All code/data files are made available in this repository. Either browse/download files individually or use a service such as wget to download all files at the same time (e.g., "wget -r -np -nH -R index.html* https://data.iac.ethz.ch/Sippel_et_al_2019_DailyDetection/"). Code availability: We provide all code to reproduce the main results and all figures and Extended Data figures of the main paper. Data availability: We provide all files/data needed to reproduce the study's results as well as all intermediate data. This includes the trained fingerprints from CMIP5 (described in detail in Methods), and in addition we provide the daily test statistic time series for all models, reanalysis, and observations (that is, the models, reanalysis and observations projected onto the fingerprints). Due to storage and copyright constraints, original/raw CMIP5 files, reanalyses and observational datasets have to be downloaded from their original sources (given in Acknowledgements). This README file contains: 1. Directory Structure 2. Reproduction of Figures 3. Statistical Learning Code for Fingerprint extraction from CMIP5 archive 4. Processing data from original sources 1. Directory Structure # ---------------------------------------------------------------------- /code: Contains all code/functions to run scripts and reproduce figures. /data: Contains all intermediate data to reproduce the study's results. This includes _metadata (e.g. land/water files, observational masks, etc.), a template for processing CMIP5 data before fingerprint extraction (cmip5_processed_01/cmip5_DoM_anom005_mvar_tmp.RData), the trained fingerprints (cmip5_trained_fingerprints_LMO/) and the time series of the daily test statistic derived by projecting models, reanalyses, and observations onto the fingerprints (/OBS_REAN_data4figures). /scripts: Contains scripts used for fingerprint extractions (01_cmip5), and scripts to extract emergence statistic from observations (02_obs). /figures: Contains all scripts to reproduce all figures. 2. Reproduction of Figures: # ---------------------------------------------------------------------- To reproduce figures go to /figures. Each folder (e.g., /_fig02) contains a .R file that is executable and produces the respective figure, using code and data from the /code, /scripts, and /data directories. It's also possible to explore our results further using (1) the test statistic time series from models, reanalyses or observations, and/or (2) the extracted fingerprints from CMIP5. Both are read into memory for instance in /_fig03/_fig03_SIMPLEFIG.R. Additionally all raw figures are already contained in each figure directory as .pdf or .png files. 3. Statistical Learning Code for Fingerprint extraction from CMIP5 archive # ---------------------------------------------------------------------- The fingerprints extracted from the CMIP5 archive are available in /data/cmip5_trained_fingerprints_LMO/CMIP5.tas_ann_anom_mvar_HadISDH_masks_LMO.RData (for AGMT target metric) and /data/cmip5_trained_fingerprints_LMO/CMIP5.TOA_r10_mvar_HadISDH_masks_LMO_HIST.RData (for TOA imbalance target metric). Both files contain the extracted regression coefficient maps (i.e., fingerprints), as well as the test data obtained through leaving each model out from training and cross-validation sequentially (see Methods). The files also contain comprehensive error statics for each training scenario. All R-Code for the extraction of fingerprints from the CMIP5 archive, using the ridge regression statistical learning method (from the glmnet package, Friedman et al. (2010)) is available in: - /scripts/01_cmip5/_01_trainCMIP5_tas_anom_diffCV.R (AGMT target metric) - /scripts/01_cmip5/_01_trainCMIP5_TOA_r10_diffCV.R (TOA imbalance target metric) Both scripts use "leave-one-model-out cross-validation" (described in Methods) and the main function to extract fingerprint is made available in /code/_01_train_predict_glmnet_diffCV.R. 4. Processing data from original sources # ---------------------------------------------------------------------- To process all data from their original source requires to download all original daily data from CMIP5, Reanalyses and Observations. To facilitate processing from the original sources, we provide the detailed steps: - All original files used from the CMIP5 archive are specified in /data/_metadata/cmip5_files.txt - Regrid all original data onto a 5x5° regular grid (specified in /data/_metadata/global_5d00.txt) - Process all model data, reanalyses, and observations into a nxp matrix with n samples (i.e., along the time axis) and p spatial locations (72x36 grid cells). To facilitate this processing, a sample file from CMIP5 is provided in /data/cmip5_processed_01/cmip5_DoM_anom005_mvar_tmp.RData, and the functions used to read-in CMIP5 data are given in code/_00_READ_CMIP5_functions.R. The CMIP5 files processed this way can be directly used to run the fingerprint extraction scripts (detailed above). Then, the reanalysis and observational data processed in this way can be directly projected onto these fingerprints (for each month separately, as specified in Methods). References: # ---------------------------------------------------------------------- Jerome Friedman, Trevor Hastie, Robert Tibshirani (2010). Regularization Paths for Generalized Linear Models via Coordinate Descent. Journal of Statistical Software, 33(1), 1-22. URL http://www.jstatsoft.org/v33/i01/.