cmaq-2019-modeling-platform
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
parent directory.. | ||||
<!DOCTYPE html> <html> <body> <h1>Downloading data from S3 Bucket</h1> This dataset is available as part of the AWS Open Data Program, therefore egress fees are not charged to either the host or the person downloading the data.<br> Note, this dataset contains the CMAQ 2019 Modeling Platform with inputs and scripts that are configured for CRACMM2 in CMAQv5.5 but these same inputs can be used for CRACMM version 1 or 2 and CMAQ v5.4 with minor modifications.using the CRACMMV1 emissions, epic files, and icbc's.<br> Link to the bucket described here: <br> <a href="https://cmaq-2019-modeling-platform.s3.amazonaws.com/index.html">cmaq-2019-modeling-platform</a><br> Link to dataverse entry for the data contained within this bucket: <a href="https://dataverse.unc.edu/dataset.xhtml?persistentId=doi:10.15139/S3/WZNGQB">CMAQ Model Version 5.4-5.5 CRACMM Input Data -- 1/1/2019 - 12/31/2019 12km CONUS Dataverse</a><br> The 2019 met inputs are available under the EQUATES S3 bucket:<br> <a href="https://cmas-equates.s3.amazonaws.com/index.html#CMAQ_12US1/INPUT/2019/">EQUATES 2019 MCIP Input</a> <br> <br> <br> Instructions for downloading the aws command line.<br> <br> <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">Instructions to Download AWS Command Line</a> <br> Once you have it installed, you can use it without credentials.<br> Example commands:<br> To query how much data is in a bucket: <br> <br> <pre> aws s3 --no-sign-request ls --summarize --region=us-east-1 --human-readable --recursive s3://cmaq-2019-modeling-platform </pre> Output: <br> <pre> 2025-01-22 17:17:42 813.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190522 2025-01-22 17:17:42 769.0 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190523 2025-01-22 17:17:42 760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190524 2025-01-22 17:17:42 760.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190525 2025-01-22 17:17:42 760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190526 2025-01-22 17:17:42 760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190527 2025-01-22 17:17:42 760.5 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190528 2025-01-22 17:17:42 749.1 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190529 2025-01-22 17:17:42 760.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190530 2025-01-22 17:17:42 760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190531 2024-11-15 12:44:16 30.9 KiB index.html 2024-11-15 13:05:34 3.8 KiB readme.html Total Objects: 7271 Total Size: 6.5 TiB </pre> To query how much meterology data is in the EQUATES bucket <pre> aws s3 --no-sign-request ls --summarize --region=us-east-1 --human-readable --recursive s3://cmas-equates/CMAQ_12US1/INPUT/2019 </pre> Output: <br> <pre> ... ... 2024-06-05 13:25:32 707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191227.contents.txt 2024-06-05 13:25:33 6.6 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191227.tar 2024-06-05 13:28:05 707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191228.contents.txt 2024-06-05 13:28:06 6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191228.tar 2024-06-05 13:31:25 707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191229.contents.txt 2024-06-05 13:31:25 6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191229.tar 2024-06-05 13:32:26 707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191230.contents.txt 2024-06-05 13:32:27 6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191230.tar 2024-06-05 13:36:11 707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191231.contents.txt 2024-06-05 13:36:11 6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191231.tar Total Objects: 905 Total Size: 2.7 TiB </pre> Command to copy files from the bucket to your local filesystem (the system where you are running the aws cli) and place it under the path that you specify for /your_local_path. To obtain the emissions, icbc, and epic files: <pre> aws s3 --no-sign-request --region=us-east-1 cp --recursive s3://cmaq-2019-modeling-platform/ /your_local_path/ </pre> To obtain the meterology files: <pre> aws s3 --no-sign-request --region=us-east-1 cp --recursive s3://cmas-equates/CMAQ_12US1/INPUT/2019 /your_local_path/ </pre> To obtain all input data and run scripts for a two day run use the s3 download script available here. Note, you need to edit the script to specify the local directory on your machine. To run a 2 day run starting on 20190522 - 20190523, you will need to download 123G of data using the following script: <pre> wget https://cmaq-2019-modeling-platform.s3.amazonaws.com/cmaq-2019-modeling-platform/CMAQv53_TS/2019_12US1/scripts/s3_copy_nosign_2019_cracmm_opendata_to_local_2019.csh ##Edit lines 16 and 17 to specify the path of your local directory for the input data ./s3_copy_nosign_2019_cracmm_opendata_to_local_2019.csh </pre> <br> <br> Please submit any questions or difficulties as a new issue in the CMAS Data Warehouse Category on the CMAS User Forum.<br> <a href="https://forum.cmascenter.org/c/cmas-data-warehouse/96">CMAS User Forum</a><br> </body> </html>