EPA/600/R-12/716 | September 2012 | www.epa.gov/research
vvEPA
Drifted States
Environmental Protection
Agency
HYDROLOGICAL SIMULATION PROGRAM - FORTRAN (HSPF) DATA
FORMATTING TOOL (HDFT)
"_ Hydrological Simulation Program • Fortran (HSPF): base
Ecosystems Research Division, Athens GA 30605
National Exposure Research Laboratory
Office of Research and Development
-------
HYDROLOGICAL SIMULATION PROGRAM-FORTRAN (HSPF) DATA
FORMATTING TOOL (HDFT)
By
Mohamoud, Y.M., S. Motamarri, R. Mahajan, M. Panunto, R. Parmar
M. Galvin, D. Stehr, and
Ecosystems Research Division
National Exposure Research Laboratory
Office of Research and Development
United States Environmental Protection Agency
Athens Georgia 30605
U.S. Environmental Protection Agency
Office of Research and Development
Washington, DC 20460
-------
Disclaimer
This report has been reviewed by the National Exposure Research Laboratory (NERL) -
Ecosystem Research Division (ERD), U.S. Environmental Protection Agency (USEPA) in
Athens Georgia and approved for publication. Approval does not signify that the contents
necessarily reflect the views and policies of the U.S. Environmental Protection Agency, nor does
mention of trade names or commercial products constitute endorsement or recommendation for
use. Although a reasonable effort has been made to assure that the results obtained are correct,
the HSPF formatting and conversion tools described in this report may require further testing and
evaluation by developers and future HDFT users. Therefore, the author and the U.S.
Environmental Protection Agency are not responsible and assume no liability whatsoever for any
results or any use made of the results obtained from these programs, nor for any damages or
litigation that result from the use of these programs for any purpose.
Citation:
Mohamoud, Y.M., S. Motamarri, R. Mahajan, M. Panunto, R. Parmar, M. Galvin, and D. Stehr
(2012). HSPF Data Formatting Tool (HDFT). U.S. Environmental Protection Agency,
Washington, DC, EPA/600/R-12/716.
11
-------
Table of Contents
Disclaimer ii
Table of Contents iii
List of Figures v
List of Tables vii
Foreword viii
Acknowledgements ix
Chapter 1: Introduction 1
HDFT (Web-Version) 3
Chapter 2: Data-Pasting Option of the HDFT Web-Version 4
2.1. Data requirements 5
2.2. Formatting options for different datatypes 5
2.3. Column Format (user-specified formatting option) 7
2.3.1. Formatting Column Format Data 7
2.3.2. Available Unit Conversion Options 9
2.3.3. Exporting Formatted and Converted Data 12
2.4. USGS NWIS Format 12
2.4.1. Formatting NWIS Data 13
2.4.2. Available Unit Conversion Options 14
2.4.3. Exporting Formatted and Converted Data 14
2.5. USGS Sediment format 14
2.5.1. Formatting USGS Sediment Data 14
2.5.2. Available Unit Conversion Options 14
2.5.3. Exporting Formatted and Converted Data 14
2.6. Other Formats 15
2.6.1. NOAAASOS(1 Minute Data) 15
2.6.2. Instantaneous time-series data (USGS) - Flow 17
2.6.3. Hourly data NCDC - U.S. Climate Reference Network (USCRN) -
Temperature 18
2.6.4. NCDC- DS3505 - Surface Data, Hourly Global - Date and Time in
Separate Columns - Solar Radiation 20
Chapter 3: HDFT Data Downloading Options (Web-version) 22
3.1. Details on example data used for explaining the downloading option: 22
3.2. Steps to follow when downloading the data are: 23
3.3. Downloading data from USGS NWIS Daily data & USGS Instantaneous Irregular
Data 23
3.4. Downloading Data from EPA-STORET 27
HDFT (Desktop Version 1.0) 31
Chapter 4: HDFT (Desktop Version) 32
4.1. Introduction 32
4.2. Data File Requirements 33
4.3. Importing Data 34
4.4. Conversion of Variable values 38
Chapter 5: Exporting HDFT-Formatted Data to WDMutil for HSPF Model Use 41
5.1. Importing daily time series data to WDM files using a new daily data formatting
script 43
5.2. Importing hourly time series data to a WDM file using new hourly data scripts 47
iii
-------
5.3. Importing sub-hourly time series data to a WDM file, using a new sub-hourly data
script 50
Appendix-A: Hydrological and Meteorological Variables and Unit Conversions 56
Appendix-B: Examples of Data Formats and Data Types 59
Appendix-C: Downloading Data from the National Climate Data Center 63
Appendix-D: STORET Data Download Options 71
Appendix-E: Obtaining USGS Station Numbers 81
IV
-------
List of Figures
Figure 1: Main Process 2
Figure 2: Main HDFT Window 7
Figure 3: Enter Data Information 8
Figure 4: Import Data Window 9
Figure 5: HDFT Main Window with Imported Data 10
Figure 6: Conversion of flow data from cfs to cms 11
Figure 7: Conversion of imported flow data to cumulative volume 11
Figure 8: Export Data Window 12
Figure 9: Data Information window - USGS NWIS Format 13
Figure 10: Data Information window- Other Formats- ASOS 16
Figure 11: Formatted ASOS 1 Minute data 17
Figure 12: Formatted NCDC hourly climate data 20
Figure 13: Data Download Window 23
Figure 14: Selection of variable window 24
Figure 15: Data Download Window with Station information extracted 25
Figure 16: Data Download Window-USGS NWIS - Entering Start and End dates 26
Figure 17: Data imported using download option- USGS NWIS Daily/ Instantaneous data 26
Figure 18: Data download window-EPA STORE! 28
Figure 19: EPA STORET- Parameters available 28
Figure 20: EPA STORET- Site information extracted 29
Figure 21: Data Download Window-EPA STORET - Entering Start and End dates 30
Figure 22: Data imported from EPA STORET 30
Figure 23: Main Desktop Version window 34
Figure 24: Import window showing the headers of the imported data file 35
Figure 25: Header window before headers were removed 35
Figure 26: Header window after headers were removed 36
Figure 27: Data ready for column identification 36
Figure 28: Column identification, variable and unit selection window 37
Figure 29: Confirmation for the date and time format 37
Figure 30: Columns selected for conversion and export 38
Figure 31: Unit conversion window 39
Figure 32: Converted flow values 39
Figure 33: Formatted data window 40
Figure 34: Name and Save the formatted data file 40
Figure 35: WDMUtil Window 42
Figure 36: An empty WDM file 43
Figure 37: HDFT exported daily flow data 44
Figure 38: Script for HDFT exported daily data 44
Figure 39: Write daily flow data to WDM file Script for HDFT exported daily data 45
Figure 40: WDM file with a single daily flow time series entry 46
Figure 41: Examples of two different hourly formats 47
Figure 42: HDFT exported hourly precipitation data 48
Figure 43: Hourly script for HDFT formatted data 48
Figure 44: Hourly precipitation data added to the WDM file 49
Figure 45: HDFT exported sub-hourly (5 minute interval) flow data 50
-------
Figure 46: Sub-hourly script for HDFT formatted data 51
Figure 47: Change interval tab of New Time Series window 52
Figure 48: Change starting date for New Time Series 53
Figure 49: A WDM file containting three data series with daily, hourly, and sub-hourly data
resolutions 54
Figure 50: Sample data to be pasted inside the tool (excluding headers) 59
Figure 51: Column format-Sample data 59
Figure 52: USGS NWIS- Sample Data with a header 59
Figure 53: USGS Sediment-Sample data with a header 60
Figure 54: ASOS -Sample data with no header 60
Figure 55: USGS instantaneous time series data - flow with a header 61
Figure 56: NCDC - U.S. Climate Reference Network data (hourly) with a header 61
Figure 57: Sample streamflow data (with no date and time separators) - USGS Instantaneous
Data Archive format 62
Figure 58: NCDC - Surface data, Hourly Global data, Date and Time in separate columns 62
Figure 59: National Climatic Data Center home page 63
Figure 60: Interactive map application page 64
Figure 61: Map Help window 64
Figure 62: GIS Map window 65
Figure 63: Layers sub window 66
Figure 64: Feature selection 67
Figure 65: Results window 67
Figure 66: Data Access Option Window 68
Figure 67: Parameter Selection window 69
Figure 68: Selection of dates and data download 70
Figure 69: HDFT Data Download Window 71
Figure 70: MyWATERS Mapper Welcome Page 72
Figure 71: Introductory sheet 72
Figure 72: HUC 8 watershed map 73
Figure 73: Information about selected HUC 8 watershed (Cuyahoga- HUC # 0411002) 73
Figure 74: Program is loading the STORET Results 74
Figure 75: List of STORET Sites showing information on available parameters and samples. .. 75
Figure 76: Results for Station Number CUVA_NPS_RORO 76
Figure 77: List of parameters, number of results, and date ranges 76
Figure 78: Data download window 77
Figure 79: List of available parameters 77
Figure 80: Site and parameter information extracted 78
Figure 81: Data downloaded from EPA STORET 79
Figure 82: Export Data Window 80
Figure 83: Exported Sample Data 80
Figure 84: HDFT Data download window 81
Figure 85: USGS NWIS home page 82
Figure 86: USGS NWIS Mapper home page 83
Figure 87: USGS Station Number 83
Figure 88: List of Sites on Map 84
VI
-------
List of Tables
Table 1: Criteria for data-formatting option 6
Table 2: Variable unit conversion options and conversion values used in HDFT 56
vn
-------
Foreword
Water availability and water quality models are widely used for environmental regulation such as
developing Total Maximum Daily Loads (TMDLs) for water quality-impaired water bodies,
environmental policy (e.g., water quality criteria development), and sustainable development and
management of land and water resources. To meet EPA's goals, the National Exposure Research
Laboratory (NERL) develops and uses fate and transport models, modeling tools and approaches
to simulate water availability and water quality constituents. The Hydrological Simulation
Program - FORTRAN (HSPF) is a comprehensive watershed model capable of simulating water
availability and water quality constituents at user-specified spatial and temporal scales. HSPF is
a mixed land-use model applicable to both urban and non-urban watersheds and was developed
by EPA in collaboration with the United States Geological Survey (USGS). HSPF is the core
watershed model in the BASINS (Better Assessment Science Integrating Point and Nonpoint
Source Pollution) modeling system. While BASINS has a number of databases available to
HSPF users, oftentimes model users need to create HSPF simulations with data from sources
other than BASINS. Because HSPF requires extensive input data, the HSPF Data-Formatting
Tool (HDFT) allows users to format model input data and import it into a WDM file. This tool is
also for users who are building their data from scratch from study areas outside of the United
States. HDFT aids HSPF's GRAY and GREEN infrastructure modeling applications that use
sub-hourly temporal resolutions. GRAY infrastructure is most often used in urban environments
where stormwater usually flows into stormwater system pipes before reaching a local stream,
lake, or wastewater treatment plant. GREEN infrastructure systems mimic natural processes to
infiltrate, evaporate, and/or reuse stormwater to maintain the pre-development hydrology and
water quality of urban environments.
Vlll
-------
Acknowledgements
I would like to thank Jimmy Bisese for assistance with the Storage Retrieval Data Warehouse
(STORET) web services and David Valentine for assistance with the Consortium of Universities
for the Advancement of Hydrologic Science, Inc. (CUAHSI) web services. I would also like to
thank Fran Rauschenberg for reviewing the report.
IX
-------
Chapter 1: Introduction
The HSPF data-formatting and unit conversion tool has two applications, web-based and
desktop. It was developed to aid users in formatting data for HSPF stormwater modeling
applications. Unlike traditional HSPF applications, stormwater modeling requires time series
data with fine temporal resolution. Properly formatted time series data are then read into
BASINS's WDMUtil program to create a WDM file for HSPF use. This document assumes
readers are familiar with WDMUtil. For more information regarding the WDMUtil tool, the user
is referred to the following link:
WDMUtil tool tutorial
http://water.epa.gov/scitech/datait/models/basins/upload/Exercise-3-WDMUtil.pdf
Note that the WDMUtil program has scripting capabilities for coarse resolution time series, but
not for time series data of hourly and sub-hourly temporal resolution. Existing HSPF data
management tools such as WDMUtil are not very compatible with data containing sub-hourly
temporal resolutions, which are necessary for modeling small urban watersheds with low impact
development practices. Thus, HDFT was developed specifically to format fine resolution data for
HSPF. The process of formatting data to build a WDM file for use in HSPF is presented in
Figure 1. The web-based HDFT program was written in Java and the desktop version in C sharp.
HDFT allows users to create single variable time-series datasets, each of which contains a
uniform date/time format. The program filters extraneous information from time-series datasets
available from publicly accessible web sites (e.g., USGS stream/river flow data, NCDC
precipitation data, and US EPA STORET water quality data), ultimately allowing users to
convert data files to formats recognized by WDMUtil formatting scripts to create WDM files
needed for use in HSPF.
-------
Locally Stored
Data
as .txtorin spreadsheet
HDFT Desktop
Version
Data from web
services
[USGS&EPAStoretJ
HDFT Web
Version
Legend
Application
Properly
Formatted Data
I
(apply .JKfJjjj formatting scripts)
Figure 1: Main Process
-------
HDFT (Web-Version)
-------
Chapter 2: Data-Pasting Option of the HDFT Web-Version
HDFT has been tested with several data formats, including streamflow, meteorological, and
water quality data obtained from the United States Geological Survey (USGS) and the National
Climate Data Center (NCDC). It can download EPA STORET, USGS NWIS, and Instantaneous
Data using CUAHSI and STORET web services. Our intention is to simplify preparing input
data to use within the HSPF model. Although many data types and formats are included in this
Report, other formats currently may be unsupported. Users are encouraged to contact HDFT
developers about unsupported data formats to facilitate their inclusion in future versions of the
program.
Although the primary function of HDFT is to format input datasets properly, convert variable
units, and export the resulting formatted data directly to a file (desktop version) or by copying
and pasting the formatted data into a document (web version), it only incorporates one variable
(e.g., precipitation) at a time. For both desktop and web versions, users must import resulting
formatted data to a WDM file for HSPF use. Time series data that include meteorological and
streamflow data is used by HSPF. Depending on a project's objectives, model users may need to
import data for one variable or for several. For example, in hydrological modeling applications,
only a few metrological parameters and streamflow data are needed. In water quality modeling
applications, however, HSPF model users need additional meteorological data such as cloud
cover, solar radiation, wind speed, and dew point temperature. While HDFT can format data for
all HSPF applications, it was specifically developed for stormwater modeling as this application
requires fine temporal resolution rainfall data ranging from one minute to one hour.
HDFT formats time series data and can convert variable units to those useable by HSPF, as
shown in Table 1, located in Appendix-A. Note that various model input data can come from
several sources such as the USGS, NCDC (ASOS 1 min data, etc.). HDFT minimizes the time
required to format, convert, and export time series data for the HSPF model. Although HDFT
can make many user-specified conversions, it only lists conversions for 10 common variables.
WDM formatting scripts provided in BASINS are not able to format hourly and sub-hourly time
series data properly, as such, additional scripts are included with the tool that allow users to
incorporate data of fine temporal resolution into WDM files.
The web-version of HDFT has two data import options: pasting data to the tool (user data) and
downloading data from the web. Pasting directly to the tool allows HDFT users to format their
locally stored data properly and the download option allows users to get data directly from
USGS or EPA STORET websites.
-------
2.1. Data requirements
• HDFT users must remove all headers/text in the data before pasting into the "Import
Data" window [see Appendix B, Figure 50].
• Must be in a tab-delimited data file (2 Column, USGS NWIS, and USGS Sediment
data format). The tool also checks for missing values and assigns "-999" whenever
missing values are identified and alerts users to deleted rows or "-999" values.
• The tool cannot handle blank dates or missing values if the data is space-delimited or
comma-delimited. FIDFT users are responsible for checking data for missing values
or blank dates before importing.
• The tool can handle only one variable at a time.
• The user must remove any table contents before importing data by right clicking on
the table and selecting Clear Contents.
• If the tool encounters parsing issues with respect to date or value, it will not proceed
and informs the user of the row number causing the error, e.g., having unintended text
in a value column or an unintended value in a date column.
2.2. Formatting options for different data types
The tool has four pre-defined formatting methods, two of which are used for USGS daily
streamflow and sediment data. The others are generic, user-specified methods for various data
formats. Table 1 provides several format selection guidelines. Note that unit conversion is
independent of data format and works the same for all input data.
The Web-based HDFT provides two options for importing data. One is and the other
, . , . . , , ,.,,,,
is . The import data option allows users to provide the tool with locally stored data
by pasting directly to the tool. The download data option first requires users to specify a USGS
or EPA STORET site, after which the tool can retrieve data for the parameter and time period of
interest.
-------
Table 1: Criteria for data-formatting option
Date Format
Date only (with date
separators)
USGS NWIS Format
USGS Sediment
Date only (without
date separators)
Date and time
(without separators)
Date and text as
single string
(with/without time)
Date and Time in
separate columns
(without separators)
Formatting
option to be
selected
Column format
USGS NWIS
USGS Sediment
Other Formats
Date format of input data
2000/0 1/0 lb
or
2000/01/01 00:00b*
2000-01-01
01 Jan 2000
20000101b
200001010000 *
MIA200001010000 *
(text can be either at
starting or at the end of
the string)
20000101 0000 *
Date format of export
data
2000/0 1/0 lb
or
2000/01/01 00:00b*
2000-01-01
2000-01-01b
2000/01/01
2000/01/01 00:00b*
2000/01/01 00:00b*
2000/01/01 00:00 *
b - Any Year, Month, and Day combinations
* - With/without time or minutes
The following sections provide a demonstration of how to import data with different formats and
from various sources (USGS, NCDC, and NNDC, etc.) in HDFT. Sample data for each format
type is available and provided with the tool.
Note: For opening large text files to modify entries or delete headers, users are encouraged to
use the Textpad software accessible at: http://www. textpad. com/.
Steps to follow when manually importing locally stored data:
• Specify data format (column, NWIS flow/sediment, other)
• Select date/time format of data and specify delimiter if used
• Specify the columns of data corresponding to date, time, and value, if necessary
• Select the variable being imported and its corresponding units
-------
2.3. Column Format (user-specified formatting option)
The column format works with any of these date formats -- YYYY/MM/DD, MM/DD/YYYY,
and DD/MM/YYYY, with or without time. Appendix B [Figure 51] illustrates sample data with
column format. Steps to follow when importing data with column format are given in the
following section.
Note: After removing headers within data text files, it is advisable to import data to a
spreadsheet. An application like Microsoft Excel allows users to copy and paste column-
formatted data into the HDFTtool easily.
2.3.1. Formatting Column Format Data
Step 1: Open/Run HDFT tool.
Step 2: A Main HDFT window is opened [Figure 2]
UUR (Ml !MO I
Figure 2: Main HDFT Window
Step 3: Press the Import Data button on the Main Window to import the locally stored data.
Step 4: A new window, Enter Data Information [Figure 3], opens.
-------
DATA FORMAT
• COLUMN FORMAT
O USGS NWIS Format
.USGS SEDIMENT FORMAT
O DTHBl FORMATS
(Single string without spaces For ex: YYYYMMDDHHMM/MIAYYYYMMDDHHMM)
DATE FORMAT
• YYYYMMDD
•'.. MMDDYYYY
C DDMMYYYY
Date separator
Time separator
^ Years
• Month
EDay
DHour
- [
H Minutes
!— '
Type of Delimiter used
O Other
Date and Time Columns • ! ••'• [Mt and Time nSngle String O Date and Time in Separate Columns
Dale column number Time Column Number Value column number Representation of missing samples
VARIABLE
• Flow
O Concentration
CLoad
VARIABLE UNITS
Precipitation
) Temperature
Solar Radiatior
O Cloud Cover
0 Wind Speed
~. Soil Moisture
Dew Point Temperature
els {cubic feet per second) | » |
Import Date and Value
Figure 3: Enter Data Information
Step 5: In the Data Format block [Figure 3] at the top left corner, select Column Format.
Step 6: Select the date format from the Date Format block [Figure 3] at the top right corner. For
this example, YYYY MM DD was selected. Make sure the year, month, and day boxes are
checked.
Note: Do not check hour and minutes because this example dataset uses only daily data.
Note: Make sure correct date formats are selected since errors can occur when date formats are
incorrect.
Step 7: Accept or enter the date and time separators to be used in the formatted output. In the
current example "-" is the date separator.
Note: The tool may not work properly if the date separator entered does not match the date
separator of the pasted data.
Step 8: Select "FLOW" from the Variable selection button of Figure 3 and select the
corresponding units shown at the bottom.
Note: The variable selected will be displayed in the variable combo-box on the Main Window
[Figure 2] after importing the data.
Step 9: Click Import Date and Value [Figure 3] to open an Import Data window [Figure 4].
-------
Import Data From Excel
- || O || X
jet the data in USGS foniuit.-R.iw data
~.o|>y and paste (Ctrl+V) it into the text area below.
:iick 'Import Data' to place the data into the Input Table.
948-10-01 259.00
948-10-02 199.00
948-10-03 184.00
948-10-04 180.00
948-10-05 612.00
948-10-06 763.00
948-10-07 342.00
948-10-08 282.00
948-10-09 276.00
948-10-10 239.00
948-10-11 221.00
948-10-12 216.00
948-10-13 207.00
948-10-14 199.00
948-10-15 195.00
948-10-16 191.00
948-10-17 195.00
Import Data
Figure 4: Import Data Window
Step 10: Paste the raw column data (using Ctrl+V) into the Import Data window, ensuring that
no headers are included. Click the Import Data button to import data into the table as shown in
Figure 5.
Note: If unit conversion is not required, go to the Export Data section (2.3.3) to export the
formatted data. If unit conversion is required, proceed to the next section about the Unit
Conversion capabilities of the HDFTtool.
2.3.2. Available Unit Conversion Options
As stated in Step 8, the selected variable and its unit are displayed in the Enter Data Information
window in the Variable and Units Combo-boxes on the Main Window [Figure 5]. By observing
the original and converted values columns below, the user can determine if the formatting and
conversion processes are complete. For the various unit conversion options available in HDFT,
users are referred to Table 2 (Appendix A). If the tool does not have the desired unit conversion,
users can specify a conversion factor manually [Figure 5].
-------
IMPORT DATA
'sq-mHes
Values in min/day
Variable Flow
USER DEFINED CONVERSIONS
CONVERSION FACTOR
Date
1948-10-01
1 948-1 0-02
1948-10-03
1948-10-04
1948-10-05
948-10-08
948-10-07
948-10-08
948-10-09
948-10-10
948-10-11
948-10-12
948-10-13
948-10-14
948-10-15
948-10-16
948-10-17
948-10-18
948-10-19
948-10-20
948-10-21
948-10-22
1948-10-23
1948-10-24
1948-10-25
1948-10-26
1948-10-27
1948-10-28
1948-10-29
1948-10-30
1948-10-31
1948-11-01
Original Values
259.0
199,0
1 84 0
180.0
6120
763.0
342.0
282 n
276.0
239.0
221.0
216.0
207.0
1990
195.0
191.0
195.0
244.0
287.0
212.0
199.0
191.0
191.0
191.0
188.0
188.0
180.0
180.0
180.0
180.0
180.0
184,0
Converted Values
259.0 *•
199.0
1 84 0
180.0
6120
783.0
342.0
282 0
276.0
239.0
221.0
216.0
207.0
1 ']<] 0
195.0
191.0
195.0
244.0
287.0
212.0
199.0
191.0
191.0
191.0
188.0
188.0
180.0
180.0
180.0
180.0
180.0
184.0 -
Units cubic feet per second (cfs) v
Figure 5: HDFT Main Window with Imported Data
Step 11: Select the desired units in the Units Combo-box located at the bottom of the Window
[Figure 5]. In the current example, units of "cfs" (cubic feet per second) are converted to "cms"
(cubic meters per second).
Step 12: The converted values are shown in the Converted Values column [Figure 6].
Note: The above two steps are common to all conversions and are used for data formats.
Note: Some variables can have multiple unit conversions.
Conversion of Flow units to mm/day (optional)
Enter the drainage area in the Drainage Area textbox and select the corresponding area units
[Figure 6]. The converted values are displayed on the table by clicking the Values in mm/day
button or by selecting the mm/day option from the Units Combo-box at the bottom of the
Window. Select only one option to convert values properly.
Determining Cumulative volumes (optional)
After formatting the Flow data, select the Cumulative Volume option from the Variable Combo-
box on the Main Window [Figure 6]. The unit in the Units Combo-box changes correspondingly;
for example, if cubic feet per second is the original unit, cubic feet will appear by default. Figure
7 illustrates conversion of flow data from cubic feet per second to cumulative flow volume (acre-
feet per day)
10
-------
Applet
IMPORT DATA
1 I
|sq.miles »
USER DEFINED CONVERSIONS
| CONVERT |
Applet started.
Date
948-10-01
948-10-02
948-10-03
948-10-04
948-10-05
948-10-06
948-10-07
948-10-08
948-10-09
948-10-10
948-10-11
948-10-12
948-10-13
948-10-14
948-10-15
948-10-16
948-10-17
'345- lu- Ifi
948-10-19
943-10-20
948-10-21
948-10-22
948-10-23
948-10-24
948-10-25
948-10-26
948-10-27
943-10-29
948-10-29
948-10-30
948-10-31
948-1 "l-01
"* nl° 11 "^
^^^^^
Variable Flow
Orioina! Values
259.0
199.0
164.0
1SOO
6120
/B30
342.0
282.0
276.0
239.0
221.0
216.0
207.0
199.0
350
91 0
95.0
44,0
87.0
12.0
39.0
91.0
91.0
91.0
88.0
88.0
800
BOO
BOO
800
800
mi
100 n
Units cubic metre
^^^^^
*
Converted Values
7.3340633
5.6350525
5.2102993
5.0970324
17 32991 D1
21.6057539
9.6843815
7.9853507
7.8154497
5.7677263
627.150231
51164383
5.8615872
56350025
5.5217851
5.4085177
5.5217851
6.9093106
3.126935
60031715
5.6350525
5.4085177
5.4085177
5.4085177
5.3235672
5.3235672
5.0970324
50970324
5.0970324
5.0970324
5.0970324
52102996
K 1V1KR71
per second (ana) |T
•
-
—
^^^•BMU
EXPORT DATA
Figure 6: Conversion of flow data from cfs (cubic feet per sec) to cms (cubic meter
per sec)
Applet
IMPORT DATA 1
h 1
Semites -|
USER DEFINED CONVERSIONS
| CONVERT |
Applet started
Date
1 918-1 0.01
518 10-02
948-1 0-03
948-10-04
948-10-05
918-10-06
948-10-07
948-10-08
84JMO-QJ
948-10-10
918-10-11
948-10-12
948-10-13
948-10-14
948-10-15
918-10-16
948-10-17
948-10-18
943-10-19
918-10-20
948-10-21
915-10-22
948-1 0-23
948-1 0-24
948-10-25
948-10-26
948-10-27
948-10-28
918JO-29
918-10- JO
918-10-31
•J18-! 1-01
Variable Cumti
Original Values
2590
199.0
181.0
180.0
512-0
7630
342.0
2820
276.0
238.0
221.0
216.0
2070
199.0
195.0
191.0
195.0
244.0
287.0
212.0
199.0
191-0
191.0
191.0
1880
188.0
180.0
180.0
180.0
180.0
180.0
184.0
Units aciefeel
^^•^^•L
ative Volume T
Converted Values
513597
908.214
1 27 :: 0.36
1630.026
2843.622
4356651
5034837
5594.043
6141 351
6615.288
7063.531
7481 .859
7892.34
8286.957
8673.642
3052 395
9439.08
3922932
10492.053
1 09 1 2 119
11307.066
11685.819
12064.572
12443.325
12816129
I s J3J
13545873
13902.813
14259753
116 i 1,693
14973.633
15338505
3
»
-
—
! EXPORT DATA
Figure 7: Conversion of imported flow data to cumulative volume
11
-------
2.3.3. Exporting Formatted and Converted Data
For the HDFT web-version, the formatted/converted data can be copied to the Clipboard and
then pasted into MS Excel or text-editing software, as a tab-delimited file, for HSPF model input.
Step 13: Click the Export Data button on the right hand side of the Main Window [Figure 7] to
open the Export Data window [Figure 8].
Step 14: On Figure 8, select the desired output Date format and click the Copy to Clipboard
button.
Note: If the converted/formatted data is in YYYY-MM-DD format, as is the example, the data can
be exported to MM/DD/YYYY or DD/MM/YYYY formats. The tool also allows users to export
only portions of the data by selecting Start and End dates. If specific dates are not selected, the
entire dataset will be exported.
Export Data
n x
Date Format
• year month day
O month day year
O day month year
Date separator |.' |
Time separator \: \
0 Years
0 Month
DHour
D Minutes
Start date
End date
1948-10-01
1949-01-23
Copy to Clipboard
Figure 8: Export Data Window
2.4. USGS NWIS Format
Figure 52 (Appendix B) illustrates a sample USGS NWIS dataset.
USGS NWIS data can have different date formats, but the USGS NWIS option of the HDFT
works with only one, YYYY-MM-DD. If the USGS NWIS data has another format, users can
select "Other Formats" (see Section 2.6) to format/convert the data properly.
USGS NWIS data can be obtained at: http://waterdata.usgs.gov/nwis.
12
-------
2.4.1. Formatting NWIS Data
Repeat Steps 1 through 4 of Section 2.3.1.
Step 5: In the Data Format block [Figure 9], select the format of the raw data, USGS NWIS
Format.
Note: If the USGS NWIS option is selected, the options in the Date Format block will be
inaccessible because the USGS NWIS option uses only YYYY-MM-DD.
Enter Data Information
DATA FORMAT
COLUMN FORMAT
• USGS NWIS Format
O USGS SEDIMENT FORMAT
O OTHER FORMATS
(Single string without spaces for ex: YYYYMMDDHHMM/MIAYYYYMMDDHHMM)
Type of Delimiter used
® Space/Tab
O Other
DATE FORMAT
®YYYYMMDD
OMMDDYYYY
ODDMMYYYY
Date separator
Time separator
@ Years
0 Month
0Day
DHour
O Minutes
Date and Time Columns (•) Date / Date and Time as Single String O Date and Time in Separate Columns
Date column number Time Column Number Value column number Representation of missing samples
VARIABLE
{i: Flow
O Concentration
O Load
VARIABLE UNITS
O Precipitation
O Temperature
O Solar Radiation
Q Cloud Cover
O Wind Speed
O Soil Moisture
Dew Point Temperature
cfs (cubic feet per second) •»
Import Date and Value
Figure 9: Data Information window - USGS NWIS Format
Repeat Steps 6 through 10 of Section 2.3.1 to import the formatted data into the main table
[Figure 2].
13
-------
2.4.2. Available Unit Conversion Options
Repeat Section 2.3.2. to convert the imported data to different units.
2.4.3. Exporting Formatted and Converted Data
Repeat Section 2.3.3. to export the formatted/converted data.
2.5. USGS Sediment format
Figure 53 (Appendix B) illustrates sample USGS Sediment data.
The USGS Sediment data has a DD-MMM-YYYY date format and HDFT converts this to
YYYY-MM-DD format before importing data into the table.
The USGS Sediment data can be obtained at: http://co.water.usgs.gov/sediment/
2.5.1. Formatting USGS Sediment Data
Repeat Steps 1 through 4 of Section 2.3.1.
Step 5: In the Data Format block [Figure 9], select the raw data's format, USGS Sediment
format.
Note: If the USGS Sediment option is selected, options in the Date Format block will be
inaccessible since the USGS Sediment option uses only DD-MMM-YYYY
Repeat Steps 6 through 10 of Section 2.3.1. to import formatted data into the Main Page table
[Figure 2].
2.5.2. Available Unit Conversion Options
Repeat Section 2.3.2. to convert the imported data to different units.
2.5.3. Exporting Formatted and Converted Data
Repeat Section 2.3.3. to export the formatted/converted data.
14
-------
2.6. Other Formats
This option works with multiple data formats to format data collected by different federal
agencies for use within HSPF.
Data requirement
• The date format should be a single string (without spaces), with date and time in the same
or separate columns
• The date/time sample may have text such as MIA2009051819000000 at the beginning or
end of the string
• Date or value should not be blank, especially when data is space-delimited
2.6.1. NOAA ASOS (1 Minute Data)
Figure 54 (Appendix-B) illustrates sample ASOS data (1 Minute Data).
ASOS data can be obtained from:
http://www.ncdc.noaa.gov/oa/climate/climatedata.htmltfasosminutedata
2.6.1.1. Importing ASOS data
Note: Clear any existing data by right clicking on the table and selecting Clear Contents.
Repeat Steps 1 through 4 of Section 2.3.1
Step 5: In the Data Format block [Figure 10], select Other Formats and then Date column and
Value column by typing numbers in their respective textboxes [Figure 10].
For the example ASOS data, the Date column number is 2 and the Value column number is 3
[Figure 10].
Step 6: Select the format from the Date Format block [Figure 10] that corresponds to the Data
format (e.g., YYYY MM DD in this example) and check the hour and minute checkboxes since
this data has date and time formats.
Step 7: Accept or enter the date and time separators that correspond to the data. In the current
example, "-" is used as the date separator.
Note: Specifying the correct date format and temporal resolution is necessary for HDFT to read
the input data properly. Date and time separators are specified, to control the characters used
by the tool to separate date/time in the formatted data.
Step 8: Select/Enter a delimiter for the dataset. In the current example, Space/Tab delimiter is
selected.
Step 9: Enter the missing samples identifier to identify them or leave the default value as "-999."
15
-------
Step 10: Since date and time are recorded as a single string for this data, select the "Date/Date
and Time as Single String" radio-button that corresponds to Date and Time columns.
Note: The Time column number text box will be enabled only if a separate Date and Time
columns radio-button is selected.
Step 11: Select the variable (e.g., precipitation) from the variable list and the corresponding units
[Figure 10].
Note: The variable selected in this step will be displayed in the Variable combo-box on the Main
Window when the data is imported.
Repeat Steps 9 and 10 of Section 2.3.1. to import the formatted data into the HDFT Main table
[Figure 11]
Raw date format-ABE2002010111321632
Formatted date - 2002-01-0111:32
DATA FORMAT
COLUMN FORMAT
C USGS NT/VIS Format
USGS SEDIMENT FORMAT
• OTHER FORMATS
(Single string without spaces for en: YYYYMMDDHHMM/MIAYYYYMMDDHHMM)
Type of Delimiter used
• Space/Tab
O Other
DATE FORMAT
• YYVYMMDD
C MMDDYYYY
DDMMYYYY
Date separatoi
Time separator ^
0 Years
0 Month
0Day
0Hour
0 Minutes
Date and Time Columns • Date / Date and Time as Single String Q Date and Time in Separate Columns
Date column number Time Column Number Value column number Representation of missing samples
2 |3 I 1-999 1
VARIABLE
'Flow if' Precipitation Q Cloud Cover O Dew Point Temperature
Concentration Temperature O Wind Speed
0 Load O Solar Radiation V Soil Moisture
VARIABLE UNITS
(inches
Import Date and Value
Figure 10: Data Information window - Other Formats- ASOS
16
-------
IMPORT DATA
Values In mm/day
USER DEFINED CONVERSIONS
CONVERSION FACTOR [T
• Multiply O Divide
Date
2002-01-01 1:32
2002-01-01 1 33
2002-01-01 1:34
2002-01-01 1:35
2002-01-01 1:36
2002-01-01 1:37
2002-01-01 1:38
2002-01-01 1:39
2002-01-01 1:40
2002-01-01 1:41
2002-01-01 1:42
2002-01-01 1:43
2002-01-01 1:44
2002-01-01 1:45
2002-01-01 1.46
2002-01-01 1.47
2002-01-01 1 48
2002-01-01 1 49
2002-01-01 1 50
2002-01-01 1:51
2002-01-01 1:52
2002-01-01 1:53
2002-01-01 1:54
2002-01-01 1:55
2002-01-01 1:56
Original Value;.
3.062
0.068
0.065
0.07
0.07
0.071
0.063
0.064
0.066
0069
0.065
0066
0.069
0.07
0.063
0.06
0063
0.062
0.062
0.065
0.066
0.064
0.064
0.061
0.064
Converted Values
0.062 •»
0.068 J
0.065
0.07
0.07
0.071
0.063
0.064
0.066
0.069
0.065
0.066
0.069
0.07
0.063
0.06
0.063
0.062
0.062
0.065
0.066
0.064
0.064
0.061
0.064 —
EXPORT DATA
Applet started.
Figure 11: Formatted ASOS 1 Minute data
2.6.1.2. Conversion of Variable Values
Repeat Section 2.3.2. to convert imported data to different units.
2.6.1.3. Exporting Data
Repeat Section 2.3.3. to export formatted/converted data.
2.6.2. Instantaneous time-series data (USGS) - Flow
Figure 55 (Appendix-B) illustrates sample Instantaneous time-series data, which usually has 15
minute intervals. The data can be obtained at: http://ida.water.usgs.gov/ida/
2.6.2.1. Importing data
Note: Clear any existing data by right clicking on the table and selecting Clear Contents.
Repeat Steps 1 through 4 of Section 2.3.1.
Step 5: In the Data Format block [Figure 10], select Other Formats. Type numbers in the Date
column and Value column in the textboxes [Figure 10].
For this example, the Date column number is 2 and the Value column number is 6 [Figure 55
(Appendix-B)].
Step 6: Select the date format from the Date Format block [Figure 10] and check the hour and
minute checkboxes since the data has date and time format.
17
-------
Step 7: Accept or enter the date and time separators corresponding to the data. In the current
example, "-" is used as the date separator.
Step 8: Select/Enter the delimiter used for the dataset. In the current example, Space/Tab
delimiter is selected.
Step 9: Enter the missing samples identifier to indicate the presence of missing data or leave the
default value as "-999".
Step 10: Since date and time are in a single string for the example data, we used the default Date
and Time columns selection "Date/Date and Time as Single String."
Note: The Date and Time column number textboxes are enabled only when the Date and Time in
Separate Columns radio-button is selected.
Step 11: Select the variable (e.g., flow) from the variable list and its corresponding units (e.g.,
cfs).
Note: The variable selected in this step will be displayed in the Variable combo-box on the Main
window when the data is imported.
Repeat Steps 9 and 10 of Section 2.3.1. to import the formatted data into the Main Page table
[see Figure 11]
Raw Date Format - 20040127000000
Formatted date - 2004-01-27 00:00
2.6.2.2. Conversion of Variable Values
Repeat Section 2.3.2 to convert the imported data to different units.
2.6.2.3. Exporting Data
Repeat Section 2.3.3 to export the formatted/converted data.
2.6.3. Hourly data NCDC - U.S. Climate Reference Network (USCRN) - Temperature
Figure 56 (Appendix-B) illustrates sample NNDC hourly climate data obtained from:
http://gis.ncdc.noaa.gov/map/cdo/
2.6.3.1 Importing data
Note: Clear any existing data by right clicking on the table and selecting Clear Contents.
Repeat Steps 1 through 4 of Section 2.3.1.
Step 5: In the Data Format block [Figure 10], select Other Formats and specify the Date column
number and Value column number. The data in this example has a Date column number of "1"
and Value column number of "14" [Figure 56 (Appendix B)].
18
-------
Step 6: Select the date format from the Date Format block [Figure 10] and check the hour
checkbox, but not the minute checkbox since this is hourly data.
Step 7: Accept or enter the date and time separators corresponding to the data. In the current
example, "-" is used as the date separator.
Step 8: Enter the missing samples identifier to identify missing samples. In the current example,
-999 is used.
Step 9: Since the data for this example is comma-delimited, select Other under the Type of
Delimiter Used and enter a comma (",") into the delimiter textbox to specify the delimiter used
for the dataset [Figure 10].
Step 10: Select the variable (e.g., temperature) from the variable list and the corresponding unit
(e.g., Centigrade).
Repeat Steps 9 and 10 of Section 2.3.1. to import the formatted data into the HDFT Main table
[Figure 12].
2.6.3.2. Conversion of Variable Values
Repeat Section 2.3.2. to convert the imported data to different units.
2.6.3.3. Exporting Data
Repeat Section 2.3.3. to export the formatted/converted data.
19
-------
IMPORT DATA
Variable j
Values in mm/day
USER DEFINED CONVERSIONS
CONVERSION FACTOR [T
(• Multiply Divide
CONVERT
2009-
2009-
1-009-
2009-
2009-
2009
2009;
:n el':,
2009..
J'OOO
2009-
2nng-
2000-
2000-
2009-
2009-
2009.
2009-
2009-
J009-
20H9-
2000
2000-
2009;
2009
2000.
2000-
20 no
2000-
2000.
2000-
2909-
• M...
2009
2009
2009-
2009-
2000
2009-
2009-
;.'0r<9-
2000
2000
2009;
20115-
2000-
2009-
2009-
-01
-01
-m [
-01 >:
01 i
-01
-01
-01
-01
•01
-01
-01
-01
01
-01
-01
-01
-01
-01
-01
-01
-01
-01
-02
-02
•02
-02 C
-n: (
• o ; c
-02 (
-02 0
-03
-02
-02
-02
-ni
- 0 ;;
-02
02
-02
-02
•02
-02
-02
-02
•02
-02
-03
J.D
3
.3
g
4
3
.1
}.1
2.9
5.1
F, -;
7 5
7.3
7.0
f. 7
i.1
5.5
4.B
4 ?
i y
3.3
2.8
1 1
).2
-8
G
0 1
).D
).2
?.4
1 0
2,0
3.7
5.0
5.6
5 4
J.7
7.2
7.5
7.3
5.2
2.3
1.4
0 7
.g
.9
7
)
1
.1
9
9
9
7
Figure 12: Formatted NCDC hourly climate data
2.6.4. NCDC- DS3505 - Surface Data, Hourly Global - Date and Time in Separate Columns -
Solar Radiation
Figure 58 (Appendix B) illustrates sample NCDC hourly global climate data. Global climate data
can be obtained from: http://gis.ncdc.noaa.gov/map/cdo/
2.6.4.1. Importing data
Note: Clear any existing data by right clicking on the table and selecting Clear Contents.
Repeat Steps 1 through 4 of Section 2.3.1.
Step 5: In the Data Format block [Figure 10], select Other Formats and the Date column and
Value column numbers in their respective textboxes.
Step 6: Select the correct date format from the Date Format block [Figure 10] and check the hour
and minute checkboxes since the data have both date and time formats.
Step 7: Accept or enter the date and time separator that corresponds to the raw data. In the
current example, "-" is used as the date separator.
Step 8: Select/Enter the raw data delimiter. In the current example, Space/Tab option is selected.
20
-------
Step 9: Select Date and Time in the separate columns radio button, the Time Column Number
textbox will now be accessible.
Step 10: Enter the column number associated with the time string.
In the current example, enter value "3" in the Date column number textbox, "4" in the Time
column number textbox, and "7" in Value column number textbox.
Step 11: Enter the missing samples identifier to identify missing samples or leave the default
value, "-999."
Step 12: Select the variable from the variable list (e.g., Solar Radiation) and the corresponding
units (e.g., Langley per Hr).
Note: The selected variable will be displayed in the Variable combo-box on the Main window
after importing the data.
Repeat Steps 9 and 10 of Section 2.3.1. to import the formatted data into the table on the Main
Page (similar to Figure 12).
Raw Date Format-20040127 0000
Formatted date - 2004-01-27 00:00
2.6.4.2. Conversion of Variable Values
Repeat Section 2.3.2. to convert the imported data to different units.
2.6.4.3. Exporting Data
Repeat Section 2.3.3. to export the formatted/converted data.
21
-------
Chapter 3: HDFT Data Downloading Options (Web-version)
In addition to handling different data formats imported into the HDFT web-version with data-
pasting, as discussed in the previous chapter, web-version users have data download options to
retrieve flow and water quality data directly from the following sources:
• USGS NWIS Data (Daily data)
• USGS Instantaneous Irregular data
• EPA-STORET
Compared to data-pasting, downloading is faster and more efficient. It is limited, however, by
the availability of web services and databases amenable to web download. The download option
uses web services provided by the Consortium of Universities for the Advancement of
Hydrological Science (CUAHSI) and EPA-STORET. A web service is a way for devices to
communicate over the web using standard protocols — specifically, they automate transfer of
available data from one location to another. For additional information regarding data download
using CUASHI and other services, HDFT users may refer to: http://www.cuahsi.org/ or to
http://www.w3.org/TR/ws-arch/ for additional information.
Using the CUASHI web services, the HDFT tool can download about 15 parameters (physical,
nutrient, microbiological). Using the EPA-STORET, the tool can download any parameter
available in the STORET database. For practical reasons, we limited the variable groups
downloaded from one station to four types — physical, nutrient, microbiological, and other.
When the data of interest have multiple measurements taken on a single day, HDFT imports only
the average value of the parameter; it also sorts the data by date. Multiple measurements in a
single day are common to USGS Instantaneous data, but less so to EPA-STORET databases.
3.1. Details on example data used for explaining the downloading option:
Data source: USGS & EPA-STORET
USGS:
Site number : 05570370
EPA-STORET
Organization ID : 11NPSWRD
Station/Location ID : CHIS_NPS_Q3
Note: The organization and station IDs are not case sensitive.
To download data from a USGS website, users must have a USGS Site number. Data
downloaded from EPA STORET require Organization and Station IDs.
For information on obtaining USGS site numbers, HDFT users may refer to Appendix-E. For
information on obtaining EPA STORET Organization and Station IDs, HDFT users may refer to
the Appendix-D.
22
-------
3.2. Steps to follow when downloading the data are:
• Enter Station ID (USGS) or Organization and Station IDs (EPA-STORET)
• Select a variable
• Extract station and parameter information - Station name, Parameter units (if available)
and Start and End dates
• Enter dates for which data is required
• Download Data
3.3. Downloading data from USGS NWIS Daily data & USGS Instantaneous
Irregular Data
Clear any existing data by right clicking on the table and selecting Clear Contents.
Step 1: On the Main Panel [Figure 12], click the Download Data button to open a Data
Download window [Figure 13] for downloading data.
Enter the Station Information.
Enter the Start and End Dates
Click 'Download Data' to place the data into the table
Click to get information for the selected site
Database
NWIS D.iity values
USGS Station Number |05570370
Organization ID
Variable
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name
Location (Latitude, Longitude)
Start Date
End Date
Number of records available
Get Site Info
Accept filter the dates for which data has to be downloaded
Start Date
Select Units
Download Data
Figure 13: Data Download Window
Step 2: Select the database from which the data must be downloaded from the Database combo-
box [Figure 13], e.g., NWIS daily values.
Step 3: Enter the USGS Station number, e.g., 05570370.
Note: Refer to Appendix-E for obtaining the USGS Station number if unavailable.
Since Organization and Station IDs are required for only EPA-STORET, the corresponding text-
boxes will be inactive for the selected database, e.g., NWIS Daily values.
23
-------
Step 4: Select the water quality parameter from the Variable combo-box [Figure 13] for which
the data must be downloaded (Flow in this example).
Step 5: Click the Get Options button [Figure 13] to open Selection of a Variable Window, which
provides additional variable selection options [Figure 14].
B Selection of variable
Select the type of variable
C 3 Discharge, m3/s
O Discharge, instantaneous, ft3/s
• Streamflow, ft*/s
O Slreamflow, m3/s
Number of samples
Data not available
Data not available
7610
Data not available
Figure 14: Selection of variable window
Step 6: Select the "Streamflow, ft3/s" option and click "Done."
Note: Some parameters may use multiple variable names (e.g., Discharge and Streamflow). To
ensure access to data, all possible variable names are queried.
Step 7: Click the Get Site Info button on the Data Download Window [Figure 13] to extract
information such as Site name, Units of the parameter, and Start and End dates of the data
available [Figure 15].
The corresponding units available for conversion are added to the Units combo-box [Figure 15].
24
-------
E5 Data Download Window
Enter the Station Information.
Enter the Start and End Dates
lick 'Download Data* to place the data into the table
Database
Click to get information for the selected site
NWIS Daily values
USGS Station Number [05570370 I
Organization ID
Variable
Flow
Get Options
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name JBIG CREEK NEAR BRYANT, IL
Location (Latitude, Longitude)
40.45393096923828 , -90-1 334533691 4062
Start Date 1971-12-01
End Date |1 992-09-30
Number of records available
Discharge, cubic feet per second
Get Site Info
Accept/Enter the dates for which data has to be downloaded
Start Date
|1 971-1 2-01 |
|1992-09-30 |
Select Units cubic feet per second (cfs) --•
Download Data
Figure 15: Data Download Window with Station information extracted
Step 8: Enter required Start and End dates based on the above available information, and select
the units for the available parameter in the Select Units combo-box, e.g., cubic feet per second
(cfs) [Figure 16].
Important: When entering Start and End dates, use the same date format (including date
separators) as displayed in Begin and Last Date textboxes. In the current example, the date
format is YYYY-MM-DD.
Step 9: After clicking the Download Data button [Figure 16], the data (date and value) will be
downloaded to the table on the Main Panel [Figure 17].
The steps for downloading data from the USGS Instantaneous irregular data are similar to
downloading USGS Daily Data. Start by selecting USGS Instantaneous irregular data from the
database combo-box in Step 2.
Repeat Section 2.3.2. to convert the imported data to different units and repeat Section 2.3.3. to
export the raw or converted data.
25
-------
• Data Download Windc
=nter the Station Information.
Enter the Start and End Dates
Click 'Download Data1 to place the data Into the table
Database
Click to get information for the selected site
NWlS-Daily values
USGS Station Number |OS570370
Organization ID
Variable
Flow
| Get Options
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name |BIO CREEK NEAR BRYANT, IL
Location (Latitude, Longitude)
40.45893096923828 -90 13345330914062
Start Date |l 971-1 2-01
Discharge, cubic feet per second
End Date [1 992-09-30
Number of records available
Get Site Info
Accept/Enter the dates for which data has to be downloaded
Start Date
|1 971 -12-01 |
|1990-09-30 |
Select Units cubic feet per second (cfs) ^
Download Data
Figure 16: Data Download Window-USGS NWIS - Entering Start and End dates
IMPORT DATA
DOWNLOAD DATA
USER DEFINED CONVERSIONS
CONVERSION FACTOR [T
•* Multiply C Divide
CONVERT
9 - ;-i
yr - 2-1
•-( - :-i
y - 2-i
y - 2-i
9 - 2-1
.:, . ;,.i
9 - 2-1
y - 2-i
y - 2-
'5 - 2-
y - 2-
y - 2-
9 - 2-
.:, . ;.
9 -' 2-
y - ;.-
y - 2-
19 - 2-
9 -' 2-
y - 2-
9 - 2-
ri ;-
9 -' 2-
y - 2-
y - 2-
19 - 2-
? • 2-
y - 2-
19 - 2-
'"i 2-
•'
19 2-01-
'? 2 di-
'3 i-ijl-
':i ?-o i-
'J 2 iji-
19 2-01-
'-i ;• r, i -
•' • i •
y '• hi
g 2 01-
'3 3-OI-
9 ?-rii-
9 2-01-
y ; n i
'3 J CM.
9 2-01-
y 2 Hi
^
.5
.5
.0
.0
1.0
5.0
1.0
.0
0.0
0.0
50
00
0.0
00
00 0
80
5.0
2.0
2.0
3.0
3.D
20
4.0
6.0
6.0
7.0
6.0
2.0
1.0
760
4.0
0.0
5.0
0.0
5.0
0.0
8.0
80
n n
^n
0.0
8.0
8.0
8.0
5,0
00
.0
0
0.0
9.5
3.5 =
8.0
9.0
11.0
15.0
11.0
9.0
10 0
70.0
350
30.0
30.0
30.0
100.0
38.0
350
32.0
32.0
330
33.0
32.0
34.0
36.0
36.0
37.0
360
32.0
31.0
176.0
64.0
40.0
35.0
30.0
25.0
20.0
18.0
180
20.0
25.0
20.0
18.0
18.0
18.0
15.0
100
7.0
7.0
10.0 »
Units cubic feet per second (cfs)
Figure 17: Data imported using download option- USGS NWIS Daily/ Instantaneous
data
26
-------
3.4. Downloading Data from EPA-STORET
Downloading data from EPA STORE! is slightly different from USGS NWIS. Data from the
USGS NWIS is downloaded using Station number only and a unique identifier for each
parameter; EPA-STORET uses organization ID, station ID, and a parameter name. EPA-
STORET parameter names and IDs are not unique, and data uploaded from different states and
organizations may have different parameter names for the same water quality constituent. For
example, some sites use "Total suspended solids" while others use "Solids, suspended." Thus,
using different names for the same parameter may give erroneous results or provide less data
than is actually available in the database. To avoid errors caused by non-unique parameter
names, an intermediate step to show all data available at a STORET site was implemented,
allowing HDFT users to view a list of variables for download which facilitates parameter
selection.
The above issues and, potentially, others may not create problems in downloading available data;
however, issues may arise in unit-conversion, requiring users to select "Other" in the Select
unit's combo-box [Figure 16]. Selecting "Other" units allows users to define conversion options
manually by entering the factor to be used to change units of the downloaded data on the left-
hand side of the Main Panel [Figure 17].
The following steps explain how to download data from EPA-STORET.
Clear any existing data on the table by right clicking on the table and select Clear Contents.
Step 1: On the Main Panel, click the Download Data button to open a Data Download window
[Figure 17].
Step 2: From the Database combo-box, select the database from which data is to be downloaded
[Figure 18], e.g., EPA-STORET.
Step 3: Enter the Organization and Station IDs in the textboxes; for the current example, use
llnpswrd and chis_nps_q3, respectively [Figure 18].
Note: to obtain Organization and Station IDs for a STORET site, refer to Appendix-D.
Since Organization and Station IDs are only required for EPA_STORET, the USGS Station
Number textbox will be inactive.
Step 4: Click the Get Options button [Figure 18] to open STORET - Parameters Available
window that provides selection options [Figure 19].
Note: The Variable combo-box is not available to select the parameter.
Step 5: From the available list, select the parameter by selecting the corresponding row and click
"DONE" [Figure 19] to close the window.
For the current example, select "Dissolved Oxygen."
27
-------
B Data Download Win do
-JIM.i the Station Information.
Enter the Start and End Dates
Click 'Download Data* to place the data into the table
Click to get information for the selected site
Database EPA-Storet
USGS Station Number
Organization ID [1 Inpswrd
Station ID |chis_nps_q3J
Get Options
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name
Location (Latitude, Longitude)
Start Date
End Date
Number of records available
Accept;Eirter the dates for which data has to be downloaded
Start Date
Select Units
NOTE: The date format should be similar to the dates displayed in Begin/ Last dale texboxes
Download Data
Figure 18: Data download window - EPA STORE!
B STORET Available Parameter* fT H X
List of available parameters
Location Type
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
River/Stream
Characteristic Type
Microbiological
Microbiological
Nutrient
Other
Physical
Physical
Physical
Physical
Physical
Physical
Physical
Physical
Physical
Characteristic Name
Fecal Conform
Total Conform
Phosphate-phosphorus as P
Nitrogen
Dissolved oxygen (DO)
rlow
Salinity
Solids, Dissolved
Solids, Total Suspended (TSS)
Specific conductance
Temperature, air
Temperature, water
Turbidity
^
•*
Select parameter /Tow and click "DONE"
[ Done |
Figure 19: EPA STORET- Parameters available
Note: Occasionally, the user may observe that a parameter name appears in two
categories/characteristic types — for example, Inorganic Nitrogen (Nitrate+Nitrite) may be
displayed in the "Nutrient" category and in "Other". The objective is to query for the exact
28
-------
name of the parameter, which is not constant for every site, to avoid issues while downloading
data. The tool will download all available data that corresponds to the selected parameter.
Step 6: Click the Get Site Info button on the Data Download window [Figure 18] to extract
information such as Site name, Units of the parameter, and Start and End dates of the data
available [Figure 20].
The units available for conversion are added to the Units combo-box [Figure 20].
H Data Download Window
inter the Station Information.
inter the Start and End Dates
Click 'Download Data* to place the data into the table
Click to get information for the selected site
Database |EPA Storet
USGS Station Number
Organization ID
l 1 npswrd
Station ID chls_nps_q3
Get Options
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name |OUEMADA CANYON NEAR OLD RANCH CANYON CORRAL
Location (Latitude, Longitude) [33.9701 473 .-119.9991 906
Start Date |l 0/03/1 993 I
Dissolved oxygen (DO), - ugl
End Date 11 2/0 4/1 9 9 4
Number of records available
Get Site Info
Accept/Enter the dates for which data has to be downloaded
Start Date |l 0/03/1993
Select Units mg I
End Date |l 2/04/1994
NOTE: The date format should be similar to the dates displayed in Begin/ Last date texboxe
Download Data
Figure 20: EPA STORET- Site information extracted
Step 7: Enter the required Start and End dates based on the above information and select the
units for the available parameter in the Select Units combo-box, e.g., ug/L [Figure 21].
Note: If the units displayed in the Units combo-box do not match units displayed with the
parameter name, select "Other"for units.
Important: While entering Start and End dates, use the same date format as displayed in the
Begin and Last date textboxes (including date separators). In the current example, the format is
MM/DD/YYYY.
Step 9: After clicking the Download data button [Figure 21], the data (date and value) will be
downloaded to the table's Main Panel [Figure 22].
Repeat Section 2.3.2. to convert the imported data to different units and repeat Section 2.3.3. to
export the raw or converted data.
29
-------
• Data D
Enter the Station Information.
Enter the Start and End Dates
Click 'Download Data* to place the data into the table
Click to get information for the selected site
Database EPfl Slorel
USGS Station Number
Organization ID J11 npswrd
Station ID |chis_nps_q3
| Gel Options
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Station Name |OUEMADA CANYON NEAR OLD RANCH CANYON CORRAL
Location (Latitude, Longitude)
133.9701 473,-1 1 9.9991 906
Start Date 10/03/1993
End Date |1 2/04/1994
Number of records available
Dissolved oxygen (DO), - ugJI
Get Site Info
Accept Enter the dates for which data has to be downloaded
Start Date |l 0/03/1993
End Date
Select Units u- Multiply " Divide
993- 0-03
393-' 0- 3
9^.v rt- R
993- 1- 1
W;- 1- 1
993- 2- 0
994-01- 5
991-01- 2
994-02- e
994-03- 6
994-03- 1
994-03- 1
9'34-OJ- fi
994-04- 9
^.M.J.IS- 4
994-05- 9
994-06- 2
994-06- E
994-07- 4
994-08- 2
994-09-01
9960.0
3000.0
3350.0
o4il 0
4430.0
3720.0
5570.0
8030.0
3280.0
4880.0
0110.0
47200
oeeo.o
300.0
1210.0
10.0
3290.0
3620.0
3430.0
360.0
29000
60.0
rinn n
f,n n
0.0
30.0
I ?20.Q
70.0
30.0
: 30.0
80.0
10.0
20.0
si; 0
; 0.0
: 10.0
.0
90.0
20.0
30.0
0.0
i-jii n
EXPORT DATA
Units [ug/l
Figure 22: Data imported from EPA STORE!
30
-------
HDFT (Desktop Version 1.0)
31
-------
Chapter 4: HDFT (Desktop Version)
4.1. Introduction
The desktop version's formatting and data-conversion capabilities are limited since its intended
use is for large datasets the web-based version cannot handle. Further, the desktop version can
read and write data text files — a flexibility the Java-based web-tool does not have because of
security concerns.
Preparing model input data can take considerable time and formatting errors may impact model
performance. This manual provides data formatting and conversion information for a web-based
tool and a desktop tool. Outputs from HDFT are importable to a WDM file, a FORTRAN binary
direct-access file used by hydrological and water quality models such as the Hydrological
Simulation Program -FORTRAN (HSPF). The primary purpose of HDFT is to create a text-
based time series data file that can be read by the WDMUtil tool and, in conjunction with
WDMUtil formatting scripts, populate a WDM file with time series data for the HSPF watershed
model.
The user interacts with the program to gather information necessary to process the imported data.
At most, 250 lines of data are initially read from the imported file, or all of it, if the file is
smaller. Once the user supplies required information, the entire file is processed and written one
line at a time to facilitate creation of extremely large data sets that cannot be read into memory
all at once.
Procedurally, the program:
• imports a text file containing time-series data,
• removes any extraneous header lines,
• identifies data contained in the file by column (date or date-time, time, variable, variable
type and variable units),
• converts variable units optionally, and
• writes the date or date-time and variable values as a time series to a text file.
Date and Date-Time Parsing:
A line in the data section of the imported file is parsed into fields. Field delimiters are commas or
white space. Fields are slotted into a spreadsheet-like grid and the user identifies a column for
the date. Because the date cannot be in multiple columns, no facility option is offered for
processing such. If the imported file has dates broken into separate white space or comma-
delimited fields for year, month, and day, the file must be edited before processing. Hourly and
finer-grained data has a time-stamp associated with the date which may be part of the date
column or in a separate column. The date and/or date-time field can be delimited (usually by "/"
or "-" characters separating month, day and year portions of the date) or they can be a string of
digits representing YYYYMMDDhhmmss or some variation thereof.
32
-------
Program limitations:
• Rows with a missing variable value will show a dummy value of -999 in place of missing
values.
• Rows with missing date or date-time values cannot be exported and will not display in
the export file.
• Separator characters (e.g., tab, comma, or pipe) in place of a missing or blank value may
cause inaccurate data to be displayed or exported.
• First rows of non-header data must contain a valid date or date-time value.
HDFT works with data that has date/date and time and single/multiple value columns, one
variable at a time. It has limited conversion and formatting capabilities since its intended use is
formatting large datasets for HSPF. This tutorial presents instructions for using HDFT with an
example instantaneous streamflow dataset from the USGS.
Unlike the web-version, the desktop version cannot download data. Nevertheless, it has a
capability the web-based version does not: it can read data directly from text files and write the
resulting formatted data to text files.
4.2. Data File Requirements
The program reads all data from a data file, displaying only 250 lines. It is able to parse many
date and time formats, a list of the data formats the tool can and cannot process are below.
• Data files should be in text format.
• Meta-data or header information should be less than 250 lines.
• Data section of the file must be comma- or white space-delimited.
• Dates must be a single string (e.g., MMDDYYYY, not "MM DD YYYY").
• The first row cannot have missing dates/values.
• Imported files cannot have footers or trailing empty lines after the data section at the end
of the file.
Examples of date and date-time formats the tool can process include:
• YYMMDD
• MMDDYYYY
• MM-DD-YY
• DD/MM/YY
• MMDDYYhhmm
• MMDDYYYY hhmm
• Jan-21-1999
• Date and time in separate columns
Examples of dates and date-time formats the tool cannot process include:
• YY MM DD
• Jan 21 199923:15
33
-------
• YYYY.MM.DD
Details of the data used to explain how the tool works:
Data used as examples for this tutorial were obtained from the Sopers Branch Station (USGS
01643395) near Hyattstown, Maryland.
The data has 619767 lines of record, is 23.9 MB in size, and spans from 2004/10/27 12:50 to
2009/09/30 23:55 at 5min intervals.
Raw data format: 20040127000000
Formats to: MM/DD/YYYY hh:mm
Figure 55 (Appendix B) illustrates the sample instantaneous data.
Note: After installation, HDFT users can follow the steps below to format and convert data for
input to the HSPF model.
Note: When opening large text files to modify entries or delete headers, users are encouraged to
use the Textpad software accessible at: http://www. textpad. com/.
4.3. Importing Data
Step 1: Download and install the program on your computer.
Step 2: Run the program by clicking on the executable file to open the Main Program window
[Figure 23].
ffl Time Series Converter Utility
Help
Import
Headeis
Exit
-
Figure 23: Main Desktop Version window
Step 3: Click the Import button on the Main Window to open a dialog box and select the data
file (01643395_20110706065138.rdb). Figure 24 illustrates importing raw data into the tool.
34
-------
Time Series Converter Utility
llJne
# retrieved: 2011-07-06 06:51:53CST
ft Data for the following station is contained in this file
n USGS 01S433S5 SOPER BRANCH AT HYATTSTOWN. MD
it This data file was retrieved from the USGS
# instantaneous data archive at
# http:A1da.water.usgs.gov
Opening C:\User5\ymonamou\Desktop\01643395_2011C File has 619767 lines
Figure 24: Import window showing the headers of the imported data file
Step 4: Click the Headers button to open the "Remove Header Lines" window [Figure 25] to
remove headers present in the raw data. Select the lines that contain text/headers by clicking on
the left-hand side of the window. Lines can be selected one at time or in multiples by clicking
first and last header lines, while holding the Shift key. If there are no headers, click Done.
a^ Remove Header Lines [ 1=1 | IBJ [^•t.^^
Line • U^
• ^retrieved: 201 1-07-06 06:51:53 CST
n
0 Data for the following station is contained in this file
g
# USGS 01643395 SOPER BRANCH AT HYATTSTOWN, MD
n
n This data file was retrieved from the USGS
0 instantaneous data archive at
n http://ida.water.usgs.gov
•
I
[ Delete Selected Rows
Done
Figure 25: Header window before headers were removed
Step 5: Click the Delete Selected Rows button to remove all selected headers/text [Figure 26]
and click Done. The raw data (with headers removed) is displayed as columns in the Time Series
Converter Utility window [Figure 27]. For long record data, HDFT users must wait a few
minutes until the data is parsed and the Identify button turns bold/active.
35
-------
Remove Header Lines
Delete Selected Rows Done
Figure 26: Header window after headers were removed
1 Time Series Converter Utility
Help
Import
Identify
Convert
Export
Exit
ColO
CoM
Col 2
Col 3
Col 4
Col 5
01643395
01643395
01643395
01643395
01643395
01643395
2CWC127CD1CCC
2CWC-127CD15G:
2CC4C127CC2ECC
2C04C127CD35CC
EST
EST
EST
EST
EST
EST
EST
EST
EST
2QM01270Q45QDIEST
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
Opening C:\Users\ymohamou\Deskklp\01643395_2011C File has 619767 lines
File appears to have 7 columns of data,
Figure 27: Data ready for column identification
Step 6: Click the Identify button on the Main Program window to identify columns of interest to
the HDFT user [Figure 27].
Step 7: Select the date and time column, Column 1, and the variable column, Column 5. Be sure
to tick Col 1 as one that contains both date and time [Figure 28],
Step 8: Select the Variable type as "Flow Rate" and the Variable Units as "cfs" in the Variable
type and Variable Units combo-boxes [Figure 28].
Step 9: Click the Test Selections button [Figure 28] which opens two dialog boxes to display the
date/date format and whether the selected variable column is numeric or not [Figure 29]. If the
selected date column and value column are correct, press OK on the Dialog box Identify Data
Columns window.
36
-------
Identify Data Columns
| Col 0 Cd 1 Col 2 Col 3
> • 20D401 270QQOQO EST 2
[meow I -M¥unn7f¥vwnn PCT t
4 \ m
Col 4 Col 5
1 2.3
i 11
>
Which column is Date? Which column is Time? Which column is Variable?
CoM CoM ^ BSE
T
[7] Column is DateTime Variable Type
Row Rate -r
Variable Units
Test Selections cfs
OK
T
Cancel
Figure 28: Column identification, variable and unit selection window
Identify Data Columns
[ CdO CoM
Col 2
t •20040127000000
In-iejrvK H-mrun-mnrvwnn
< I
Which column is Da
EST
PQT
m
CoM T
[7] Column is DateTime
[ Test Selections
Col3
2
2
Col 4 Col 5
1 2.3
1
71 >
Found Format String yyyyMMddHHm... l^"^H
Date convert:- ok:
20040127000000
|
OK I
OK
Cancel
Figure 29: Confirmation for the date and time format
Step 10: After pressing OK [Figure 29], the extracted date and value columns are displayed on
the Main Program window [Figure 30].
37
-------
1^ Time Series Converter Utility
Help
Import
Identify
Convert
Export
Opening C:\User5\ymohamou\De5ktop\01643395_2011C File has 619767 lines File appears to have? columns of data,
Figure 30: Columns selected for conversion and export
Step 11: If there is no need to convert units, proceed to the Export Section (Section 4.5). If the
user needs to convert units, information is available in Section 4.4.
4.4. Conversion of Variable values
Step 12: Click the Convert button on the Time Series Converter Utility window to open the
Converter units window [Figure 31] which displays the Variable name (e.g., Flow Rate) and
current units (e.g., cubic feet per second (cfs), selected in Step 8).
Step 13: Select the units from the "Convert variable to value with units:" combo-box; an
example is cms, cubic meter per second.
Note: The conversion options "Convert Volumetric Flow Rate to Linear Flow Rate" and
"Convert Volumetric Flow Rate to Daily Cumulative Flow " are accessible only if the selected
variable is Flow. For any variables not listed, use the "Convert Variable to Value with Units"
option.
Step 14: Click Done on the Converter Units Window to display the converted values as a new
column on the Main Program window [Figure 32].
38
-------
'-:" Convert Units
Variable identified as Flow Rate
having units of cfs
Convert variable to value with units:
-
Convert Volumetric Flow Rate to Linear Flow Rate
Drainage Area
sqmi v Drainage Area Units
(Output units are mm/day)
HH Convert Volumetric Flow Rate to Daily Cumulative Flow
cuft v Convert to Cumulative flow having units:
| Done"
Lancel
Figure 31: Unit conversion window
EB Time Series Converter Utility
Help
ESS
Date Flow Rale els
| 2.3
20040127000500 2.3
20040127001000 2.3
20040127001500 2.3
20040127002000 23
20040127002500 2.3
20040127003000 2.3
20040127003500
20040127004000 2.3
20040127004500 2.3
Flow Rate cms
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
2.3
23
2.3
0.0651287481
0.0651287481
0.0651287481
0.06512S7481
| Opening very big file C:\MOTAMARRI\JAVA PRACTISE\Time5t Showing 250 lines of big file. File appears to have 7 columns of dafa.
Figure 32: Converted flow values
As mentioned in Step 11, the resulting formatted date and variable values (including converted
values) can be exported as a text file. In the current example, the data is exported following
conversion of values.
Note: Only the date and converted values are exported after conversion, not the raw data.
39
-------
4.5. Exporting data
Step 15: Click the Export button to open Export Data in a Text File window [Figure 33] which
displays the formatted date and value columns to be exported.
Time Series Converter Utilily
Help
Dale
20040127000500 2.3
20040127001000 2.3
20040127001500 23
20040127002000 2.3
20040127002500 2.3
20040127003000 2.3
20040127003500 2.3
20,
20
20040127004000 2.3
20040127004500 2.3
Flow Rate cms
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
0.0651287481
Flow Rate cms
JO. 0651287481
01 /27/2004 00:05 0.0651287481
01/27/2004 00:10 0.0651287481
01 /27/2004 00:15 0.0651287481
01 /27/2004 00:20 0.0651287481
01 /27/2004 00:25 0.0651287481
j Opening very big file C:\MOTAMARRI\JAVA PRACTISE\Time5( Showing 250 lines of big file. File appears to have 7 columns of data.
Figure 33: Formatted data window
Step 16: Click GO [Figure 33] to save the exported file as a text file as shown in Figure 34.
Note: HDFT can only export data in a single format compatible with the HSPF model —
MM/DD/YYYY-with/without time.
Save As
(^J(^J !• Desktop
•*••**• Search Desktop
file name
Save as type: Text File (*.txt)
•*• Browse Folders
| Save
Figure 34: Name and Save the formatted data file
40
-------
Chapter 5: Exporting HDFT-Formatted Data to WDMutil for HSPF
Model Use
This section provides additional information on handling various data formats, using other
resources such as the WDMUtil tool of BASINS software. For more information on the
WDMUtil tool, WDM Scripts and BASINS software, the user is referred to the following links:
WDMutil tool tutorial
http://water.epa.gov/scitech/datait/models/basins/upload/Exercise-3-WDMUtil.pdf
BASINS Software
http://water.epa. gov/scitech/datait/model s/basins/index. cfm
BASINS - Tutorials, Training and Lectures etc
http://water.epa.gov/scitech/datait/models/basins/training.cfm
Note: Chapter 5 assumes the user has a working knowledge of WDMUtil. Information about
WDMUtil can be found in BASINS training exercise #3.
Data exported from HDFT should not have headers, as WDM files require input data without
headers. To import HDFT-formatted data to a WDM file, users can use three formatting scripts
that work with one uniform date format and three time formats. The uniform date format
(MM\DD\YYYY) is made possible by using HDFT to pre-process data. HDFT exported
datafiles can have any of the following formats:
Date and time Value
Daily MM\DD\YYYY XX.XX
Hourly MM\DD\YYYY hh XX.XX
Hourly MM\DD\YYYY hh:00 XX.XX
Sub-hourly MM\DD\YYYY hh:mm XX.XX
There are separate scripts for each of the temporal resolutions and guidelines on using each .
41
-------
Step 1: Within the WDMUtil program, click on File to create a new WDM or open an existing
one [Figure 35].
Tools Scenarios Locations Constituents Time Series Help
-Time Series - 0 of 0 available time series in list (0 not on WDM file); 0 selected.
AM I Nonel
-Dates-
No Dates are available until Timeseries are Selected
r Tools-
Figure 35: WDMUtil Window
Step 2: Begin importing data using formatting scripts that work with FIFDT formatted data
[Figure 3 6].
42
-------
WDMUtil: NEW WDM
New
Open
Scenarios Location; Constituents Tim; Series Help
Locations Constituents
Close
Exit
None
Oof 0
All None
Oof 0
All None
Time Geriei - 0 ol 0 available time iinci in Kit [0 not on WDM fie); 0 selected.
All None
Dates
No Dates are available until Timeseries are Selected
Tools
II
^
111'
222
Figure 36: An empty WDM file
5.1. Importing daily time series data to WDM files using a new daily data
formatting script.
Note: A new script for importing daily data currently exists for BASINS, however, it is not
provided in the default BASINS software package. This new script is included with the HDFT
Tool.
Note: HDFT only works with one variable at a time. For example, when importing precipitation
and temperature data, users must first import/format/export precipitation data, and then
temperature data.
Step 3: Import HDFT exported DAILY data from the project folder [Figure 37]
43
-------
C DaiIy_DATA.txt -Note... (_J(nj(X
File Edit Format View Help
10/01/1950
10/02/19SQ
10/03/1950
10/04/1950
10/05/1950
10/06/1950
10/07/1950
10/03/1950
10/09/1950
10/10/1950
10/11/1950
10/12/19EO
Z.O
2.1
11.0
2.1
1.8
1.8
14.0
£.£
3.2
2.1
2.0
1.7
Figure 37: HDFT exported daily flow data
Find DAILY_MDY_SCRIPT.ws script and click RUN [Figure 38]
Script Selection for importing C:\New Folder\New Folder\Daily_DATA.txt
Description
| Script File
Blank Script
"HDFT Formatted Daily Data"
CANew Folder\New Folder\DAILY MDY SCRIPT.ws
Run
Figure 38: Script for FtDFT exported daily data
44
-------
Step 5: Click the Write time-series to WDM button
data series number (DSN) [Figure 39].
(visible in Figure 36) and assign a
WDMUtil: NEW WDM
.. n x
Flip. Tm-iU-
Scenarios Locations Constituents Time Series Help
Scenarios Locations Constituents
0 of 1 All I None I 0 of 0 All I None I 0 of 0 All
ScriptRead
Tin
WDM Data Set Add
New data-set number 10 successfully stored on WDM filed\New Folder\rJew Folder\NEW_WDM.wdm.
& Write to \
OK
Specify Output Data-set Number(s);
Select/Enter Scenario, Location, Constituent as needed;
Data-set attributes may be updated if needed
Click Write button to store data on WDM file.
<"" Use common period for all data sets, as defined on main form
* Use full period for each data set
DSN/ID| Output DSN
i
10
Scenario
Observed
| Locatior| Constituent tt Attributes |Attr. Space| tt Data Pointer; | Time Grouc| BaseYeai
Athens
Flow
30
100
300
Years
1940
Cancel
Figure 39: Write daily flow data to WDM file Script for HDFT exported daily data
45
-------
Step 6: Click OK [Figure 39] to complete the DSN creation process [Figure 40]
" WDMUtil: NEW_WDM t_JOl~J
File Tools Scenarios Locations Constituents Time Series Help
0 of 1 All | Hone| 0 of 1 All | Hone |
1 Observed Athens
Time Series - 1 of 1 available time series in list (0 not on W
+|-|\| * 4 + ±| A «|
Type File DSN Scenario | Location Constituent | Start
0 of 1 All | Hone
Flow
IIU Filctl- 1 D Etlckrn tt*A
All | None|
| SJDay | End | EJDay |
I WDM | NEW_WDM 1 1 0 | Observed [Athens | Flow 1 1 950/1 0/1 1 33555 1 1 951 /9/30 1 33920 1
Reset | Start ^-j End
Current ^950 plO^ to filsT^^
Common ||1950|10 1 1 to|l95l|9|30 Native
\\ m1^ .h, rf^S
y 22^1 iilh o
/£-* xv/ *
-------
5.2. Importing hourly time series data to a WDM file using new hourly data scripts
Note: Hourly formatting scripts are not provided in the BASINS software package, however,
they are included with the HDFT tool.
Two hourly formatting scripts are available for use. One script imports hourly data that has
minutes represented as two zeros while the other script imports data that has only hourly format
(without minutes) [Figure 41].
Hour_With_Minutes.txt - Notepad
File Edit Format View Help
Hour Without Minutes.txt - Note.
File Edit Format View Help
- n x
JL 0/01/2 000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/01/2000
10/02/2000
10/02/2000
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
00
00
00:00
01:00
22;
23:
0. 5
10/01/2000 00 0. 5
10/01/2000 01 0.1
10/01/2000 02 0.1
10/01/2000 03 0.1
10/01/2000 04 0.1
10/01/2000 05 0.1
10/01/2000 06 0.1
10/01/2000 07 0.1
10/01/2000 08 0.1
10/01/2000 09 0.1
10/01/2000 10 0.1
10/01/2000 11 0.1
10/01/2000 12 0.1
10/01/2000 13 0.1
10/01/2000 14 0.1
10/01/2000 15 0.1
10/01/2000 16 0.1
10/01/2000 17 0.1
10/01/2000 18 0.1
10/01/2000 19 0.1
10/01/2000 20 0.1
10/01/2000 21 0.5
10/01/2000 22 0.1
110/01/2000 23 0.1
110/02/2000 00 0.1
llG/02/2000 01 0.5
Al
Figure 41: Examples of two different hourly formats
Note: Of these two hourly formats, we recommend the format which has minutes represented as
zeros. Users can format their data to this hourly format with the HDFT tool.
47
-------
Step 7: Import the hourly precipitation data previously exported to the project folder by HDFT
[Figure 42].
P HOURLY
_pREc.txt -N... [_J(n][x)
File Edit Format
01/01/1930
01/01/1980
01/01/1980
01/01/1980
01/01/1980
01/01/1980
01/01/1980
01/01/1980
01/01/1980
01/01/1930
01/01/1930
01/01/1930
01/01/1930
01/01/1980
01/01/1930
01/01/1980
00:
01:
02:
03:
04:
OS:
06:
07:
08:
09:
10:
11:
12:
13:
14:
1£:
View Help
00
00
00
00
00
00
00
00
00
00
00
00
DO
00
00
00
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0. 000
0.000 v
Figure 42: FtDFT exported hourly precipitation data
Step 8: As this is hourly data with minutes represented as zeroes, select the correct hourly data
script [Figure 43] and follow Steps 5 and 6 to write the hourly precipitation data to the WDM file
[Figure 44].
i Tools Scenarios Locations Constituents Time Series Help
Scenarios Locations Constituents
OofO I All I None I 0 of 0 All I None I 0 of 0 All I None
Blank Script
"HDFT Formatted Daily Data"
"Formatted Hourly Data Without Minutes"
C: \B asins_T ools\H D FT \scriplsW ewS criptsT est\H ourly_ Find
Dates
No Dates are available until Timeseries are Selected
Tools
Figure 43: Hourly script for FtDFT formatted data
48
-------
WDMUtil: NEW WDM
File Tools Scenarios Locations Constituents Time Series Help
Scenarios Locations
OoM All I Nonel 0 of 1 All I None I
I- ](n]fx]
Observed
Constituents
0 of 2 All
Flow
Free
Time Series - 2 of 2 available time series in list (0 not on WDM file); 1 selected.
Type File
| DSN | Scenario | Location | Constituent | Start _ | SJDay | End _ | EJDay
WDM NEW WDM 10 Observed Athens Flow
WDM NEW WDM Observed Athens Free
1950/10/1 33555 1951/9/30 33920
1980/1 /1 44239 1982/12/31 45335
Dates
Reset | Start
End
TSlep,Unils
Current [1980 [TIT to
Common 11980 |T |T to [l982[T2|3T (Native
Tools
Figure 44: Hourly precipitation data added to the WDM file
49
-------
5.3. Importing sub-hourly time series data to a WDM file, using a new sub-hourly
data script
Note: A Sub-hourly formatting script is not provided in the default BASINS software package,
however, this script is included with the tool.
Step 9: Import sub-hourly HDFT exported data from the project folder [Figure 45]
C FLOW_shift_DATA33.t... Q(
File Edit Format View Help
10/01/2004 00:00 0.5
10/01/2004 00:05 0.4
10/01/2004 00:10 0.
10/01/2004 00:15 0.
10/01/2004 00:20 0.
10/01/2004 00:25 0.
10/01/2004 00:30 0.
10/01/2004 00:35 0.4
10/01/2004 00:40 0.4
10/01/2004 00:45 0.4
10/01/2004 00:50 0.
10/01/2004 00:55 0.
10/01/2004 01:00 0.
10/01/2004 01:05 0.
10/01/2004 01:10 0.
10/01/2004 01:15 0.4
IB
*l
• — '
v
Figure 45: HDFT exported sub-hourly (5 minute interval) flow data
50
-------
Step 10: Select the sub-hourly data script [Figure 46] and follow Steps 5 and 6 to write the sub-
hourly data to the WDM file.
e Tools 5 ' "uents Time Ser
Scenarios Locations
ofO I All I Hone I 0 of 0 All I None I 0 of 0
Blank Script
"HDFT Formatted Daily Data"
"HDFT Formatted Hourly Data With Minutes"
"Formatted Hourly Data Without Minutes"
["HDFT Formatted'SuD-hourjt Data"'
C:\Basins_T ools\H D FT \scripts\DAI LY_M DY_S CRIP"
C:\Basins_Tools\HDFT\scripts\NewS criptsT est\H oui
C:\Basins_Tools\HD FT \scripts\N ewS criptsT est\H oui
C:\Basins Tools\HDFT\scripts\SUB HOURLY MM
Edit
Find.
Forget
Debug
Cancel
Dates
Tools
No Dates are available until Timeseries are Selected
Figure 46: Sub-hourly script for FtDFT formatted data
Note: The sub-hourly data shifts back one time step when imported into the WDM program. For
example, a date starting at midnight will shift back to the day before, i.e., 2004-10-01 00:00 will
shift to 2003-09-30 23:55. Step 11 corrects this problem.
Step 11: Click on List/Edit Time Series Button I1—-1 (visible in Figure 44) and under the File
tab click New Time Series. Once the New Time Series window is open, specify the sub-hourly
time step of the data on the Change Interval Tab [Figure 47].
51
-------
New Time Series
Change Interval | Add/Remove Dates | Shift Dates) Math | Table Filter Values 1
Base on existing time series
31
Time Step
3 Year End |~~
Aggregation JAver/Same _^J
New Properties
Scenario ScriptRead Location |Athens
ID fl2
Units
Constituent [Flow
Description |C:\Basins_Tools\HDFT\HDFT_Document\F
] Save in |CABasins_Tools\HDFT\HDFT_Documej.L|
Ok
Cancel
Figure 47: Change interval tab of New Time Series window
52
-------
Step 12: On the Shift Dates tab of the New Time Series window, enter the correct starting
date/time next into the New Start entry boxes. Also specify the correct Location, Constituent, ID
and WDM file to save the new time series in and hit OK [Figure 48].
New Time Series
Change Interval | Add/Remove Dates Shift Dates Math 1 Table Filter Values 1
Base on existing time series
|ScriptRead asdf fdsa 82 C:\Basins_ToolsMHDFT\HDFT_Document\NEW_WDM.
Old Start [2004 f9 [30 [23 [55
New Properties
Scenario ScriptRead Location [Athens Constituent (Flow
ID fl2~
Units
Description | C:\Basins_Tools\H D FT\HDFT_Docurnent\F
T I Save in
FT \HDFT Docurnent\N EV*' WDM.wdrr
Ok
Cancel
Figure 48: Change starting date for New Time Series
Step 13: Add the new time series to the WDM file by clicking the Add to Time-Series list button
i (visible in Figure 44).
Step 14: End of the WDM data entry tutorial [Figure 49]
53
-------
WDMUtil: NEW WDM
File Tools Scenarios Locations Constituents Time Series Help
Scenarios Locations
Oof 2 All I Nonel 0 of 1 All I None I
Constituents
Oof 2
None
Observed
ScriptRead
Athens
Flow
Free
Time Series - 3 of 3 available time series in list (0 not on WDM file); 1 selected.
Type | File
I DSN I Scenario | Location | Constituent | Start | SJDay | End
WDM NEW_WDM 10
WDM NEW WDM 11
Observed Athens Flow
Observed Athens Free
WDM NEW WDM 12 Observed Athens Flow
1950/10/1 33555 1951/9/30 33920
1980/1/1 44239 1982/12/31 45335
2004/10/1 53279 2004/12/31 53371
Dates
Reset I Start
End
TSlep,Units
Current [2004[TofT to
Common ||2004[lO |T to |2004fl2f3T [Native
Tools
I:*::::::
111
222
1
Figure 49: A WDM file containting three data series with daily, hourly, and sub-
hourly data resolutions
5.4. Scripts available in BASINS for NCDC data formats
Hourly precipitation data - NCDC format (TD-3240)
Hourly precipitation data in the format below works with the "HPCP-NCDC-ARCH" script
already available in BASINS.
Hourly precipitation data obtained from: http://gis.ncdc. noaa.gov/map/cdo/?thm=themePrecip
Sample:
HPD304174QOHPCPHI19830100010020100000000 2500000000
HPD304174QOHPCPHI1983010Q1000917QOQ00001 1800000001
HPD3Q417400HPCPHI19830100110040100000001 0200000001
HPD304174QOHPCPHI1983010Q1400224000Q0002 2500000002
HPD3Q41740QHPCPHI198301001S0190100000001 0200000001
HPD304174QQHPCPHI1983010Q1600602QQOQOQ01 0400000001
HPD304174QOHPCPHI19830100230100200000002 0300000001
HPD3Q417400HPCPHI1983010Q240080100000001 0200000004
HPD3Q4174QOHPCPHI19830100300041700000001 1800000001
HPD3041740QHPCFHI1983010031005Q100000001 0200000002
1900000002 2000000002 2100000004 2200000002 2300000009
0300000002 2500000004
0300000003 0400000002 0500000002 0600000001 0700000002
0500000001 1800000002 2000000001 2500000006
0900000004 1000000005 1100000002 1400000003 1500000004
0300000003 0400000003 0500000002 0600000001 0700000001
2100000001 2500000003
0300000002 0400000001 2500000006
54
-------
Hourly precipitation data - NCDC format (TD-3240) - another Hourly data format
A. With Station Name:
An existing WDM script -"HPCP_NCDC_OL" in BASINS can format NCDC TD-3240
data containing Station Name information.
Sample:
COOPID,STATION NAME
,CD,ELEM,ON,YEAR,MO,DA,TIME,HOUR01,F,F,TIME,HOURQ2,F,F,TIME,HOHRQ3,F,F,TIME
f
180015,
180015,
180015,
180015,
180015,
180015,
180015,
180015,
ABERDEEN
ABERDEEN
ABERDEEN
ABERDEEN
ABERDEEN
ABERDEEN
ABERDEEN
ABERDEEN
PHILLIPS
EHIllIFS
PHILLIPS
PHILLIPS
PHILLIPS
PHILLIPS
PHILLIPS
PHILLIPS
FI
FI
FI
FI
FI
FI
FI
FI
,QO,HPCP,HT,1984,Q1,
,OQ,HPCP,HT,1984,01,
,00,HPCP,HT,1984,01,
,QO,HPCP,HT,1984,01,
,QO,HPCP,HT,1984,Q1,
,QO,HPCP,HT,1984,Q1,
,QO,HPCP,HT,1984,01,
,OQ,HPCP,HT,1984,Q1,
01,0100,
04,0100,
10,0100,
14,0100,
18,0100,
19,0100,
24,0100,
30,0100,
f r
00000,g,
00000, ,
00000, ,
00000, ,
00000, ,
00000, ,
00000, ,
00000, ,
r f
,0200,
,0200,
,0200,
,0200,
,0200,
,0200,
,0200,
,0200,
t
00000,
00000,
00000,
00000,
00000,
00010,
00000,
BOOOO,
ft f
, ,0300,
, ,0300,
, ,0300,
, ,0300,
, ,0300,
, ,0300,
, ,0300,
, ,0300,
t
00000,
30000,
00000,
00000,
00000,
00000,
00000,
00000,
I I
, ,0400
, ,0400
, ,0400
, ,0400
, ,0400
, ,0400
, ,0400
, ,0400
B. Without Station name:
A modified version of the WDM script-"HPCP_NCDC_OL" in BASINS, renamed
"HPCP_NCDC_OL_woStationName," can format NCDC TD-3240 data that contains no
Station Name information. The modified script is supplied by HDFT.
NCDC First Order Summary of the Day (TD-3210) data
The WDM script "SOD_OL" in BASINS can format this data format (TD-3210).
Sample:
OQQ04725NY
00004725NY
OOQ04725NY
OQ004725NY
00004725NY
00004725NY
00004725NY
00004725NY
00004725NY
Q0004725NY
3INGHAMTON
3INGHMION
3INGHAMTON
3INGHAMTON
3ING.HAMION
3INGHAMTON
3INGHAMION
3INGHAMION
3INGHAMTON
3INGHAMTON
LINK
LINK
LINK
LINK
LINK
LINK
LINK
LINK
LINK
LINK
FLD
FLD
FLD
FLD
FLD
FLD
FLD
FLD
FLD
FLD
1990
1990
1990
1990
1990
1990
1990
1990
1990
1990
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
14213N07559W
4877AWND24
4877CLDG24
4877DPNT24
4877DPTP24
4877HTDG24
4877MNRH24
4S77MXRH24
4877PKGS24
4877PRCP24
4877PRES24
16924
024
824
20024
3524
6124
9624
8803724
124
2802024
11624
024
624
16424
3724
4824
7824
7702624
024
2843024
8524
024
924
20724
3424
5124
7524
5501424
024
2844024
12024 9224 10024 9
024 024 024
1624 724 624
30324 20224 19424 20
2724 3624 3724 3
4124 5224 5224 5
9324 8824 8824 9
7702524 8803224 8802624 6602
124 024 024
2813024 2833024 2824024 2821
55
-------
Appendix-A: Hydrological and Meteorological Variables and Unit
Conversions
Table 2: Variable unit conversion options and conversion values used in HDFT
Variable
Flow
Units
From
Cubic meters per
sec (cms)
Acre-feet per day
(acre -ft/day)
Cubic feet per sec
(cfs)
Acre-feet per day
(acre -ft/day)
Cubic feet per sec
(cfs)
Cubic meters per
sec (cms)
Conversion
(Multiply by)
35.314
0.504
0.0283
0.0143
1.983
70.029
To
Cubic feet per
sec (cfs)
Cubic meters
per sec (cms)
Acre-feet per
day (acre-
ft/day)
Cumulative flow
Cubic meters per
sec (cms)
Acre-feet per day
(acre -ft/day)
Cubic feet per sec
(cfs)
Acre-feet per day
(acre -ft/day)
Cubic feet per sec
(cfs)
Cubic meters per
sec (cms)
3051187.205
43570.348
2446.576
1233.775
1.983
70.029
Cubic feet
Cubic meters
Acre-feet
Millimeters per day
Area : Square
miles
Cubic feet per sec
(cfs)
Cubic meters per
sec (cms)
Acre-feet per day
(acre -ft/day)
Area : Acres
Cubic feet per sec
(cfs)
Cubic meters per
sec (cms)
Acre-feet per day
(acre -ft/day)
(0.945/Area)
(33.6/Area)
(0.476/Area)
(604/Area)
(21349.67/Area)
(304.8/Area)
mm/day
mm/day
HSPF
default
56
-------
Concentration
Load
Precipitation
Temperature
Solar Radiation
^1r\iirl (~*r\\7t*r
Area : Square
Kilometers
Cubic feet per sec
(cfs)
Cubic meters per
sec (cms)
Acre-feet per day
(acre -ft/day)
Micrograms per
liter (ng/1)
Milligrams per
liter (mg/L)
Kilograms per
day (Kg/day)
Tonnes per day
(Tonnes/day)
Kilogram per
second (Kg/sec)
Pounds/day
(Lb/day)
Tonnes per day
(Tonnes/day)
Kilograms per
second (Kg/sec)
Pounds/day
(Lb/day)
Kilograms per
day (Kg/day)
Kilograms per
second (Kg/sec)
Pounds/day
(Lb/day)
Kilograms per
day (Kg/day)
Tonnes per day
(Tonnes/day)
Inches per hour
(Inches/hr)
Millimeters per
hour (mm/hr)
Centigrade (°C)
Fahrenheit (°F)
Watts per square
meter (Watt/m2)
OVtn
(2.447/Area)
(86.4/Area)
(1.233/Area)
0.001
1000
2.208
2204.622
190728.477
0.453
1000
86400
0.000045
0.001
86.4
5.243xlO'6
0.0000115
0.1157
25.4
0.0393
[9*(°C)/5]+32
[5*(°C)/9]-32
0.0858
mm/day
Milligrams per
liter (mg/L)
Micrograms
per liter (|ig/l)
Pounds/day
(Lb/day)
Kilograms per
day (Kg/day)
Tonnes per
day
(Tonnes/day)
Kilograms per
second
(Kg/sec)
Millimeters
per hour
(mm/hr)
Inches per
hour
(Inches/hr)
Fahrenheit (°F)
Centigrade
(°C)
Langley per
hour
(Langley/hr)
Oktn
-------
Dew Point
Temperature
Wind Speed
Soil Moisture
Centigrade (°C)
Fahrenheit (°F)
Meters per
second (mt/sec)
Miles per hour
(miles/hr)
Inches per hour
(Inches/hr)
Millimeters per
hour (mm/hr)
[9*(°C)/5]+32
[5*(°C)/9]-32
2.237
0.447
25.4
0.0394
Fahrenheit (°F)
Centigrade
(°C)
Miles per hour
(miles/hr)
Meter per
second (m/sec)
Millimeters
per hour
(mm/hr)
Inches per
hour
(Inches/hr)
Fahrenheit (°F)
Miles per hour
(miles/hr)
Inches per hour
(Inches/hr)
58
-------
Appendix-B: Examples of Data Formats and Data Types
usgs sediment data sample.cfm - Notepad
File Edit Format View Help
S* United States Geological Survey
* station Name: MONOCACY R AT JUG BRIDGE NR FREDERICK, MD
* Station ID: 01643000
* Basin Area (sq. mi.): 817
t Latitude: 392316
* Longitude: 0772248
¥ -99 indicates missing data
;tation_id
31643000
)1 64 3 000
)1 64 3000
)1643000
)1643000
)1643000
)1643000
51643000
)1 64 3 000
water_year
Ol-Oct-1960
02-0ct-1960
03-0ct-1960
04-0ct-1960
05-0ct-1960
06-0ct-1960
07-0ct-1960
08-0ct-1960
09-0ct-1960
date
1961
1961
1961
1961
1961
1961
1961
1961
1961
Q_(cfs) Q_(cms) C_(mg/L)
^^3~
139
131
131
131
137
145
137
137
4.
3.
3.
3.
3.
3.
4.
3.
3.
36
94
71
71
71
88
11
88
88
5
2
2
2
2
1
1
1
1
^^^
1
1
1
1
0
o
0
0
X
L_(ton/day)
~o"7
0.
0.
0.
0.
0
0
0
0
u^^
91
91
91
91
V/
Figure 50: Sample data to be pasted inside the tool (excluding headers)
C Untitled - Notepad
File Edit Format View Help
2000/01/01
2000/01/02
2000/01/03
2000/01/04
2000/01/05
01
02
03
04
05
00
00
00
00
00
1
2
3
4
5
2000/01/01
2000/01/02
OR 2000/01/03
2000/01/04
2000/01/05
1
2
3
4
5
Figure 51: Column format -Sample data
C usgs nwis format sample.txt - Notepad
File Edit Format View Help
aqency cd site
•><; 1S=: 1 fid
LTSla b
USGS
USGS
USGS
USGS
USGS
USGS
USGS
USGS
USGS
USGS
USGS
UXDHii DUU
01582500
01582500
01582500
01582500
01582500
01582500
01582500
01582500
01582500
01582500
01582500
no datetime
14n 1 fK
^UJ-U-U^-^J.
2010-02-22
2010-02-23
2010-02-24
2010-02-25
2010-02-26
2010-02-27
2010-02-28
2010-03-01
2010-03-02
2010-03-03
2010-03-04
01_00060_00003 01_00060_00003_cd 1
Z5T
280
383
368
382
364
324
313
319
331
352
370
A
A
A
A
A
A
A
A
A
A
A
A
X
J-
s
\/
^f
Figure 52: USGS NWIS- Sample Data with a header
59
-------
I usgs sediment data sample.cfm - I
File Edit Format View Help
¥ united states Geologicalsurvey^
* station Name: MONOCACY R AT JUG BRIDGE NR FREDERICK, MD
* station ID: 01643000
* Basin Area (sq. mi.): 817
* Latitude: 392316
f Longitude: 0772248
* -99 indicates missing data
5tation_id
)1 64 3000
11643000
31643000
)1 64 3000
31643000
)1 64 3000
J1643000
J1643000
)1643000
water_year
01-oct-1960
02-0ct-1960
03-0ct-1960
04-0ct-1960
05-0ct-1960
06-0ct-1960
07-0ct-1960
08-0ct-1960
09-0ct-1960
date
1961
1961
1961
1961
1961
1961
1961
1961
1961
Q_(cfs) Q_(cms) c_(mg/L)
139
131
131
131
137
145
137
137
4. 36
3. 94
3.71
3.71
3.71
3. 88
4.11
3.88
3.88
5
2
2
2
2
1
1
1
1
1
1
1
1
0
0
0
0
X
I (ton/day)
0. 91
0.91
0.91
0.91
0 f
o ^r
o ^^^^
0
Figure 53: USGS Sediment -Sample data with a header
0 ASOS 1 min sample - Notepad
File Edit Format View Help
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
14737KABE
ABE20Q2Q10111321632
ABE2 002 0101113 3163 3
ABE2Q02Q1Q111341634
ABE2Q02Q1Q111351635
ABE2Q02Q10111361636
ABE2 002 01 01113 7163 7
ABE2 002 0101113 8163 8
ABE2QQ2Q1Q111391639
ABE2 002 0101114 0164 0
ABE2 002 010111411641
ABE2 002 0101114 2164 2
ABE2002010111431643
ABE2002010111441644
ABE2 002 0101114 5164 5
ABE 2 002 0101114 6164 6
0.
0.
0.
0.
0.
0.
0.
0.
0,
0.
0.
0.
0,
0.
0.
062
068
065
070
070
071
063
064
066
069
065
066
069
070
063
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
300
291
294
306
311
311
309
321
320
298
287
272
261
270
268
9
8
9
10
9
9
10
10
9
7
6
7
8
8
6
301
281
305
305
315
301
287
333
317
286
256
266
257
283
262
9
11
11
12
10
11
13
11
9
9
7
8
9
10
6
Figure 54: ASOS -Sample data with no header
60
-------
C 01643395_20110307105502.rdb -Notepad
File Edit Format View Help
retrieved:
C5T
t Data for the following station is contained in this file
USGS 01643395 SOPER BRANCH AT HYATTSTOWN, MD
It
t This data file was retrieved from the USGS
f instantaneous data archive at
http://ida.water, usgs. gov
It
|t This file consists of tab-separated columns of the
following fields.
X
column
column definition
site_no USGS site identification number
t date_time date and time in format (YYYYMMDDhhmmss)
t tz_cd time zone
t dd internal USGS sensor designation C'data descriptor'')
accuracy_cd accuracy code
;ite_no date_time
L5N 14N 6S
tz_cd
2N
dd
5S
accuracy_cd
16N IS
value
IS
prec
remark
)1643395
J1643395
0.643395
H.643395
)1643395
J1643395
U.643395
0.643395
20040127000000
20040127000500
20040127001000
20040127001500
20040127002000
20040127002500
20040127003000
20040127003500
EST
EST
EST
EST
EST
EST
EST
EST
1
1
1
1
1
1
1
1
2. 3
2. 3
2.3
2.3
2. 3
2. 3
2.3
2.3
Date col = 2 Value col = 6
Figure 55: USGS instantaneous time series data - flow with a header
P- Colham io.Hd.txt - Notepad
File Edit Format View Help
A watklnsvlIle 5 SSE [03F7BEJ:
Dote col = 1
Value col= 14
X
YYYHMDDHH^lr5_l,T5_;,T5_3,T5_4,T5_5,T5_6,T5_7,T5_8,T5_9,T5_10,T5_ll,T5_12jT_OFFICIAL]T_MIN,T_MAX, P5_l, P5_Z , P5_3 , P5_4 ,
, ^5-1 , M'-U, P5_9, P5J.O, P5_ll, P5J.2, P_OFFICIAL, SOLARAD, SOLRAD_MN, SOLRAD_MX, bk_b I LI, WlNuSPD, WS_MAX, WS_STD, SUR_TEMP, ST_MK
r_STD,RH05,RHlO,RHl5,RH20,RH25,RH30.RH35,RH40,RH45,RH50,RH55,RH60,RH_STD,TRH05,TRH10,TRH15,TRH20,TRH25,TRH30,TRH35,TR
,TRH50,TRH55,TRH60,TRH_HIN,TRH_MAX,TRH_STD,SM1005,SM2005,SH3005,SM1010,SM201Q,SM3Q10,SM102Q,SM2020,SM3020,SM1050,SM2
0,SM1100,SM2100,SM3100,ST1005,ST2005,ST3005,ST1010,ST2010,ST3010,ST1020,ST2020,ST3020,ST1Q50,ST2050,ST3050,ST1100,ST
LOO
2009110101,10. 6,10. 5,10.4,10.3,10.2,10.1,9.9,9.9,9.8,9.7,9.6,9.5,10.0,9. 5,10.6,0.0,0.0,0.0,0.0,0.0,0.0,0.
3,0.0,0.0,0.0,0.0,0.0,2.44, 5.1,0. 55,9.72,8.93,10.9,0.6,92.3,92.2,92.4,92.0,92.1,92.2,92. 5,92.2,92. 5,92.6,
10.6,10. 5,10.38,10.29,10.14,10.03,9.97,9.85,9.81,9.68,9.6,9.6,10.66,0.35,9.95,12.14,12.32,10.61,9. 51,11.
42,12.02,13.97,26.48,23.7,23.41,17. 55,17.0,17.27,17.75,17.92,18.0,18.4,18.2,18.25,18.6,18.6,18.45,19.29,
2009110102,9.4,9.4,9.4,9.3,9.4,9.2,9.2,9.2,9.2,9.1,9.1,9.1,9.3,9.1,9. 5,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.
D, 0.0, 0.0, 0.0, 2. 6,4.42,0. 5,8.79,8. 5,9.04,0.13,93.3,93.2,93.2,93.4,93.3,93. 5,93. 5,93. 5,93.4,93.4,93. 5,93. 5
9.39,9.43,9.28,9.31,9.24,9.26,9.17,9.17,9.1,9.1,9. 59,0.14,9.73,12.03,12.23,10. 58,9. 52,11.61,9.42,9.33,9.
26.38,23.71,23.46,17.35,16.77,17.1,17.7,17.65,17.9,18.2,18.0,18.07,18.6,18.6,18. 59,19.29,19.44,18.84
3009110103, 9.1,9. 0,8. 9, 9. 0,8. 9, 8. 8, 8. 7, 8. 7, 8. 6, 8. 6, 8. 6, 8. 6, 8. 8, 8. 5,9.1,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.
3,0.0,0.0,0.0,2.48,4.17,0. 52,8.27,7.92,8.64,0.2,93.2,93.4,93.3,93.1,93.1,93.4,93. 5,93.2,93.3,93.2,93.0,93
4,9.05,8.94,8.87,8.79,8.82,8.67,8.71,8.65,8.63,8.62,9.16,0.18,9.68,11.83,12.2,10. 59,9. 54,11.65,9.48,9.25
32.26.37.23.94.23.47.17.15.16. 5 5.16. 9.17. 4.17. 4.17. 7 5.18. 2.17. 97.17. 97. IS. 6. IS. 6. IS. 55.19.29.19. 5. IS. 87
0,0.0,0.0,0.
92.7,92.9,0.
51,9.37,9.21
19.49,18.9
0,0.0,0.0,0
,0.18,9. 52/9
75,16.39,^2.
0,0.
.0,0.2,9.14,
9.83,16.36,1
Figure 56: NCDC - U.S. Climate Reference Network data (hourly) with a header
61
-------
~j\ Unfilled - Notepad
File Edit Format View Help
# retrieved: 2011-07-06 09:23:01 CST
#
# Data for the following station is
a
contained in
this
file
n
# USGS 01643395 SOPER BRANCH AT HYATTSTOWN , MD
#
# This data file was retrieved from
# instantaneous data archive at
# http://ida.water.usgs.gov
#
#
#
site no date time tz cd dd
15N 14N 6S 2N 55
01643395 20040127000000 EST
01643395 20040127000500 EST
01643395 20040127001000 EST
01643395 20040127001500 EST
01643395 20040127002000 EST
01643395 20040127002500 EST
01643395 20040127003000 EST
01643395 20040127003500 EST
01643395 20040127004000 EST
01643395 20040127004500 EST
01643395 20040127005000 EST
the uses
accuracy cd
16N
2
2
2
2
2
2
2
2
2
2
2
is
1
1
1
1
1
1
1
1
1
1
1
value
is
2.3
2. 3
2.3
2. 3
2.3
2. 3
2.3
2. 3
2.3
2.3
2.3
precision remark
2
2
2
2
2
2
2
2
2
2
2
Figure 57: Sample streamflow data (with no date and time separators) - USGS Instantaneous
Data Archive format
P NCDC Global Hourly (Date and Time in separete columns) - Notepad | _ || G || X |
File Edit
Format
View Help
(identification
USAF
999999
999999
999999
999999
999999
999999
999999
999999
999999
NCDC
63850
63850
63850
63850
63850
63850
63850
63850
63850
Date
20040501
20040501
20040501
20040501
20040501
20040501
20040501
20040501
20040501
HrMn
0000
0100
0200
0300
0400
0500
0600
0700
0800
I
I
I
I
I
I
I
I
I
I
SOLAR
Type SlRad Q
CRN05
CRN05
CRN05
CRN05
CRN05
CRN05
CRN05
CRN05
CRN05
41.
0.
0.
0.
0.
0.
0.
0.
0.
7 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
Q
0
0
0
0
0
0
0
0
0
SrMin
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9
9
9
9
9
9
9
9
9
Q Q
9 0
9 0
9 0
9 0
9 0
9 0
9 0
9 0
9 0
SrMax
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9999.
9
9
9
9
9
9
9
9
9
Q
9
9
9
9
9
9
9
9
9
Q SrStd
0
0
0
0
0
0
0
0
0
25.
0.
0.
0.
0.
0.
0.
0.
0.
4
0
0
0
0
0
0
0
0
•A
Q Q
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
V
Figure 58: NCDC - Surface data, Hourly Global data, Date and Time in separate columns
62
-------
Appendix-C: Downloading Data from the National Climate Data Center
NCDC:_!_National Climatic Data Center_(NCDC)J!l
www.ncdc.noaa.gov/
NOAA Satellite and Information Service \/
National Environmental Sate*te, Data, and Information Service (NESDIS)
National Climatic
Data Center
(/ S Dfpartmv* of (attorn t
NCDC News
ClrnaleFAQs
AboUNCOC
Data 8t Products
»DataQritieOCWl|
'-". :-.t-: V : :r''-^7.
•>jidi l)vM.n<
i Access tool;
Workf $ Largest Archive of Oinwte Data
National Climatic
Data Center
Protecting the Past, Revealing the Future
OnateHvertones
Mdacfcalo
HttofAQ
Clln
Reoioftal Cfctat* Center s
I: -t'lrTrl
Gtobol Hazards
Figure 59: National Climatic Data Center home page
To access data, follow these steps:
Step 1: Click on the "Climate Data Online" link [Figure 59] to open an interactive map
application web page [Figure 60].
Step 2: Click on the "Interactive Map Application" figure to open an interactive GIS-based
world map. Initially, a help window displays to provide information on downloading the data
[Figure 61].
Step 3: Close the Map Help window to view the world map [Figure 62].
63
-------
•NCAA
NATIONAL CLIMATIC DATA CENTER
NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION
Home Climate Data Online Contact (It About NCDC Help
Home > Climate Data Online
Climate Data Online
Search by Country/State/ZIP code
Select a Climate Theme Map
Select a map for a specific data set or climate theme below, or directly access all legacy content.
Figure 60: Interactive map application page
Have questions about the data? Need some
assistance? Use the links below to quickly find the
answers you need.
Online help
Check request status
Request assistance
Need technical documentation or system access?
View data samples & documentation
NCDC Web Services
CDO Web Services Documentation
t/ Map Help
Welcome Navigation Basemaps Layers Legend
IBI
Identify looK
GIS Resource* NEXRAD About
Welcome
Where do I ttart?
Step 1
Use the 1ft Layer controls to make the desired map layer visible.
Step 2
Navigate to your area of interest.
Step 3
With the O Identify or j Select Tools, select stations on the map.
Step 4
In the Results window, dick the & Data or the ^ Graph icon to access or view data
for a single station.
Figure 61: Map Help window
64
-------
Step 4: At the top of the page, select the required format of the data from the Observations
combo box [Figure 62]. For HSPF Tool applications, select the Hourly/Sub-Hourly option.
Step 5: The user can zoom to the area of interest using zooming options or the locator text box at
the right-hand top corner of the web page.
Hourly/Sub-Hourly Observational Data
a location bvlow to rwtrett the map)
Watkinsville, Georgia, United States
Watkinsville Kindergarten, school, Watkinsville, Ge
City of Watkinsville, administrative division, United
>'"di-!'in:-"'l.i- P.-it Otfi ..-. r.-iv.t nttn:-
Current Tool: Move Map
Map Scale 1:147,743.799
Latitude: -46.054683 Longitude: -1,406230
Figure 62: GIS Map window
65
-------
Step 6: In the layers sub-window on the web page, select layers or the type of data required. In
the current example, select/check All to view the features represented as dots in various colors
[Figure 63].
r»« County
Hourly Precipitation •
IrtXP.AD Sites, MOJ*IC, and DMa O
Figure 63: Layers sub window
66
-------
Step 7: From the toolbar at the top of the page, click on the "i" button [Figure 64].
Hourly/Sub-Hourly Observational Data
•**•*•• (»*' IVII * n a77
• •
NEXRAD Mosaic: Present time
Ob*«rV4
Coun I y
Figure 64: Feature selection
Step 8: The mouse turns into a cross-wire, allowing users to click on any dot in the enabled data
layers from which data can be downloaded via a Results window [Figure 65].
Ha
1*3
1 rec
jrly Global
Station *
WAT KINS VI LLE B SSE
ord found
ku
AWS
999999
WB AN
63850
COOPID
099I7S
Period of Record
200-4/Q4/30 - 2012/08/06
State
GA
Country
u .
Call Sign
F7BE
AWSBAN
9999996385
Legend
Latitude
33.784*
| .,,,
liL Graph
Longitud
•83.39"
CIS)
V
Figure 65: Results window
67
-------
Step 9: In the Results window, select/check the site of interest from the list and click the Get
Selected Data button, highlighted in Figure 65. A small Data Access Options [Figure 66]
window opens with two options. Select Advanced and click the Access Data button.
Hourly Global
Use checkboxes below for single/mult.
access (n- Data Access Options
a
* AWS WP.Af - Simplified
y Call Sign AWSBAN Latitude Longitud
WATK1NSVILLE 5 $
'. _. Advanced
F78E 999999638S 33-78^° -83.39°
I record found
..
Figure 66: Data Access Option Window
68
-------
Step 10: After clicking the Access Data button, a new Tab opens to display a list of parameters
[Figure 67]. Select the parameter of interest and click on Continue to select dates for which data
must be downloaded [Figure 68].
NOAA Satellite and Information Service vV'v
National Environmental Satellite. Data, and Information Service (NESDIS)
National Climatic
Data Center (
U I- Orywrrwtf vt CWMM
DOC > HOAA > NESDIS > NCDC SearchFteld
[_Search_NCDCj
Lanrf-Based Data / NHDC CDO / Product Starch / Help
NNDC CLIMATE DATA ONLINE
Surface Data, Hourly Global
for «rlri inl stations 1 stations
S-l... i ]>,i,, ElemeiiKsl
Hourly Solar Radiation Section *"*
Hourly Surface Temperature Section ™
Hourly Surface Temperature Sensot Section ™
Hourly Temperature Extreme Section (CRN) ™
Hourly Temperature Section (CRN) ~"
Hourly Wind Section ~
jquid Precipitation (By Minute) ***
quid Precipitation Greatest Amount In 24 Hours, For The Month '
Jquid Precipitation Maximum Short Duration. For Tn« Month ***
| Clear Selections ] | Previous Page ] | New Search]
Figure 67: Parameter Selection window
Step 11: Select the dates, delimiter, output format, and then select Only Obs. On the Hour button
[Figure 68].
Step 12: Enter a valid email address and click Submit Request [Figure 68]. The data will be
emailed as a text file.
69
-------
NNDC CLIMATE DATA ONLINE
Surface Data, Hourly Global
for selected stations - 1 stations. Last available data: 2012/04/28
Select Date Restrictions:
© Us* Date Range = OR :
'iv.u Month Day Hoiu
Fioin 2011 v 01 v oi v 00 v
To
2012 v 04 v 01 v 23
0 Select Onh- Ob*, on the Hoiu
O Use Selected Dates *
Year MoiithDav Horn
Select Output Foimat:
Delimited, without station name •*
Select Output Format Drlimitrr:
PfflHT^r
Space
Output via: FTP
| Continue | [ Cleat Selections | [ Previous Page | [ New Search j
NNDC CLIMATE DATA ONLINE
DS3505 - Surface Data. Hourly Global, Request Summm >
Entirr Datsisrl Selected Stations - includes 1 mriotu (Stt alectadstations btlotv)
Date Ranee (Ytar / Month / Day / Hour )
231 IflMM/OO to 2012/04/01/23
Selected Onrput Foiinat:
Coneii Dthnutt d. without tintion nuw
Selected Output Media
FTP
Hourly Obs Available
Vww Iftrtnloiy
Ontpnt File Sis* (bytes):
205968
H Inveutoiy ReMew I have i sieved the tfffgtfffvFJe to see if the
n .ill .(hit- ht'/iir,' ottl^tllkf Suini* tim«- JM-LI-I,!-. m 4-l«MntMlt<; in.tv b<* lui .
leqnesteil penoii of leroid is
in^nv sianous do not i epoii r>-piy honi . but on'r *v«v J hmox. and voinrninr5 onh diuuig i
Inn
Pleas*
a \ .ili«l «lnail addles* bvlow so w*
-------
Appendix-D. STORET Data Download Options
If you know the Organization ID and Station Number of a particular EPA STORET site, you can
enter site information in the respective textboxes of the HDFT Data download Window [Figure
69].
m Data Download Window
Enter the Station Information.
Enter the Start and End Dates
Click 'Download Data1 to place the data into the table
Click to get information for the selected site
Database
USGS Station Number
Organization ID
EPA-Storet
11NPSWRD
Station ID |cuva_nps_roro
NOTE: Please be patient, it may take a minute sometimes to retrieve/download data from the server
Figure 69: HDFT Data Download Window
Users can obtain Organization ID and Station ID from the STORET database using
MyWatersMapper, accessed at: http://www.epa.gov/waters/enviromapper/
My Water sMapper is a web tool that dynamically displays EPA STORET data. It contains
numbered HUCs (i.e., 6-, 8-, and 12-digit watersheds), National Hydrology Dataset and other
water-related map layers. It functions similarly to the USGS Mapper, however, EPA STORET
sites are only visible at particular zoom levels.
Step 1: On the welcome page of My WATERS Mapper, click anywhere on the map to launch the
Main Window [Figure 70].
71
-------
^^l_l t^ Mttf Sm« tn*m***i> f, min<*i Hinr.
IEASN TH£ ISSUES I SCeWttTEOMOlMY I LAWS t MBUUTKMS I «MOT EM
MyWAMRS M.(()pfi
YMI at km: in K.-H « - A ,:- - - loan •
Welcome to the new MyWATERS Mapper'
HMWTO i MtpMt
• ALL IF* OTHCAtlA Mnmrffeo
(L,
M«p«j«r
Cf*«U (Ull<
t
-------
Figure 72: HUC 8 watershed map.
Step 4: Click on the HUC 8 watershed where your site is likely to be found [Figure 72], but
click on as many HUC 8 watersheds as needed until you find the specific site of the STORET
site-of-interest [Figure 73].
Cleveland
ihtt
EPA MyWATFRS Mapper
.'. Banufta j
G^ K. ..i,
Sc I.
Features at this Location
Th«tt l> I feature *t thlf lootion.
Hydraloqk Unit tod* (HUC*)i 041100O2
N»m»i Cuy»hog«
Size (acm)i S12788.97S824
Sfat«(w|km)i 207S.183144
Intenectlnq Stjt»ti OH
SUbon K«ultsi
Mass
Show Monitoring
Station Results
j ;.-.'.•,*•..-
c^ / u~
Figure 73: Information about selected HUC 8 watershed (Cuyahoga- HUC # 0411002)
Step 5: As shown in Figure 73, click Show Monitoring Station Results to open a new window.
The program will search for results and display Organization IDs and Station Numbers for all
STORET sites -- HUC 0411002 in this example.
73
-------
It may take some time to load the information, especially if the number of stations is large and/or
if the number of parameters associated with each station is large [Figure 74].
Query Rosolt* X
Subbasins (HUC8) results for 04110002
Loading, please wait...
Figure 74: Program is loading the STORET Results
Step 6: From the query results window [Figure 75], select the Organization ID and Station
Number of the STORET Site of interest. HDFT users can get these essential pieces information
and stop, or continue to test if the STORET web download is working properly and the
parameter-of-interest is available.
74
-------
Query Results
Subbasins (HUC8) results for 04110002
1 View Results as Tabular Grid View Res
Organization
11NPSWRD
N»tion»l Park
Service Water
Resources
Division
11NPSWRD
National Park
Service Water
Resource*
Division
11NPSWRD
Station
CUVA_NPS_RORO
Cuvahoga River
at Rod-side
Road Bridge
CUVA_HPS_RORU
Robinson Run
CUVA_NPS_SACR
Start Date
04/02/1990
04/23/1985
04/30/1986
ults as Nested Tree Nodes
Characteristics Results
5 1233
^H 1
24
24
•?:31
1032
Get more results for a single monitoring station:
-All Stations-
•
Download
XML Data
Dowi
CSV
11NPSWRD / CUVA_NPS_RIRU
11NPSWRD / CUVA_HPS_RORO
11NPSWRD / CUVA_HPS_RORU
11NPSWRD / CUVA_NPS_SACR
11NPSWRD / CUVA_NPS_SARU
11NPSWRO / CUVA HPS SPCR
• 11NPSWRD / CUVA HPS SR 303
Figure 75: List of STORET Sites showing information on available
parameters/characteristics (5) and the total number of samples (1235).
Step 7: As shown in Figure 75, click the ALL Stations combo box and select Organization ID
and Station Number from the list to get detailed information on that location [Figure 76].
75
-------
Water Monitoring Locations
results for 11NPSWRD, CUVA_NPS_RORO
[ Vtav* Results as Tabular Grid 1 View Results as Nested Tree Nodes
Organization Station Project Activity Type Date
11MPSWRO CUVA_NPS_R CUVA0001 NFAOO_OOOS Field Msr/Obs 2008-07-15
National Park ORO \\Custody_lD:
Service Water Cuy»hog» 2008070013
Resources River at
Division Rockslde
Road Bridge
11NPSWRO CUVA_NPS_R CUVAOOI
National Park ORO
Service Water Cuyahoga
Resources River at
Division Rockside
1 Road Bridoe
)1 NFAOO.OOOS Field Msr/Obs 2008-07-16
\\Custodv_ID:
2008070016
V
Get more results for a single monitoring station: 11NPS WRD / CUVA_NPS_RORO
,
Get
More
Details
from
STORET
Download
XML Data
^J
Download
CSV Data
Download
KML File
Figure 76: Results for Station Number CUVA_NPS_RORO
Step 8: As shown in Figure 76, click on Get More Details from STORET to obtain information
shown in Figure 77. Now that you have detailed information about the site, you do not need
additional information from MyWATERS Mapper.
Suion: CUVA_MPS.RC*0 Cvy»hog» Ri* « flockiidt Ro»j Bmljt ^^ /
Oi^nizaSi)Kl1MPSWTC Njbonal Park Service WU«T Reswcts Dxtstyi ^1
SI.IBOB Iiltmi44ion
'liiiuiy Iy[i* 's?ti'Stie»m
SttondJiy Tyji*
.ortii4»loiii)iiiii(l»..".:r-!' ??«C65 Lcngsud* -916S5iJ3. HAT62
•Itvitiwi 59E. 'Jivt39
5W« 3HO
CMHrty CJ'fAHCCv*
tyhclogk Huh :^"1X'J Cvuniiq*
Numb »i ')! R'siittv Dal* 1
tag
AJUS-W-A)
3QOW&2!
1991-09-22
Figure 77: List of parameters, number of results, and date ranges.
76
-------
Step 9: In the data download menu shown in Figure 78, select STORET database and paste the
Organization ID and Station Number in their respective textboxes.
Ddld Download Window
the Station Information.
the Stait and End Dates
:iick 'Download Data' to place the data into Hie tabte
Click to get Information for the selected site
Database
EPA Store!
USGS Station Number
Organization ID 11NPSYVRD
Station ID 'Ci/rt_nps._roro
NOTE: Please be patient, it may take a minute sometimes to retrieve download data from the server
Figure 78: Data download window.
Step 10: Click Get Options to view the information shown in Figure 79.
SIOHII Available Ptierntttti
Lhrt of «y«iibli paramM
M ,li : ,
EstrwichU ton
;.• .! ',•. I-:.
24fir P«KH lo monrtonnj *v«nt
T«mp«r«
-------
Step 11: Next, click the Get Site Info button [Figure 80] on the Data Download window to
display site information, parameter information such as number of samples, Start/End dates of
the data available, latitude and longitude of the site, etc. [Figure 80].
Cickto get information for the selected site
Database EPA Stocet
USGS Station Number
11NPSWRD Station 10 |cuva_nps_foro
Get Options
NOTE: Please be patient, rt may take a minute sometimes to retrieve''download data from the server
Station Name Cwahoga River at Rockside Road Bridge
Location (Latitude, Longitude) 41 3944055 . -81 6295233
Start Date 04/02/1990 End Date |09f22/1991
Number of records available
Fecal Conform, cfu'lOOml 225
Get Site Info
AcceptCnter the dates for which data has to be downloaded
Start Date 04/02/1990
Select Units CFU
End Date 09/22/1991
Download Data
Figure 80: Site and parameter information extracted
Step 12: Click the Download Data button [Figure 80] to download data to the table on the Main
Page [Figure 81].
78
-------
Step 13: Click Export Data on the Main Page [Figure 81] to open an Export Data window
[Figure 82].
DOWNLOAD MIA
IWO-M-W
i
4MOO
199044-03
••• :
• '
: -
43000
.
MOOO
JWOO
17000
17000
-
180000
.•
OMO
-
8CCOO
I
40000
I
IWWH-18
: -
!MCO
I
'(>;«•:•
; -:
i'OOPO
610000
0*0
o»«io»t»
Figure 81: Data downloaded from EPA STORE!
Step 14: Select the date format and check the appropriate checkboxes. Enter date and time
delimiters and dates (Start and End) for which data must be exported. Default dates will apply to
the entire dataset [Figure 82].
79
-------
Step 15: Click the Copy to Clipboard button [Figure 82].
Export Data
- n x
Date Format
• year month day
O month day year
O day month year
Date separator \l
Time separator |T_
0 Years
0 Month
Start date
End date
1990-04-02
1991-09-22
Copy to Clipboard
Figure 82: Export Data Window
Step 16: Paste the data into a text file or other application [Figure 83].
04/02/1990
04/03/1990
04/04/1990
04/05/1990
04/06/1990
04/08/1990
04/09/1990
04/10/1990
04/11/1990
04/12/1990
04/13/1990
04/14/1990
04/15/1990
04/16/1990
04/17/1990
04/18/1990
04/19/1990
04/21/1990
04/22/1990
04/23/1990
04/24/1990
Figure 83:
4800.0
4700.0
4300.0
2800.0
2500.0
1700.0
1000.0
1500.0
18000.0
9000.0
3000.0
2900.0
4000.0
1300.0
400.0
3900.0
100.0
61000.0
7000.0
3600.0
2100.0
Exported Sample Data
80
-------
Appendix-E: Obtaining USGS Station Numbers
To download flow/water quality data from USGS NWIS to the Water Quality Prediction Tool,
users must know the USGS Station Number [Figure 84].
Data Download Window
Entei the Station Information.
ntfi the Start and End Dates
Hick 'Download Data' to place the data into the t.ihle
< in k to get ititorrn.itton rot the selected srfe
Database NIMS Daily values
USGS Station Number JD1667500
Organization^
Station ID
Variable
Figure 84: HDFT Data download window
The USGS NWIS Mapper is a user-friendly map containing all USGS gauges, with links to
available water data for individual gauges at the following link:
http://wdr.water.usgs.gov/nwisgmap/
Using the USGS NWIS Mapper, HDFT users can obtain the required USGS Station Number for
which data is needed. The downloading capability of the Water Quality tool is made possible by
the web services of CUASHI (http://his.cuahsi.org/wofws.html).
Follow the steps below to extract the USGS station number:
Step 1: In the internet browser, go to the USGS WNIS Home page [Figure 85] at:
http://waterdata.usgs.gov/nwis
Step 2: Click on the map icon [Figure 85] to get to the USGS NWIS Mapper page [Figure 86].
81
-------
News updated Apnl, 2012
USGS Water Data for the Nation
Current Conditions
Site Information
Search for Sites With Data
Sites with real-time or recent surface-water, groundwater, or water-quality data
Descriptive site information for all sites with links to all available water data for
individual sites.
Map of all sites with links to all available water data for individual sites.
Frequent Searches By Data Category
Surface Water
Groundwatcr
Water flow and levels in streams and lakes.
water levels in wells.
Chemical and physical data for streams, lakes, spnngs, wells and other sites.
Figure 85: USGS NWIS home page
Step 3: The user can zoom to the location of the station with zoom tools or the Zoom To option
at the top of the map (highlighted in Figure 86).
For example, the user can select the state or type in city name/address/ zip code.
Step 4: After choosing the location, click GO to zoom to the area/point of interest. The user can
view USGS locations represented by small green triangles [Figure 87]. Click on the triangles to
view their USGS Station Numbers and Station Names.
The user can also view a list of sites and available details in the zoomed area by clicking the List
Sites button on the left-hand side of the screen (highlighted in Figure 88), which opens a List of
Sites on Map window [Figure 88].
82
-------
We /come.
For best results, please read our
Instructions and Disclaimer.
0 Surface-Water Sites 11
(streams, Jakes, open water)
A Any data (not clickable)
rj Groundwater Sites
(wefts, subsurface)
21
D Spring Sites
fj Atmospheric Sites
(climate, weather, deposition)
11
rj Other Sites
(facilities, land, glaciers, water use)
11
p> E Alaska
I'T' I Pi American Samoa
EKE |rras
California
: Colorado
, Connecticut
I Delaware
District of Columbia
Florida
Georgia
Guam
Hawaii
Idaho
Illinois
Indiana
Iowa
Saskatchewan
Figure 86: USGS NWIS Mapper home page
National Water Information System: Mapper
Ok* < ur» re *te«u in 4Xi
(Curnrx loom *»* * 1 1 )
. Surf »l ..*.*-' Ml. .^
__ . Itlt
• . rpi»
-*.-•(
D 5
SpnnT.ni.
Figure 87: USGS Station Number
83
-------
©about: blank
SiUSGS
List of Sites from NAMS Mapper
Request for Active sites in data base with Any data.
Site Types are Surface Water.
This list contains 21 sites.
Created on 5/17/112 at 11:07 (GMT-4).
Bounding coordinates are (-83.63,33.71,-83,34.1).
Site
Number
022175051
Site Name
0 MIDDLE OCONEE RIVER NEAR JEFFERSON, GA
APALACHEE RIVER .40 MILES US TJS 78 NR
BOGART, GA
MIDDLE OCONEE RIVER NEAR ARCADE, GA
MIDDLE OCONEE RIVER AT GA 330, NEAR
STATHAM, GA
APALACHEE RIVER NEAR BOSTWICK, GA
MIDDLE OCONEE RIVER NEAR ATHENS, GA
BROOKLYN BRANCH AT FORTSON DR, AT
ATHENS, GA
BROOKLYN CREEK AT DUDLEY DR, AT ATHENS,
GA
Figure 88: List of Sites on Map
84
------- |