Difference between revisions of "How to ingest heat flux data"

From openwfm
Jump to navigation Jump to search
(place holder)
 
 
Line 1: Line 1:
 +
When dealing with real data, it is often necessary to reproject and interpolate raw data sources onto a WRF simulation grid.  This is possible
 +
using WPS in a manner similar to the method used [[How to run WRF-Fire with real data|here]].  This page illustrates this procedure on
 +
MODIS heat flux data.  The source data is converted and interpolated from its native geotiff format into a WRF compatible netCDF file where it can be
 +
merged into a wrfinput file or analyzed using standard netCDF tools.
  
 +
=Converting data to geotiff=
 +
 +
The method outlined here requires data in a geotiff format; however, not all data comes in this form.  Most GIS utilities provide converters to save data
 +
in geotiff format.  One free and cross platform utility for accomplishing this is the [http://www.gdal.org/gdal_translate.html gdal_translate] command, which
 +
is part of the Geospatial Data Abstraction Library (GDAL) [http://www.gdal.org/].  Often it is sufficient to run the following command to perform the conversion:
 +
gdal_translate -of GTiff <source dataset> <destination dataset>
 +
GDAL command line scripts are also capable of performing reprojection as well as simple processing of nearly any GIS file format.
 +
 +
=Interpolating data onto a model grid using geogrid=
 +
 +
Using the original <tt>namelist.wps</tt> used to create the domain.  It is relatively simple to rerun geogrid with an additional dataset from
 +
your source data.  In this example, we will use the namelist linked [https://gist.github.com/3696702 here] and the geotiff data at [[File:O9820-B65-P1 fire mask.tif]].
 +
*Compile WPS as described [[WPS with GeoTIFF support|here]] to enable geotiff support.
 +
*Edit the geogrid table at <tt>geogrid/GEOGRID.TBL</tt> to contain the new variable.  See the list of table parameters  [http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3.1/users_guide_chap3.htm#_Description_of_GEOGRID.TBL here].  You can remove all of the variables present
 +
with the exception of <tt>LANDUSEF</tt> to save on computation time.  A minimal table is as follows:
 +
<pre>===============================
 +
name=LANDUSEF
 +
        priority=1
 +
        dest_type=categorical
 +
        z_dim_name=land_cat
 +
        landmask_water =  modis_30s:17            # Calculate a landmask from this field
 +
        landmask_water = modis_lakes:17,21        # Calculate a landmask from this field
 +
        landmask_water =  usgs_lakes:16,28        # Calculate a landmask from this field
 +
        landmask_water =    default:16            # Calculate a landmask from this field
 +
        dominant=LU_INDEX
 +
        interp_option =  modis_30s:nearest_neighbor
 +
        interp_option =        30s:nearest_neighbor
 +
        interp_option =  usgs_lakes:nearest_neighbor
 +
        interp_option = modis_lakes:nearest_neighbor
 +
        interp_option =          2m:four_pt
 +
        interp_option =          5m:four_pt
 +
        interp_option =        10m:four_pt
 +
        interp_option =      default:four_pt
 +
        rel_path=  modis_30s:modis_landuse_20class_30s/
 +
        rel_path=        30s:landuse_30s/
 +
        rel_path=  usgs_lakes:landuse_30s_with_lakes/
 +
        rel_path= modis_lakes:modis_landuse_21class_30s/
 +
        rel_path=          2m:landuse_2m/
 +
        rel_path=          5m:landuse_5m/
 +
        rel_path=        10m:landuse_10m/
 +
        rel_path=    default:landuse_2m/
 +
===============================
 +
name=FIREMASK
 +
        priority=1
 +
subgrid=yes
 +
        dominant_only=FIREMASK
 +
        dest_type=categorical
 +
        z_dim_name=firecat
 +
        fill_missing = 0.
 +
        interp_option=default:nearest_neighbor
 +
        abs_path=default:./firemask
 +
===============================</pre>
 +
*Create a new directory under <tt>WPS</tt> (here we use <tt>firemask</tt>) and copy the geotiff file into this directory.
 +
*Create an index file in the directory you created.  The format of the index file is described [[WPS_with_GeoTIFF_support#Using_GeoTIFF_with_WPS|here]] and [http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3.1/users_guide_chap3.htm#_Description_of_index here].  In this example, our index file will contain
 +
the following
 +
<pre>geotiff=O9820-B65-P1_fire_mask.tif
 +
type=categorical
 +
category_min=1
 +
category_max=9
 +
tile_bdr=3</pre>
 +
*Run <tt>geogrid.exe</tt>
 +
 +
At this point, the newly interpolated data is in the file <tt>geo_em.d01.nc</tt> under the variable given in the table (here <tt>FIREMASK</tt> in [[File:FIREMASK Geo em.nc.gz]]).  The data can be added to a wrfinput or wrfrst file using ncks [http://nco.sourceforge.net/] using the following command:
 +
ncks -A -v FIREMASK geo_em.d01.nc wrfinput_d01
 
[[Category:Howtos|Ingest heat flux data]]
 
[[Category:Howtos|Ingest heat flux data]]

Latest revision as of 08:12, 11 September 2012

When dealing with real data, it is often necessary to reproject and interpolate raw data sources onto a WRF simulation grid. This is possible using WPS in a manner similar to the method used here. This page illustrates this procedure on MODIS heat flux data. The source data is converted and interpolated from its native geotiff format into a WRF compatible netCDF file where it can be merged into a wrfinput file or analyzed using standard netCDF tools.

Converting data to geotiff

The method outlined here requires data in a geotiff format; however, not all data comes in this form. Most GIS utilities provide converters to save data in geotiff format. One free and cross platform utility for accomplishing this is the gdal_translate command, which is part of the Geospatial Data Abstraction Library (GDAL) [1]. Often it is sufficient to run the following command to perform the conversion:

gdal_translate -of GTiff <source dataset> <destination dataset>

GDAL command line scripts are also capable of performing reprojection as well as simple processing of nearly any GIS file format.

Interpolating data onto a model grid using geogrid

Using the original namelist.wps used to create the domain. It is relatively simple to rerun geogrid with an additional dataset from your source data. In this example, we will use the namelist linked here and the geotiff data at File:O9820-B65-P1 fire mask.tif.

  • Compile WPS as described here to enable geotiff support.
  • Edit the geogrid table at geogrid/GEOGRID.TBL to contain the new variable. See the list of table parameters here. You can remove all of the variables present

with the exception of LANDUSEF to save on computation time. A minimal table is as follows:

===============================
name=LANDUSEF
        priority=1
        dest_type=categorical
        z_dim_name=land_cat
        landmask_water =   modis_30s:17            # Calculate a landmask from this field
        landmask_water = modis_lakes:17,21         # Calculate a landmask from this field
        landmask_water =  usgs_lakes:16,28         # Calculate a landmask from this field
        landmask_water =     default:16            # Calculate a landmask from this field
        dominant=LU_INDEX
        interp_option =   modis_30s:nearest_neighbor
        interp_option =         30s:nearest_neighbor
        interp_option =  usgs_lakes:nearest_neighbor
        interp_option = modis_lakes:nearest_neighbor
        interp_option =          2m:four_pt
        interp_option =          5m:four_pt
        interp_option =         10m:four_pt
        interp_option =      default:four_pt
        rel_path=   modis_30s:modis_landuse_20class_30s/
        rel_path=         30s:landuse_30s/
        rel_path=  usgs_lakes:landuse_30s_with_lakes/
        rel_path= modis_lakes:modis_landuse_21class_30s/
        rel_path=          2m:landuse_2m/
        rel_path=          5m:landuse_5m/
        rel_path=         10m:landuse_10m/
        rel_path=     default:landuse_2m/
===============================
name=FIREMASK
        priority=1
	subgrid=yes
        dominant_only=FIREMASK
        dest_type=categorical
        z_dim_name=firecat
        fill_missing = 0.
        interp_option=default:nearest_neighbor
        abs_path=default:./firemask
===============================
  • Create a new directory under WPS (here we use firemask) and copy the geotiff file into this directory.
  • Create an index file in the directory you created. The format of the index file is described here and here. In this example, our index file will contain

the following

geotiff=O9820-B65-P1_fire_mask.tif
type=categorical
category_min=1
category_max=9
tile_bdr=3
  • Run geogrid.exe

At this point, the newly interpolated data is in the file geo_em.d01.nc under the variable given in the table (here FIREMASK in File:FIREMASK Geo em.nc.gz). The data can be added to a wrfinput or wrfrst file using ncks [2] using the following command:

ncks -A -v FIREMASK geo_em.d01.nc wrfinput_d01