Difference between revisions of "How to convert data for Geogrid"

From openwfm
Jump to navigation Jump to search
(Created page with 'Running WRF-Fire using real data requires additional datasets not included in the standard WPS input data [http://www.mmm.ucar.edu/wrf/src/wps_files/geog_v3.1.tar.gz tarball]. F…')
 
 
(37 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Running WRF-Fire using real data requires additional datasets not included in the standard
+
{{users guide}}
WPS input data [http://www.mmm.ucar.edu/wrf/src/wps_files/geog_v3.1.tar.gz tarball].  For these,
 
it is necessary to convert your source data into the
 
[http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3.1/users_guide_chap3.htm#_Writing_Static_Data geogrid data format].
 
The geogrid data format consists of a directory of tiled binary files with names indicating the index range contained in each tile.
 
For instance, a file name of <tt>00101-00200.00051-00100</tt> would consist of columns 101 through 200 and rows 51 through 100.
 
The files themselves contain only an array integers, with no associated metadata.
 
The metadata for the dataset is contained in a file called <tt>index</tt>, which contains a series ''keyword=value'' statements
 
telling geogrid the details of the data tiles, such as width, height, the number of bytes per integer, etc.  The index file also contains metadata
 
for the data set itself, such as the projection information and physical units.
 
  
We have made available a series of scripts and utilities ([http://github.com/jbeezley/autoWPS autoWPS]) intended to automate many of the steps
+
This wiki page explains how to transform geotiff files to geogrid using convert_geotiff script in [https://github.com/openwfm/wrfxpy.git wrfxpy].
required to create a dataset for a WRF real run. A utility program included in this repository, WPSGeoTiff, is a simple command line utility that
 
takes a GeoTiff file and writes converts the data into geogrid binary format.  The GeoTiff format was chosen because it is a highly ubiquitous format
 
for geotagged data and it has a cross-platform and open source c library called [http://trac.osgeo.org/geotiff/ libGeoTIFF].  Most, if not all, GIS application
 
software is capable of converting or exporting a dataset as a GeoTiff file.
 
  
=Prerequisites=
+
== Get Anaconda3 distribution ==
  
The WPSGeoTiff utility requires two external libraries in order to compile.  You should pick a location to install the libraries, for instance <tt>${HOME}/opt</tt>.
+
Download and install the Python 3 [https://www.anaconda.com/products/individual Anaconda Python] distribution for your platform. We recommend an installation into the user's home directory.
* [http://www.libtiff.org/ libTIFF]
 
* [http://trac.osgeo.org/geotiff/ libGeoTIFF]
 
Both are written in c, and include standard GNU auto-configuration scripts.
 
  
=Obtaining the WPSGeoTiff source=
+
== Install environment ==
  
The autoWPS package is available as a git repository hosted at [http://github.com/jbeezley/autoWPS github].  The repository can be cloned with the
+
Create the python environment:
[http://git-scm.com/ git] command:
 
<code>git clone git://github.com/jbeezley/autoWPS.git</code>
 
A tarball of the latest release can also be obtained
 
at [http://github.com/jbeezley/autoWPS/tarball/master http://github.com/jbeezley/autoWPS/tarball/master].
 
  
=Compiling WPSGeoTiff=
+
<pre>
 +
conda update -n base -c defaults conda
 +
conda create -n wrfx python=3 gdal netcdf4 pyproj paramiko dill h5py psutil proj4 pytz flask pandas
 +
conda activate wrfx
 +
conda install -c conda-forge simplekml pygrib f90nml pyhdf xmltodict basemap rasterio scipy
 +
pip install MesoPy python-cmr
 +
</pre>
  
Inside the main directory of the autoWPS release is a subdirectory called WPSGeoTiff.
+
Note that <tt>conda</tt> and <tt>pip</tt> are package managers available in the Anaconda Python distribution.
 +
 
 +
== Get wrfxpy repository ==
 +
 
 +
Get repository wrfxpy and checkout to convert_geotiff branch. Shortly, it is going to be available in the master branch.
 +
git clone https://github.com/openwfm/wrfxpy
 +
cd wrfxpy
 +
git checkout convert_geotiff
 +
 
 +
== Run convert_geotiff.sh ==
 +
 
 +
Run the code to generate geogrid from geotiff files. Run the following line for each variable to process.
 +
./convert_geotiff.sh geotiff_file geo_data_path wrf_variable_name [bbox]
 +
 
 +
where:
 +
 
 +
* <tt>geotiff_file</tt> - path to a geotiff file to be converted to geogrid
 +
* <tt>geo_data_path</tt> - any path where to save all the variables
 +
* <tt>wrf_variable</tt> - WRF variable created on src/geo/var_wisdom.py
 +
* <tt>bbox</tt> - optional, bounding box to sample the original geotiff file from, using WGS84 coordinates. Format: min_lon,max_lon,min_lat,max_lat
 +
 
 +
If you want to know more information and the existing options for wrf_variable, run ./convert_geotiff.sh without inputs
 +
./convert_geotiff.sh
 +
 
 +
Some examples:
 +
./convert_geotiff.sh elevation.tif geo_data ZSF
 +
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT
 +
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT -112.8115,-112.1661,39.4820,39.9750
 +
 
 +
== Results from convert_geotiff.sh ==
 +
 
 +
The convert_geotiff.sh script will generate a folder specified by the user by <tt>geo_data_path</tt> with
 +
 
 +
* '''Variable folders:''' A folder for each variable previously run (ZSF and NFUEL_CAT folders) containing:
 +
** a geogrid file of the variable.
 +
** an index file of the variable.
 +
* '''geogrid_tbl.json:''' A JSON file with all the geogrid information for each variable.
 +
* '''GEOGRID.TBL:''' A text file with the information to add to GEOGRID.TBL (depends on src/geo/var_wisdom.py, so only copy what you need).
 +
* '''index.json:''' A JSON file with index information for each variable.
 +
 
 +
[[Category:WRF-Fire]]
 +
[[Category:Howtos|Convert data for Geogrid]]
 +
[[Category:Data]]

Latest revision as of 01:59, 17 February 2021

Back to the WRF-SFIRE user guide.

This wiki page explains how to transform geotiff files to geogrid using convert_geotiff script in wrfxpy.

Get Anaconda3 distribution

Download and install the Python 3 Anaconda Python distribution for your platform. We recommend an installation into the user's home directory.

Install environment

Create the python environment:

 conda update -n base -c defaults conda
 conda create -n wrfx python=3 gdal netcdf4 pyproj paramiko dill h5py psutil proj4 pytz flask pandas
 conda activate wrfx
 conda install -c conda-forge simplekml pygrib f90nml pyhdf xmltodict basemap rasterio scipy
 pip install MesoPy python-cmr

Note that conda and pip are package managers available in the Anaconda Python distribution.

Get wrfxpy repository

Get repository wrfxpy and checkout to convert_geotiff branch. Shortly, it is going to be available in the master branch.

git clone https://github.com/openwfm/wrfxpy
cd wrfxpy
git checkout convert_geotiff

Run convert_geotiff.sh

Run the code to generate geogrid from geotiff files. Run the following line for each variable to process.

./convert_geotiff.sh geotiff_file geo_data_path wrf_variable_name [bbox]

where:

  • geotiff_file - path to a geotiff file to be converted to geogrid
  • geo_data_path - any path where to save all the variables
  • wrf_variable - WRF variable created on src/geo/var_wisdom.py
  • bbox - optional, bounding box to sample the original geotiff file from, using WGS84 coordinates. Format: min_lon,max_lon,min_lat,max_lat

If you want to know more information and the existing options for wrf_variable, run ./convert_geotiff.sh without inputs

./convert_geotiff.sh

Some examples:

./convert_geotiff.sh elevation.tif geo_data ZSF
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT -112.8115,-112.1661,39.4820,39.9750

Results from convert_geotiff.sh

The convert_geotiff.sh script will generate a folder specified by the user by geo_data_path with

  • Variable folders: A folder for each variable previously run (ZSF and NFUEL_CAT folders) containing:
    • a geogrid file of the variable.
    • an index file of the variable.
  • geogrid_tbl.json: A JSON file with all the geogrid information for each variable.
  • GEOGRID.TBL: A text file with the information to add to GEOGRID.TBL (depends on src/geo/var_wisdom.py, so only copy what you need).
  • index.json: A JSON file with index information for each variable.