Difference between revisions of "How to convert data for Geogrid"

From openwfm
Jump to navigation Jump to search
(Created page with 'Running WRF-Fire using real data requires additional datasets not included in the standard WPS input data [http://www.mmm.ucar.edu/wrf/src/wps_files/geog_v3.1.tar.gz tarball]. F…')
 
 
(37 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
Running WRF-Fire using real data requires additional datasets not included in the standard
+
{{users guide}}
−
WPS input data [http://www.mmm.ucar.edu/wrf/src/wps_files/geog_v3.1.tar.gz tarball].  For these,
 
−
it is necessary to convert your source data into the
 
−
[http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3.1/users_guide_chap3.htm#_Writing_Static_Data geogrid data format].
 
−
The geogrid data format consists of a directory of tiled binary files with names indicating the index range contained in each tile.
 
−
For instance, a file name of <tt>00101-00200.00051-00100</tt> would consist of columns 101 through 200 and rows 51 through 100.
 
−
The files themselves contain only an array integers, with no associated metadata.
 
−
The metadata for the dataset is contained in a file called <tt>index</tt>, which contains a series ''keyword=value'' statements
 
−
telling geogrid the details of the data tiles, such as width, height, the number of bytes per integer, etc.  The index file also contains metadata
 
−
for the data set itself, such as the projection information and physical units.
 
  
−
We have made available a series of scripts and utilities ([http://github.com/jbeezley/autoWPS autoWPS]) intended to automate many of the steps
+
This wiki page explains how to transform geotiff files to geogrid using convert_geotiff script in [https://github.com/openwfm/wrfxpy.git wrfxpy].
−
required to create a dataset for a WRF real run. A utility program included in this repository, WPSGeoTiff, is a simple command line utility that
 
−
takes a GeoTiff file and writes converts the data into geogrid binary format.  The GeoTiff format was chosen because it is a highly ubiquitous format
 
−
for geotagged data and it has a cross-platform and open source c library called [http://trac.osgeo.org/geotiff/ libGeoTIFF].  Most, if not all, GIS application
 
−
software is capable of converting or exporting a dataset as a GeoTiff file.
 
  
−
=Prerequisites=
+
== Get Anaconda3 distribution ==
  
−
The WPSGeoTiff utility requires two external libraries in order to compile.  You should pick a location to install the libraries, for instance <tt>${HOME}/opt</tt>.
+
Download and install the Python 3 [https://www.anaconda.com/products/individual Anaconda Python] distribution for your platform. We recommend an installation into the user's home directory.
−
* [http://www.libtiff.org/ libTIFF]
 
−
* [http://trac.osgeo.org/geotiff/ libGeoTIFF]
 
−
Both are written in c, and include standard GNU auto-configuration scripts.
 
  
−
=Obtaining the WPSGeoTiff source=
+
== Install environment ==
  
−
The autoWPS package is available as a git repository hosted at [http://github.com/jbeezley/autoWPS github].  The repository can be cloned with the
+
Create the python environment:
−
[http://git-scm.com/ git] command:
 
−
<code>git clone git://github.com/jbeezley/autoWPS.git</code>
 
−
A tarball of the latest release can also be obtained
 
−
at [http://github.com/jbeezley/autoWPS/tarball/master http://github.com/jbeezley/autoWPS/tarball/master].
 
  
−
=Compiling WPSGeoTiff=
+
<pre>
 +
conda update -n base -c defaults conda
 +
conda create -n wrfx python=3 gdal netcdf4 pyproj paramiko dill h5py psutil proj4 pytz flask pandas
 +
conda activate wrfx
 +
conda install -c conda-forge simplekml pygrib f90nml pyhdf xmltodict basemap rasterio scipy
 +
pip install MesoPy python-cmr
 +
</pre>
  
−
Inside the main directory of the autoWPS release is a subdirectory called WPSGeoTiff.
+
Note that <tt>conda</tt> and <tt>pip</tt> are package managers available in the Anaconda Python distribution.
 +
 
 +
== Get wrfxpy repository ==
 +
 
 +
Get repository wrfxpy and checkout to convert_geotiff branch. Shortly, it is going to be available in the master branch.
 +
git clone https://github.com/openwfm/wrfxpy
 +
cd wrfxpy
 +
git checkout convert_geotiff
 +
 
 +
== Run convert_geotiff.sh ==
 +
 
 +
Run the code to generate geogrid from geotiff files. Run the following line for each variable to process.
 +
./convert_geotiff.sh geotiff_file geo_data_path wrf_variable_name [bbox]
 +
 
 +
where:
 +
 
 +
* <tt>geotiff_file</tt> - path to a geotiff file to be converted to geogrid
 +
* <tt>geo_data_path</tt> - any path where to save all the variables
 +
* <tt>wrf_variable</tt> - WRF variable created on src/geo/var_wisdom.py
 +
* <tt>bbox</tt> - optional, bounding box to sample the original geotiff file from, using WGS84 coordinates. Format: min_lon,max_lon,min_lat,max_lat
 +
 
 +
If you want to know more information and the existing options for wrf_variable, run ./convert_geotiff.sh without inputs
 +
./convert_geotiff.sh
 +
 
 +
Some examples:
 +
./convert_geotiff.sh elevation.tif geo_data ZSF
 +
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT
 +
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT -112.8115,-112.1661,39.4820,39.9750
 +
 
 +
== Results from convert_geotiff.sh ==
 +
 
 +
The convert_geotiff.sh script will generate a folder specified by the user by <tt>geo_data_path</tt> with
 +
 
 +
* '''Variable folders:''' A folder for each variable previously run (ZSF and NFUEL_CAT folders) containing:
 +
** a geogrid file of the variable.
 +
** an index file of the variable.
 +
* '''geogrid_tbl.json:''' A JSON file with all the geogrid information for each variable.
 +
* '''GEOGRID.TBL:''' A text file with the information to add to GEOGRID.TBL (depends on src/geo/var_wisdom.py, so only copy what you need).
 +
* '''index.json:''' A JSON file with index information for each variable.
 +
 
 +
[[Category:WRF-Fire]]
 +
[[Category:Howtos|Convert data for Geogrid]]
 +
[[Category:Data]]

Latest revision as of 01:59, 17 February 2021

Back to the WRF-SFIRE user guide.

This wiki page explains how to transform geotiff files to geogrid using convert_geotiff script in wrfxpy.

Get Anaconda3 distribution

Download and install the Python 3 Anaconda Python distribution for your platform. We recommend an installation into the user's home directory.

Install environment

Create the python environment: 

 conda update -n base -c defaults conda
 conda create -n wrfx python=3 gdal netcdf4 pyproj paramiko dill h5py psutil proj4 pytz flask pandas
 conda activate wrfx
 conda install -c conda-forge simplekml pygrib f90nml pyhdf xmltodict basemap rasterio scipy
 pip install MesoPy python-cmr

Note that conda and pip are package managers available in the Anaconda Python distribution.

Get wrfxpy repository

Get repository wrfxpy and checkout to convert_geotiff branch. Shortly, it is going to be available in the master branch. 

git clone https://github.com/openwfm/wrfxpy
cd wrfxpy
git checkout convert_geotiff

Run convert_geotiff.sh

Run the code to generate geogrid from geotiff files. Run the following line for each variable to process. 

./convert_geotiff.sh geotiff_file geo_data_path wrf_variable_name [bbox]

where:

  • geotiff_file - path to a geotiff file to be converted to geogrid
  • geo_data_path - any path where to save all the variables
  • wrf_variable - WRF variable created on src/geo/var_wisdom.py
  • bbox - optional, bounding box to sample the original geotiff file from, using WGS84 coordinates. Format: min_lon,max_lon,min_lat,max_lat

If you want to know more information and the existing options for wrf_variable, run ./convert_geotiff.sh without inputs 

./convert_geotiff.sh

Some examples: 

./convert_geotiff.sh elevation.tif geo_data ZSF
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT
./convert_geotiff.sh fuel.tif geo_data NFUEL_CAT -112.8115,-112.1661,39.4820,39.9750

Results from convert_geotiff.sh

The convert_geotiff.sh script will generate a folder specified by the user by geo_data_path with

  • Variable folders: A folder for each variable previously run (ZSF and NFUEL_CAT folders) containing:
    • a geogrid file of the variable.
    • an index file of the variable.
  • geogrid_tbl.json: A JSON file with all the geogrid information for each variable.
  • GEOGRID.TBL: A text file with the information to add to GEOGRID.TBL (depends on src/geo/var_wisdom.py, so only copy what you need).
  • index.json: A JSON file with index information for each variable.
Retrieved from "http://wiki.openwfm.org/mediawiki/index.php?title=How_to_convert_data_for_Geogrid&oldid=4333"