Difference between revisions of "Fire Data Sources"

From openwfm
Jump to navigation Jump to search
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
Many sources of weather, terrain, and fuel data for initialization, assimilation, and verification are freely available through public sites, many in the form of files viewable in Google Earth. Fire ignition tim/location data and imagery of fire extent at later times for specific incidents is still generally in the form of custom datasets where research or suppression teams were available to collect it.
 
Many sources of weather, terrain, and fuel data for initialization, assimilation, and verification are freely available through public sites, many in the form of files viewable in Google Earth. Fire ignition tim/location data and imagery of fire extent at later times for specific incidents is still generally in the form of custom datasets where research or suppression teams were available to collect it.
  
 +
==Fuel Maps==
 +
 +
* LandFire http://landfire.cr.usgs.gov
 +
 +
==10-hr Dead Fuel Moisture==
 +
 +
* Remote Automated Weather Station (RAWS) data    http://raws.wrh.noaa.gov/roman/ discontinued
 +
* [https://github.com/openwfm/MesoPy/blob/master/MesoPy.py#L74 MesoPy] access to http://api.mesowest.net/v2/
 +
 +
==Live Fuel Moisture==
 +
 +
* http://wfas.net based on NVDI data
 +
 +
==Fire Ignition Time and Location Data==
 +
 +
* Custom
 +
 +
==Fire Perimeters==
 +
* Main NIFC Open Data Site: http://data-nifc.opendata.arcgis.com/
 +
 +
* Open IRWIN Incident Data (points): http://data-nifc.opendata.arcgis.com/datasets/68637d248eb24d0d853342cba02d4af7_0
 +
 +
* Open Perimeter Data (National Incident Feature Service): http://data-nifc.opendata.arcgis.com/datasets/wildfire-perimeters
 +
 +
* NWCG Geospatial Subcommittee (GSC): https://www.nwcg.gov/committees/geospatial-subcommittee
 +
 +
* GSC SOP’s: https://www.nwcg.gov/publications/pms936
 +
 +
* EGP: https://egp.nwcg.gov/egp/default.aspx
 +
 +
* Incident Information System http://inciweb.nwcg.gov/
 +
 +
* Geospatial Multi-Agency Coordination (GEOMAC) http://www.geomac.gov/ is decommissioned as of May 1, 2020 and links to NIFC
 +
 +
==Fire Detection==
 +
* Firemapper https://fsapps.nwcg.gov/afm/
 +
* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml
 +
* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite
 +
 +
=== HDF File Information ===
 +
HDF stands for Hierarchical Data Format, there are two current standards; HDF4 (.hdf extension) and HDF5 (.h5 extension). All of our sources for active fire data and geolocation data use HDF4, though this could change in the future.
 +
 +
==== Reading HDFs ====
 +
Matlab uses 'hdfread' to read a dataset into an array, requires filename and the path to the relevant data inside the hdf file. Here's an example call:
 +
 +
fire_mask = hdfread('MOD14.A2016266.1505.006.2016266215413.hdf', '/fire mask')
 +
 +
Where the filename is 'MOD14.A2016266.1505.006.2016266215413.hdf' (more on the naming convention later), and the path inside the hdf to the firemask data is '/fire mask'
 +
 +
More on matlab hdf here https://www.mathworks.com/help/matlab/ref/hdfread.html
 +
 +
Python information: http://hdfeos.org/software/pyhdf.php
  
−
==Surface weather data==  
+
==== Useful links: ====
 +
* HDF4 User Guide: https://support.hdfgroup.org/release4/doc/UsrGuide_html/UG_PDF.pdf
 +
* Software useful for browsing the data structure, finding data paths, etc.: http://www.giss.nasa.gov/tools/panoply/
 +
* A way to grab only parts of an HDF file from a server? : http://hdfeos.org/software/hdf5_handler.php (looks to require server side support, need to check if our sources support it)
  
  
−
* The Meteorological Assimilation Data Ingest System (MADIS) http://madis.noaa.gov/  compiles surface weather data from many networks (http://madis.noaa.gov/network_info.html).
+
=== MODIS Data: ===
 +
==== MOD14 Files: ====
 +
Level 2 active fire data (750(?)m resolution) from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that fire data is under MYD14 with the same properties). Contains 4 fields important for our purposes:
 +
* fire_mask: Array containing sensor data for each pixel. Classes:
 +
** 0 missing input data
 +
** 1 not processed (obsolete)
 +
** 2 not processed (obsolete)
 +
** 3 non-fire water
 +
** 4 cloud
 +
** 5 non-fire land
 +
** 6 unknown
 +
** 7 fire (low confidence)
 +
** 8 fire (nominal confidence)
 +
** 9 fire (high confidence)
 +
* FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.
 +
* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
 +
* FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.
 +
These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD14/
  
−
* Mesowest  http://mesowest.utah.edu/index.html Also supports layers with fire info.
+
Example Filename: 'MOD14.A2016266.0615.006.2016266201759.hdf' ~200KB - 800KB
  
−
* Remote Automated Weather Station (RAWS) data    http://raws.wrh.noaa.gov/roman/
+
[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf
  
−
* METAR, the international standard code format for hourly surface weather observations, data are available at  http://weather.noaa.gov/weather/metar.shtml
+
(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
  
−
* NOAA National Climate Data Center land based data http://www.ncdc.noaa.gov/oa/land.html
+
Geolocation metadata can be found at ftp://ladsweb.nascom.nasa.gov/geoMeta/6/TERRA
  
−
==Vertical profiles of atmospheric properties==
+
Download the file for the corresponding date and check to see if a granule you are considering lies within certain bounds. No analog for VIIRS files is currently known.
  
−
* Atmospheric soundings  http://weather.uwyo.edu/upperair/sounding.html
+
==== MOD03 Files: ====
 +
Level 1 geolocation data from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that geolocation data is under MYD03 with the same properties). Contains 2 fields important for our purposes:
 +
* Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask
 +
* Longitude: Array (same size as fire_mask) containing the longitude of each pixel
 +
These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD03/
  
−
* Atmospheric profiler data  http://weather.uwyo.edu/upperair/profiler/  (not working currently)
+
Example Filename: 'MOD03.A2016266.2210.006.2016267050406.hdf' ~28MB - 32MB
  
−
* NOAA National Climate Data Center Integrated Global Radiosonde Archive  http://www.ncdc.noaa.gov/oa/climate/igra/index.php
+
[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf
  
−
==Three-dimensional Weather Data==
+
(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
  
−
* The NOAA National Operational Model Archive and Distribution System (NOMADS) http://nomads.ncdc.noaa.gov/
+
=== VIIRS Data: ===
 +
==== VNP14IMG_NRT files: ====
 +
Level 2 active fire data (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 4 fields important for our purposes:
 +
* fire_mask: Array containing sensor data for each pixel. Classes (Note: these are slightly different than the MOD14 fire mask classes):  
 +
** 0 not-processed (non-zero QF)
 +
** 1 bowtie
 +
** 2 glint
 +
** 3 water
 +
** 4 clouds
 +
** 5 clear land
 +
** 6 unclassified fire pixel
 +
** 7 low confidence fire pixel
 +
** 8 nominal confidence fire pixel
 +
** 9 high confidence fire pixel
 +
* FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.
 +
* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
 +
* FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.
  
−
* NOAA National Center for Environmental Prediction model products  ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/
+
These can be found at: ftp://nrt3.modaps.eosdis.nasa.gov/allData/5000/VNP14IMG_NRT/
  
−
==Fuel Maps==
+
They are held behind a password wall, registration is easy at https://urs.earthdata.nasa.gov/users/new
 +
 
 +
Example Filename: 'VNP14IMG_NRT.A2016323.0748.001.hdf' ~ 0.5MB - 5MB
 +
 
 +
[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].hdf
 +
 
 +
(These files are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
  
−
* LandFire http://landfire.cr.usgs.gov
+
==== NPP_IMFTS_L1 files: ====
 +
Level 1 geolocation files (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 2 fields important for our purposes:
 +
* Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask
 +
* Longitude: Array (same size as fire_mask) containing the longitude of each pixel
 +
These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/5000/NPP_IMFTS_L1/
  
−
==10-hr Dead Fuel Moisture==
+
No password this time.
  
−
* Remote Automated Weather Station (RAWS) data    http://raws.wrh.noaa.gov/roman/
+
Example Filename: 'NPP_IMFTS_L1.A2016323.0930.001.2016323192201.hdf' ~ 1GB (yikes!)
  
−
==Live Fuel Moisture==
+
[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf
  
−
* Custom
+
(The files here are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
  
−
==Fire Ignition Time and Location Data==
+
=== Algorithms, investigations, etc ===
  
−
* Custom
+
This section will contain a discussion of the generation and structure of our files to be used with the wrfxpy
  
−
==Fire Perimeters==
+
Tools and websites to help locate the data...?
  
−
* Geospatial Multi-Agency Coordination (GEOMAC) http://www.geomac.gov/
+
To be continued...
−
* Firemapper http://firemapper.sc.egov.usda.gov/activefiremaps.php
 
−
* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml
 
−
* Incident Information System http://inciweb.nwcg.gov/
 
−
* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satelite
 
  
 
==Fire Location and Intensity Data==
 
==Fire Location and Intensity Data==
Line 59: Line 157:
  
 
[[Category:Data]]
 
[[Category:Data]]
 +
 +
==See Also==
 +
* [[Weather Data Sources]]

Latest revision as of 00:18, 23 August 2020

Many sources of weather, terrain, and fuel data for initialization, assimilation, and verification are freely available through public sites, many in the form of files viewable in Google Earth. Fire ignition tim/location data and imagery of fire extent at later times for specific incidents is still generally in the form of custom datasets where research or suppression teams were available to collect it.

Fuel Maps

10-hr Dead Fuel Moisture

Live Fuel Moisture

Fire Ignition Time and Location Data

  • Custom

Fire Perimeters

  • Geospatial Multi-Agency Coordination (GEOMAC) http://www.geomac.gov/ is decommissioned as of May 1, 2020 and links to NIFC

Fire Detection

HDF File Information

HDF stands for Hierarchical Data Format, there are two current standards; HDF4 (.hdf extension) and HDF5 (.h5 extension). All of our sources for active fire data and geolocation data use HDF4, though this could change in the future.

Reading HDFs

Matlab uses 'hdfread' to read a dataset into an array, requires filename and the path to the relevant data inside the hdf file. Here's an example call:

fire_mask = hdfread('MOD14.A2016266.1505.006.2016266215413.hdf', '/fire mask')

Where the filename is 'MOD14.A2016266.1505.006.2016266215413.hdf' (more on the naming convention later), and the path inside the hdf to the firemask data is '/fire mask'

More on matlab hdf here https://www.mathworks.com/help/matlab/ref/hdfread.html

Python information: http://hdfeos.org/software/pyhdf.php

Useful links:


MODIS Data:

MOD14 Files:

Level 2 active fire data (750(?)m resolution) from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that fire data is under MYD14 with the same properties). Contains 4 fields important for our purposes:

  • fire_mask: Array containing sensor data for each pixel. Classes:
    • 0 missing input data
    • 1 not processed (obsolete)
    • 2 not processed (obsolete)
    • 3 non-fire water
    • 4 cloud
    • 5 non-fire land
    • 6 unknown
    • 7 fire (low confidence)
    • 8 fire (nominal confidence)
    • 9 fire (high confidence)
  • FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.
  • FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
  • FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.

These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD14/

Example Filename: 'MOD14.A2016266.0615.006.2016266201759.hdf' ~200KB - 800KB

[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

Geolocation metadata can be found at ftp://ladsweb.nascom.nasa.gov/geoMeta/6/TERRA

Download the file for the corresponding date and check to see if a granule you are considering lies within certain bounds. No analog for VIIRS files is currently known.

MOD03 Files:

Level 1 geolocation data from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that geolocation data is under MYD03 with the same properties). Contains 2 fields important for our purposes:

  • Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask
  • Longitude: Array (same size as fire_mask) containing the longitude of each pixel

These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD03/

Example Filename: 'MOD03.A2016266.2210.006.2016267050406.hdf' ~28MB - 32MB

[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

VIIRS Data:

VNP14IMG_NRT files:

Level 2 active fire data (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 4 fields important for our purposes:

  • fire_mask: Array containing sensor data for each pixel. Classes (Note: these are slightly different than the MOD14 fire mask classes):
    • 0 not-processed (non-zero QF)
    • 1 bowtie
    • 2 glint
    • 3 water
    • 4 clouds
    • 5 clear land
    • 6 unclassified fire pixel
    • 7 low confidence fire pixel
    • 8 nominal confidence fire pixel
    • 9 high confidence fire pixel
  • FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.
  • FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
  • FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.

These can be found at: ftp://nrt3.modaps.eosdis.nasa.gov/allData/5000/VNP14IMG_NRT/

They are held behind a password wall, registration is easy at https://urs.earthdata.nasa.gov/users/new

Example Filename: 'VNP14IMG_NRT.A2016323.0748.001.hdf' ~ 0.5MB - 5MB

[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].hdf

(These files are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

NPP_IMFTS_L1 files:

Level 1 geolocation files (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 2 fields important for our purposes:

  • Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask
  • Longitude: Array (same size as fire_mask) containing the longitude of each pixel

These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/5000/NPP_IMFTS_L1/

No password this time.

Example Filename: 'NPP_IMFTS_L1.A2016323.0930.001.2016323192201.hdf' ~ 1GB (yikes!)

[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

(The files here are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

Algorithms, investigations, etc

This section will contain a discussion of the generation and structure of our files to be used with the wrfxpy

Tools and websites to help locate the data...?

To be continued...

Fire Location and Intensity Data

See Also

Retrieved from "http://wiki.openwfm.org/mediawiki/index.php?title=Fire_Data_Sources&oldid=4317"