Fire Data Sources - Revision history

Jmandel: /* Fire Perimeters */

2020-08-23T00:18:30Z

Fire Perimeters

@@ Line 19: / Line 19: @@
 ==Fire Perimeters==
 * Incident Information System http://inciweb.nwcg.gov/
 ==Fire Detection==

Jmandel: /* Fire Detection */ updated firemapper links

2017-05-23T16:07:29Z

Fire Detection: updated firemapper links

← Older revision		Revision as of 16:07, 23 May 2017
Line 24:		Line 24:

	==Fire Detection==		==Fire Detection==
	* Firemapper ~~http~~://~~firemapper~~.~~sc.egov.usda~~.gov/~~activefiremaps.php~~		* Firemapper https://fsapps.nwcg.gov/afm/
	* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml		* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml
	* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite		* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite

Dburke at 20:04, 8 February 2017

2017-02-08T20:04:55Z

← Older revision		Revision as of 20:04, 8 February 2017
Line 72:		Line 72:

	(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)		(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

			Geolocation metadata can be found at ftp://ladsweb.nascom.nasa.gov/geoMeta/6/TERRA

			Download the file for the corresponding date and check to see if a granule you are considering lies within certain bounds. No analog for VIIRS files is currently known.

	==== MOD03 Files: ====		==== MOD03 Files: ====

Dburke: /* Fire Detection */

2016-12-06T06:45:39Z

Fire Detection

← Older revision		Revision as of 06:45, 6 December 2016
Line 27:		Line 27:
	* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml		* Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml
	* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite		* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite

			=== HDF File Information ===
			HDF stands for Hierarchical Data Format, there are two current standards; HDF4 (.hdf extension) and HDF5 (.h5 extension). All of our sources for active fire data and geolocation data use HDF4, though this could change in the future.

			==== Reading HDFs ====
			Matlab uses 'hdfread' to read a dataset into an array, requires filename and the path to the relevant data inside the hdf file. Here's an example call:

			fire_mask = hdfread('MOD14.A2016266.1505.006.2016266215413.hdf', '/fire mask')

			Where the filename is 'MOD14.A2016266.1505.006.2016266215413.hdf' (more on the naming convention later), and the path inside the hdf to the firemask data is '/fire mask'

			More on matlab hdf here https://www.mathworks.com/help/matlab/ref/hdfread.html

			Python information: http://hdfeos.org/software/pyhdf.php

			==== Useful links: ====
			* HDF4 User Guide: https://support.hdfgroup.org/release4/doc/UsrGuide_html/UG_PDF.pdf
			* Software useful for browsing the data structure, finding data paths, etc.: http://www.giss.nasa.gov/tools/panoply/
			* A way to grab only parts of an HDF file from a server? : http://hdfeos.org/software/hdf5_handler.php (looks to require server side support, need to check if our sources support it)


	=== MODIS Data: ===		=== MODIS Data: ===
Line 45:		Line 65:
	* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.		* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
	* FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.		* FP_confidence: Variable length vector (the number of pixels identified as firepixels) containing confidence of fire identification for each firepixel in the mask.
	These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD14/		These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD14/

			Example Filename: 'MOD14.A2016266.0615.006.2016266201759.hdf' ~200KB - 800KB

			[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

	(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)		(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)

	==== MOD03 Files: ====		==== MOD03 Files: ====
	Level 1 geolocation data from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that ~~fire~~ data is under ~~MYD14~~ with the same properties). Contains 2 fields important for our purposes:		Level 1 geolocation data from the MODIS instrument on the Terra satellite (Aqua satellite also has a MODIS instrument, that geolocation data is under MYD03 with the same properties). Contains 2 fields important for our purposes:
	* Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask		* Latitude: Array (same size as fire_mask) containing the latitude of each pixel in the firemask
	* Longitude: Array (same size as fire_mask) containing the longitude of each pixel		* Longitude: Array (same size as fire_mask) containing the longitude of each pixel
	These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD03/		These can be found at: ftp://ladsweb.nascom.nasa.gov/allData/6/MOD03/

			Example Filename: 'MOD03.A2016266.2210.006.2016267050406.hdf' ~28MB - 32MB

			[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

	(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)		(The files here are delivered in 5 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
Line 60:		Line 88:
	==== VNP14IMG_NRT files: ====		==== VNP14IMG_NRT files: ====
	Level 2 active fire data (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 4 fields important for our purposes:		Level 2 active fire data (375m resolution) from the VIIRS instrument on the Suomi-NPP Satellite. Contains 4 fields important for our purposes:
	* fire_mask: Array containing sensor data for each pixel. Classes (Note: these are different than the MOD14 fire mask classes):		* fire_mask: Array containing sensor data for each pixel. Classes (Note: these are slightly different than the MOD14 fire mask classes):
	** 0 (~~will be added later~~)		** 0 not-processed (non-zero QF)
	** 1		** 1 bowtie
	** 2		** 2 glint
	** 3		** 3 water
	** 4		** 4 clouds
	** 5		** 5 clear land
	** 6		** 6 unclassified fire pixel
	** 7		** 7 low confidence fire pixel
	** 8		** 8 nominal confidence fire pixel
	** 9		** 9 high confidence fire pixel
	* FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.		* FP_latitude: Variable length vector (the number of pixels identified as firepixels) containing latitude of all firepixels in the mask.
	* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.		* FP_longitude: Variable length vector (the number of pixels identified as firepixels) containing longitude of all firepixels in the mask.
Line 78:		Line 106:

	They are held behind a password wall, registration is easy at https://urs.earthdata.nasa.gov/users/new		They are held behind a password wall, registration is easy at https://urs.earthdata.nasa.gov/users/new

			Example Filename: 'VNP14IMG_NRT.A2016323.0748.001.hdf' ~ 0.5MB - 5MB

			[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].hdf

	(These files are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)		(These files are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
Line 88:		Line 120:

	No password this time.		No password this time.

			Example Filename: 'NPP_IMFTS_L1.A2016323.0930.001.2016323192201.hdf' ~ 1GB (yikes!)

			[Dataset name].A[Year][DayofYear].[24hrTime].[Collection number].[collection time?].hdf

	(The files here are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)		(The files here are delivered in 6 minute intervals, and are organized by year, then by Julian Day Number (n corresponds to the nth day of the year).)
Line 95:		Line 131:
	This section will contain a discussion of the generation and structure of our files to be used with the wrfxpy		This section will contain a discussion of the generation and structure of our files to be used with the wrfxpy

	~~This section will contain extra information~~ to help ~~find other viirs/modis~~ data ~~and tools to view the internal structures of HDFs~~. ~~Later will add average file sizes and other info~~.		Tools and websites to help locate the data...?

	~~Perhaps a way to only grab parts of the HDFs from the servers~~? ~~http://hdfeos.org/software/hdf5_handler.php~~

	To be continued...		To be continued...

Dburke: /* Fire Detection */

2016-12-05T07:45:09Z

Fire Detection

@@ Line 26: / Line 26: @@
 * Firemapper http://firemapper.sc.egov.usda.gov/activefiremaps.php
 * Latest MODIS from Forest Service http://activefiremaps.fs.fed.us/data/kml/conus_latest_AFM_bundle.kml
-* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satelite
+* Modis raw data ftp://152.61.133.5/MOLT/MOD14.005 Tera, ftp://152.61.133.5/MOLA/MOD14.005 Aqua satellite
 ==Fire Location and Intensity Data==

Jmandel: /* Fire Location and Intensity Data */

2016-09-05T06:40:15Z

Fire Location and Intensity Data

← Older revision		Revision as of 06:40, 5 September 2016
Line 34:		Line 34:

	[[Category:Data]]		[[Category:Data]]

			==See Also==
			* [[Weather Data Sources]]

Jmandel: /* 10-hr Dead Fuel Moisture */

2016-09-05T06:39:34Z

10-hr Dead Fuel Moisture

← Older revision		Revision as of 06:39, 5 September 2016
Line 7:		Line 7:
	==10-hr Dead Fuel Moisture==		==10-hr Dead Fuel Moisture==

	* Remote Automated Weather Station (RAWS) data http://raws.wrh.noaa.gov/roman/		* Remote Automated Weather Station (RAWS) data http://raws.wrh.noaa.gov/roman/ discontinued
			* [https://github.com/openwfm/MesoPy/blob/master/MesoPy.py#L74 MesoPy] access to http://api.mesowest.net/v2/

	==Live Fuel Moisture==		==Live Fuel Moisture==

Jmandel: split off Weather Data Sources]

2016-09-05T04:48:19Z

split off Weather Data Sources]

← Older revision		Revision as of 04:48, 5 September 2016
Line 1:		Line 1:
	Many sources of weather, terrain, and fuel data for initialization, assimilation, and verification are freely available through public sites, many in the form of files viewable in Google Earth. Fire ignition tim/location data and imagery of fire extent at later times for specific incidents is still generally in the form of custom datasets where research or suppression teams were available to collect it.		Many sources of weather, terrain, and fuel data for initialization, assimilation, and verification are freely available through public sites, many in the form of files viewable in Google Earth. Fire ignition tim/location data and imagery of fire extent at later times for specific incidents is still generally in the form of custom datasets where research or suppression teams were available to collect it.


	~~==Surface weather data==~~


	* The Meteorological Assimilation Data Ingest System (MADIS) http://madis.noaa.gov/ compiles surface weather data from many networks (http://madis.noaa.gov/network_info.html).

	* Mesowest http://mesowest.utah.edu/index.html Also supports layers with fire info.

	* Remote Automated Weather Station (RAWS) data http://raws.wrh.noaa.gov/roman/

	* METAR, the international standard code format for hourly surface weather observations, data are available at http://weather.noaa.gov/weather/metar.shtml

	* NOAA National Climate Data Center land based data http://www.ncdc.noaa.gov/oa/land.html

	~~==Vertical profiles of atmospheric properties==~~

	* Atmospheric soundings http://weather.uwyo.edu/upperair/sounding.html

	* Atmospheric profiler data http://weather.uwyo.edu/upperair/profiler/ (not working currently)

	* NOAA National Climate Data Center Integrated Global Radiosonde Archive http://www.ncdc.noaa.gov/oa/climate/igra/index.php

	~~==3D Weather Data==~~

	* The NOAA National Operational Model Archive and Distribution System (NOMADS) http://nomads.ncdc.noaa.gov/

	* NOAA National Center for Environmental Prediction model products ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/

	~~===NARR===~~
	~~====Description====~~
	* North American Regional Reanalysis http://www.esrl.noaa.gov/psd/data/gridded/data.narr.html
	** 3h, 32km resolution 1979 to now about 18 month delay
	** [http://www.esrl.noaa.gov/psd/data/narr/warning.html Converted from GRIB to NetCDF4 (HDF5) as of 2016/02/11]

	~~====Data access====~~
	* http://nomads.ncdc.noaa.gov/data/narr
	* ftp://ftp.cdc.noaa.gov/Datasets/NARR

	==Fuel Maps==		==Fuel Maps==

Jmandel: Jmandel moved page Data sources to Fire Data Sources: split

2016-09-05T04:47:22Z

Jmandel moved page Data sources to Fire Data Sources: split

← Older revision	Revision as of 04:47, 5 September 2016
(No difference)

Jmandel: /* Description */

2016-09-05T04:38:17Z

Description

← Older revision		Revision as of 04:38, 5 September 2016
Line 32:		Line 32:
	====Description====		====Description====
	* North American Regional Reanalysis http://www.esrl.noaa.gov/psd/data/gridded/data.narr.html		* North American Regional Reanalysis http://www.esrl.noaa.gov/psd/data/gridded/data.narr.html
	** ~~8-times~~, ~~Daily and Monthly means for~~ 1979~~/01/01~~ to ~~present, 32km resolution~~		** 3h, 32km resolution 1979 to now about 18 month delay
	** [http://www.esrl.noaa.gov/psd/data/narr/warning.html Converted from GRIB to NetCDF4 (HDF5) as of 2016/02/11]		** [http://www.esrl.noaa.gov/psd/data/narr/warning.html Converted from GRIB to NetCDF4 (HDF5) as of 2016/02/11]

	====Data access====		====Data access====
	* http://nomads.ncdc.noaa.gov/data/narr		* http://nomads.ncdc.noaa.gov/data/narr