Skip to content
/ SICE Public

Operational Sentinel-3 snow and ice products (SICE) - Geological Survey of Denmark and Greenland (GEUS)

License

Notifications You must be signed in to change notification settings

GEUS-SICE/SICE

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

https://img.shields.io/badge/License-GPLv3-blue.svg https://github.com/GEUS-SICE/SICE/workflows/CI/badge.svg

Table of Contents

Sentinel-3 (S3) Albedo Processing Pipeline

  1. Fetch S3 OLCI & SLSTR products
  2. Process OLCI & SLSTR using SNAP GPT
  3. Process using pySICE
  4. Build daily mosaic

The steps above are encapsulated in ./S3_wrapper.sh.

In more detail:

Fetch S3 OLCI & SLSTR products

Process with SNAP

  • This step is a combination of running GPT on ./S3.xml with ./S3_proc.sh.
  • Inputs: OLCI and SLSTR scenes
  • Outputs: A folder OLCI timestamp

pySICE

  • Run ./sice.py passing in one of the folders generated in the previous step.
  • WARNING: This step is slow, and may take > 24 hours.

Mosaic

  • As implemented in ./dm.sh and ./dm.grass.sh, for each day
    • Combine all the files from to form a mosaic
    • Mask out clouds, then…
    • When scenes overlap, use minimum SZA

Outputs

File NameDescriptionUnits
grain_diametermm
snow_specific_surface_aream2kg-1
ndsinormalized difference snow index
ndbinormalized difference blue ice index
auto_cloudin development
szasun zenith angle
vzaviewing zenith angle
saasun azimuth angle
vaaviewing azimuth angle
albedo_bb_planar_nirnear infrared broadband planar albedo700-2400 nm
albedo_bb_planar_swshortwave broadband planar albedo300-2400 nm
albedo_bb_planar_visvisible broadband planar albedo300-700 nm
albedo_bb_spherical_nirnear infrared broadband spherical albedo700-2400 nm
albedo_bb_spherical_swshortwave broadband spherical albedo300-2400 nm
albedo_bb_spherical_visvisible broadband spherical albedo300-700 nm
Oa01..21_reflectancetop of the atmosphere reflectance (OLCI bands 1 through 21)
rBRR_01..21bottom of the atmosphere surface reflectance (OLCI bands 1 through 21)
albedo_spectral_planar_01..21spectral planar albedo (OLCI bands 1 through 21)
diagnostic_retrievalSee next table.
Diagnostic CodeDescription
0clean snow
1polluted snow
6polluted snow for which r0 was calculated and not derived from observations
7polluted snow of calculated spherical albedo in bands 1 and 2 >0.98 reprocessed as clean snow
100sza>75, no retrival
102TOA reflectance at band 21 < 0.1, no retrieval
104grain_diameter < 0.1, no retrieval, potential cloud flag
-nimpossible to solve polluted snow albedo equation at band n

Debugging & Testing

Change the year and day for loops in ./S3_wrapper.sh to one year and one day. E.g.

for year in 2017; do # one year
  for doy in 227 180; do # two example days
  # loop contents
  done
done

Development Environment

This work is developed with the following software versions

Bash

bash --version
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Python

Run the following using ./SICE.yml in order to set up an identical Anaconda Python environment.

conda env create -f SICE.yml

And then run conda activate SICE to activate the SICE environment.

ResolvePackageNotFound error can be raised. In that case, run conda env export --no-builds > environment.yml instead.

Parallel

parallel --version
GNU parallel 20161222
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015,2016
Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.

Web site: http://www.gnu.org/software/parallel

When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'.

GRASS

grass --version 2>&1
GRASS GIS 7.4.0

Geographic Resources Analysis Support System (GRASS) is Copyright,
1999-2018 by the GRASS Development Team, and licensed under terms of the
GNU General Public License (GPL) version >=2.
 
This GRASS GIS 7.4.0 release is coordinated and produced by
the GRASS Development Team with contributions from all over the world.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.

GPT

~/local/snap/bin/gpt --diag
SNAP Release version 7.0
SNAP home: /home/kdm/local/snap/bin//..
SNAP debug: null
SNAP log level: null
Java home: /home/kdm/local/snap/jre
Java version: 1.8.0_202
Processors: 8
Max memory: 18.7 GB
Cache size: 1024.0 MB
Tile parallelism: 8
Tile size: 512 x 512 pixels

To configure your gpt memory usage:
Edit snap/bin/gpt.vmoptions

To configure your gpt cache size and parallelism:
Edit .snap/etc/snap.properties or gpt -c ${cachesize-in-GB}G -q ${parallelism} 

SNAP

SNAP extensions used in this toolchain are listed in SNAP_extensions.txt.

Misc Notes & Code Snippets

Generate ice mask

  • Use the icemask from BedMachine v3
grass72 -c EPSG:3413 ./Gtmp
r.in.gdal input=NetCDF:~/data/Greenland/Morlighem_2017/BedMachineGreenland-2017-09-20.nc:mask output=icemask

g.region raster=icemask
g.region res=500 -ap
g.region zoom=icemask

d.mon start=wx0
d.erase
d.rast icemask

r.mapcalc "mask = if(icemask == 4, null(), icemask)" --o
d.rast mask
g.region zoom=mask

r.out.gdal -c -m input=mask output=mask.tif type=Byte createopt=COMPRESS=DEFLATE --o
exit
trash Gtmp

Buffered ice mask

grass -c mask.tif ./Gtmp
r.in.gdal input=mask.tif output=mask
# 50 cells = 25 km @ 500 m
r.mapcalc "ice = if(mask == 2, 1, null())"
r.grow input=ice output=ice_grow radius=50 new=1

r.null ice_grow null=100
r.clump input=ice_grow output=clumps
r.stats -c clumps sort=asc
for ID in $(r.stats -c clumps sort=asc | head -n7 | cut -d" " -f1); do
  r.mapcalc "ice_grow = if(clumps == ${ID}, 1, ice_grow)" --o
done
r.null ice_grow setnull=100

r.out.gdal input=ice_grow output=ice_mask_buffer.tif format=GTiff type=Byte createopt="COMPRESS=DEFLATE"
exit
trash Gtmp

Footprint

Get GL outline by drawing in Google Earth, export KML, then:

ogrinfo -al GL_outline.kml  | grep LINESTRING | sed s/\ 0//g

Regional masks

Regional masks stored in the masks folder are clips from the ESA global Land Cover products (download here). Masks were downloaded for 2018 and are available at 300m and 1km resolutions ({region}_300m.tif and {region}_1km.tif, respectively). A description of the 22 ESA global Land Cover products can be found here.

Post-processing, analysis and visualisation

Post-processing, analysis and visualisation tools have been implemented in the SICE toolbox.

About

Operational Sentinel-3 snow and ice products (SICE) - Geological Survey of Denmark and Greenland (GEUS)

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •