Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal for new BUFR table entries for EPS-SG (EUMETSAT) : IASI-NG Level 2 and VII Level 2 #161

Closed
antoinemerle opened this issue Apr 25, 2023 · 26 comments · Fixed by #174
Milestone

Comments

@antoinemerle
Copy link
Contributor

antoinemerle commented Apr 25, 2023

Initial request

This document proposes new BUFR entries for the following L2 products :

Product Presented to TT-TDCF group reviewed/corrected new code fixed code pushed Data sample provided Approved READY
Common IASI-NG L2 info R
IASI-NG L2 TWV R
IASI-NG L2 CLD R
IASI-NG L2 SFC R
IASI-NG L2 GHG still under investigation about pressure R
IASI-NG L2 CO_ R
IASI-NG L2 NAC and 03 R
IASI-NG L2 SO2 still under investigation about pressure R
VII L2 CLD R
VII L2 CTP R
VII L2 OCA R
VII L2 WVV and WVI R

This document proposes new BUFR entries for the following L2 products :

  • IASI-NG L2 TWV
  • IASI-NG L2 CLD
  • IASI-NG L2 SFC
  • IASI-NG L2 CO_
  • IASI-NG L2 GHG
  • IASI-NG L2 NAC
  • IASI-NG L2 O3_
  • IASI-NG L2 SO2
  • VII L2 CLD
  • VII L2 CLD
  • VII L2 CTP
  • VII L2 OCA
  • VII L2 WVV
  • VII L2 WVI

Amendment details

New entries in Table B :

Add the following elements to BUFR Table B :

F X Y Element name Unit Scale Reference value Bit width Instrument
0 12 083 Temperature difference between observed and computed K 2 -10000 16 IASI-NG L2
0 10 073 Height of plume m 2 0 21 IASI-NG L2
0 08 071 Type of cost estimation Code table 0 0 3 IASI-NG L2
0 12 169 Brightness temperature difference K 3 -100000 18 IASI-NG L2
0 15 007 Estimated amount of chemical constituent DU 2 0 16 IASI-NG L2
0 33 116 First guess in bounds check Flag Table 0 0 17 IASI-NG L2
0 33 117 In bounds check Flag Table 0 0 17 IASI-NG L2
0 40 080 Cloud test results for IASI-NG level 2 Flag Table 0 0 5 IASI-NG L2
0 40 075 Retrieval error covariance matrix for humidity in principal component domain Numeric 0 0 21 IASI-NG L2
0 40 076 Retrieval error covariance matrix for temperature in principal component domain Numeric 0 0 21 IASI-NG L2
0 25 144 Number of visual channels used in processing Numeric 0 0 6 VII-02-OCA
0 25 145 Number of infrared channels used in processing Numeric 0 0 6 VII-02-OCA
0 20 139 Final cloud model assumption Code table 0 0 4 VII-02-OCA
0 13 094 log 10 of Cloud optical thickness Numeric 3 -2000 14 VII-02-CTP,VII-02-OCA
0 33 118 Relative error percentage % 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage % 0 -100 8 VII-02-WVV, VII-02-WVI

New entries in existing code and flag table

  Code Table 0 08 072 :
Action Value New meaning
Add 5 Ash
Update 6 Reserved
  Code Table 0 08 029: Surface type
Add 19 Forests
Add 20 Shrublands
Add 21 Savanna
Add 22 Desert
Add 23 Grasslands
Add 24 Wetlands
Add 25 Snow/Ice
Add 26 Coast
Update 27 - 254 Reserved
  Code Table 0 33 028: Snapshot overall quality
Action Value New meaning
Add 5 General degradation
Update 6 Reserved

New flag tables and related code

Code Table 0 08 071 : Type of cost estimation
Value Meaning
0 First guess
1 Y vector
2 X vector
3 to 6 Reserved
7 Missing
Flag Table 0 33 116 : First guess in bounds check
Bit Meaning
1 Temperature profile
2 Water-vapour profile
3 Ozone profile
4 Surface skin temperature
5 Surface air temperature
6 Surface air dew point temperature
7 Surface pressure
8 Surface skin temperature error estimate(quality indicator)
9 Surface air temperature error estimate(quality indicator)
10 Surface air dew point temperature error estimate (quality indicator)
11 Surface pressure error estimate (quality indicator)
12 Surface emissivity coarse infrared spectrum
13 Surface emissivity microwave spectrum
14 CO2 profile
15 CH4 profile
16 N2O profile
All 17 Missing
Flag Table 0 33 117: In bounds check
Bit Meaning
1 Temperature profile
2 Water-vapour profile
3 Ozone profile
4 Surface skin temperature
5 Surface air temperature
6 Surface air dew point temperature
7 Surface pressure
8 Surface skin temperature error estimate(quality indicator)
9 Surface air temperature error estimate(quality indicator)
10 Surface air dew point temperature error estimate (quality indicator)
11 Surface pressure error estimate (quality indicator)
12 Surface emissivity coarse infrared spectrum
13 Surface emissivity microwave spectrum
14 CO2 profile
15 CH4 profile
16 N2O profile
All 17 Missing
Flag Table 0 40 080: Cloud test results for IASI-NG level 2
Bit Meaning
1 Thin cirrus detection executed
2 Thin cirrus detection indicates a cloud
3 Aerosol dust detection executed
4 Aerosol dust detected
All 5 Missing
Code Table 0 20 139 : Final cloud model assumption for VII L2
Value Meaning
0 Clear
1 Single layer water
2 Single layer ice
3 2-layers (water/ice)
4 single layer ash
5 2-layers ash
6-14 Reserved
15 Missing

New table D entries

New element Element name
3 10 087 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (TWV, CLD, SFC, GHG)
3 10 088 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV)
3 10 089 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud mask cloud detection (CLD)
3 10 090 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC)
3 10 091 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG)
3 10 092 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2)
3 10 093 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_)
3 10 094 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) and level 2 ozone total column, ozone profile (O3_)
3 10 095 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)
3 10 096 Metop-SG Visible Infrared Imager (VII), level 2 common part for VII L2 CLD, CTP, OCA, WVI, WVV
3 10 097 Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD)
3 10 098 Metop-SG Visible Infrared Imager (VII), level 2 Cloud top pressire (CTP)
3 10 099 Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA)
3 10 100 Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV)

IASI L2 - Common part for TWV,CLD, SFC, GHG from EUM-PRS

Add the following sequence 3 10 087 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 PRS common part for TWV,CLD,SFC,GHG

Here's the formatted table in Markdown:

Table references Element name Note
0 01 007 Satellite identifier
0 01 031 Identification of originating/generating centre
0 02 019 Satellite instruments
0 02 020 Satellite classification
0 04 001 Year
0 04 002 Month
0 04 003 Day
0 04 004 Hour
0 04 005 Minute
2 02 131 Change scale
2 01 138 Change data width
0 04 006 Second
2 01 000 Change data width
2 02 000 Change scale
0 05 001 Latitude (high accuracy)
0 06 001 Longitude (high accuracy)
0 07 024 Satellite zenith angle
0 05 021 Bearing or azimuth
0 07 025 Solar zenith angle
0 05 022 Solar azimuth
0 05 043 Field of view number
0 05 040 Orbit number
2 01 133 Change data width
0 05 041 Scan line number
2 01 000 Change data width
2 02 126 Change scale
0 07 001 Height of station
2 02 000 Change scale
0 40 052 Indication of super-adiabatic and super-saturation in final retrieval
0 40 047 Validation flag for IASI or IASI-NG level 1 product
0 08 065 Sun-glint indicator
2 04 004 Add associated field with 4 bits Adding 4 bits to give quality information on the data to be processed
0 31 021 Associated field significance 9 for specifying for each of the incoming variable the status of the data retrieved.
0 02 019 Satellite instruments For MetImage
0 02 019 Satellite instruments For the embedded MWS
0 01 030 Numerical model identifier For the numerical model ECMWF
2 04 000 Cancel associated field
0 02 019 Satellite instruments Back to instrument IASI-NG
0 40 046 Cloudiness summary
0 33 116 First guess in bounds check First guess means retrieved with PWLR method and It checks that the values are within the expected range for the correspondent geophysical parameter
0 33 117 In bounds check In bounds check means retrieved with OEM. It checks that the values are within the expected range for the correspondent geophysical parameter
0 40 080 Cloud test results for IASI-NG level 2
0 40 051 Convergence of the iterative retrieval
0 08 003 Vertical significance (satellite observations) Set to 0=surface
0 21 166 Land fraction
0 22 046 Sea ice fraction
0 10 001 Height
0 08 023 First-order statistics Standard deviation
0 10 001 Height
0 08 023 First-order statistics Cancel the first-order-statistics
0 08 003 Vertical significance (satellite observations) Cancel the vertical significance

IASI L2 - TWV

Details on 3 10 088

Add the following sequence 3 10 88: Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV)

Here's the formatted table in Markdown:

Table references Element name Note
3 10 087 IASI NG L2 common information part The common information shared for TWV, CLD, SFC and GHG
0 40 053 Number of iterations used for retrieval
1 02 003 Replication of 2 elements 3 times
0 08 071 Type of cost estimation
0 25 149 Optimal estimation cost
1 06 000 Delayed replication of 6 descriptor
0 31 000 Short delayed replication factor
1 01 000 Delayed replication of 1 descriptor
0 31 002 Extended delayed descriptor replication factor
0 40 076 Retrieval error covariance matrix for temperature in principal component domain
1 01 000 Delayed replication of 1 descriptor
0 31 002 Extended delayed descriptor replication factor
0 40 075 Retrieval error covariance matrix for humidity in principal component domain
0 40 067 Quality indicator for atmospheric temperature
0 40 066 Quality indicator for atmospheric water vapour
2 02 126 Change scale
0 13 098 Integrated water vapour (column)
2 02 000 Cancel
1 07 101 Replication of 7 elements 101 times For 101 pressure layer
2 02 131 Change scale
2 01 138 Change width
0 07 004 Pressure At a given pressure
2 01 000 Cancel
2 02 000 Cancel
0 12 101 Air temperature Air temperature at given pressure
0 13 001 Specific humidity Humidity at given pressure

note for the short delayed replication Basically we will have either or not a matrice of error for temperature and humidity for each point. While we are reading the error_data_index that is describing the index of the point : If we have 255 in the value of the error_data_index, that means there is no errors, we will put 0 for the shor delayed replication factor (meaning we skip the other descriptors bellow)
if we a number (corresponding to the current index) we will put 1 and then the 6 following descriptors will be replicated meaning there is actually some error for that particular point

IASI-NG L2 CLD

Details on 3 10 89

Add the following sequence 3 10 089 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud detection (CLD)
Here's the formatted table in Markdown:

Table references Element name Note
3 10 087 IASI NG L2 common information part The common information shared for TWV, CLD, SFC and GHG
0 12 083 Temperature difference between observed and computed Difference in Kelvin float number between the observed temperature and the computed temperature
0 08 043 Atmospheric chemical or physical constituent type Set to Dust
0 40 044 Dust index
0 08 043 Atmospheric chemical or physical constituent type Cancel Dust
0 20 056 Cloud phase Set to 1 for liquid
0 13 109 Ice/liquid water path Liquid mass
0 20 056 Cloud phase Set to 2 for ice
0 13 109 Ice/liquid water path Ice mass
0 08 003 Vertical significance (satellite observations) Set vertical significance to cloud top
1 09 002 Replication of 9 descriptors 2 times Loop other two layers of clouds (one at a first specific pressure and a second one at a second specific pressure)
0 20 056 Cloud phase Describe the physical state of the cloud
0 20 081 Cloud amount in segment Fractional cloud cover in percentage
2 02 131 Change scale
2 01 138 Change width
0 20 016 Pressure at top of cloud Pressure at cloud top
2 01 000 Cancel
2 02 000 Cancel
0 12 101 Air temperature Air temperature at cloud top
0 20 131 Effective radius of cloud hydrometeors Cloud drop effective radius at cloud top

IASI-NG L2 SFC

Details on 3 10 090

Add the following sequence 3 10 090 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC)

Here's the generated Markdown for the table:

Table references Element name Note
3 10 087 IASI NG L2 common information part The common information shared for TWV, CLD, SFC and GHG
0 08 003 Vertical significance (satellite observations) Set Vertical significance to surface
0 40 067 Quality indicator for atmospheric temperature
0 40 066 Quality indicator for atmospheric water vapour
1 02 012 Replication of 02 descriptors 12 times 12 channels
0 05 042 Channel number Number of channels
0 14 050 Emissivity Emissivity at surface
0 12 001 Temperature / air temperature Air temperature at surface
2 02 131 Change scale
2 01 138 Change width
0 10 004 Pressure Pressure at surface
2 01 000 Cancel
2 02 000 Cancel
0 08 003 Vertical significance (satellite observations) Cancel Vertical Significance
0 12 083 Temperature difference between observed and computed Difference between the observed temperature and the computed temperature
0 11 012 Wind speed at 10m
0 08 043 Atmospheric chemical or physical constituent type Set Atmospheric chemical to Dust
0 40 044 Dust index
0 08 043 Atmospheric chemical or physical constituent type Cancel Dust

IASI-NG L2 GHG

Details on 3 10 091

Add the following sequence 3 10 091 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG)

Table references Element name Note
3 10 087 IASI NG L2 common information part The common information shared for TWV, CLD, SFC and GHG
1 04 003 Replicate 4 descriptors 3 times Loop other the 3 different chemical constituent type
0 08 043 Atmospheric chemical or physical constituent type N20,CH4 and CO2
1 02 007 Replicate 2 descriptors 7 times
0 07 004 Pressure At a given pressure
0 15 021 Integrated mass density

IASI L2 - Common part for CO_, NAC, O3, SO2 from EUM-PRS

Add the following sequence 3 10 092 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2)

Table references Element name note
0 01 007 Satellite identifier
0 25 060 Software identification
0 01 031 Identification of originating/generating centre
0 02 019 Satellite instruments
0 02 020 Satellite classification
0 04 001 Year
0 04 002 Month
0 04 003 Day
0 04 004 Hour
0 04 005 Minute
2 02 131 Change scale
2 01 138 Change data width
0 04 006 Second
2 01 000 Change data width
2 02 000 Change scale
0 05 001 Latitude (high accuracy) Geocentric latitude at sounder pixel centre
0 06 001 Longitude (high accuracy) Geocentric longitude at sounder pixel centre
0 07 024 Satellite zenith angle Measurement zenith angle at sounder pixel centre
0 05 021 Bearing or azimuth Measurement azimuth angle at sounder pixel centre
0 07 025 Solar zenith angle Solar zenith angle at sounder pixel centre
0 05 022 Solar azimuth Solar azimuth angle at sounder pixel centre
0 05 043 Field of view number Field of view number
0 05 040 Orbit number Orbit number
2 01 133 Change data width
0 05 041 Scan line number Scan line number
2 01 000 Change data width
2 02 126 Change scale
0 10 007 Height Altitude of surface
2 02 000 Change scale

IASI-NG L2 CO_

Details on 3 10 093

Add the following sequence 3 10 093 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_)

Table references Element name note
3 10 092 IASI NG L2 common information part The common information shared for CO, NAC, O3 and SO2.
0 08 046 Atmospheric chemical or physical constituent type 4 Carbon monoxide CO
0 15 021 Integrated mass density Mass content of Carbon monoxide
0 40 056 General retrieval quality General retrieval quality flag
0 40 058 Number of vectors describing the char. matrices Number of vectors describing the characterization matrices
0 40 059 Number of layers actually retrieved Number of layers actually retrieved
0 40 060 Number of profiles retrieved in scanline Number of for carbon monoxide profiles retrieved in scanline
0 40 057 IASI level 2 retrieval flags IASI2 retrieval flag containing the 040054 and 04055 flag
1 03 019 Repeat 3 descriptors 19 times Loop other 19 column
0 40 061 Air partial columns on each retrieved layer molecules/cm
0 40 062 A-priori partial columns on each retrieved layer Molecules/cm
0 40 063 Scaling vector multiplying the a priori vector in order to define the retrieved vector Scaling vector multiplying the a priori CO vector in order to define the retrieved CO vector.
1 01 190 Repeat1 descriptor 190 times
0 40 065 Main eigenvectors of the sensitivity matrix Main eigenvectors of the sensitivity matrix

IASI-NG L2 NAC and IASI-NG L2 O3

Details on 3 10 094

Add the following sequence 3 10 094 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) and level 2 ozone total column, ozone profile (O3_)

Table references Element name note
3 10 092 IASI NG L2 common information part The common information shared for CO, NAC, O3 and SO2.
0 08 046 Atmospheric chemical or physical constituent type Here code for ozone(O3) or nitric acid (HNO3)
0 15 021 Integrated mass density Mass content of nitric acid or of ozone
0 40 056 General retrieval quality flag General retrieval quality flag
0 40 058 Number of vectors describing the char. matrices Number of vectors describing the characterization matrices
0 40 059 Number of layers actually retrieved Number of layers actually retrieved
0 40 060 Number of profiles retrieved in scanline Number of HNO3 or O3 profiles retrieved in scanline
0 40 057 IASI level 2 retrieval flags IASI 2 retrieval flag containing the 040054 and 04055 flag
1 03 041 Repeat 3 descriptor 41 times Loop other each layer
0 40 061 Air partial columns on each retrieved layer Air partial columns on each retrieved layer molecules / cm2
0 40 062 A-priori partial columns for each retrieved layer A-priori partial columns for HNO3 or O3 in each retrieved layer
0 40 063 Scaling vector multiplying the a priori vector in order to define the retrieved vector Scaling vector multiplying the a priori HNO3 or O3 vector in order to define the retrieved HNO3 or O3 vector.
1 02 021 Repeat 2 descriptor 21 times Layer repeat 861
1 01 041 Repeat 1 descriptor 41 times
0 40 065 Main eigenvectors of the sensitivity matrix Main eigenvectors of the sensitivity matrix

IASI-NG L2 SO2

Details on 3 10 095

Add the following sequence 3 10 095 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)

Table references Element name note
3 10 092 IASI NG L2 common information part The common information shared for CO, NAC, O3 and SO2.
0 08 046 Atmospheric chemical or physical constituent type 8=SO2
0 40 068 General retrieval quality flag for SO2 General retrieval quality flag for SO2
0 10 073 Height of plume Altitude of the plume
0 15 007 Estimated amount of chemical constituent SO2 amount at the retrieved plume altitude
0 12 169 Brightness temperature difference Indicative brightness temperature difference in K
1 02 005 Repeat 2 descriptor 5 times 5 different altitude / pressure
0 07 004 Pressure Static pressure at different altitude
0 15 007 Estimated amount of chemical constituent Amount of chemical constituent (DU)

VII L2 common information part

Add the following sequence 3 10 096 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)

Table references Element name note
3 01 005 Originating centre/sub-centre
0 01 007 Satellite identifier
0 02 019 Satellite Instruments
0 02 020 Satellite classification
0 25 060 Software identification
3 01 011 Year, month, day
3 01 013 Hour, minute, second
0 05 040 Orbit number
0 05 001 Latitude (high accuracy)
0 06 001 Longitude (high accuracy)
2 02 126 Change scale
0 07 001 Height of Station
2 02 000 Cancel
0 07 024 Satellite zenith angle
0 05 021 Bearing or azimuth (degree true)
0 07 025 Solar zenith angle
0 05 022 Solar azimuth (degree true)
0 33 028 Snapshot overall quality Processing condition indicator (0=nominal, 1=General degradation)

VII L2 CLD

Details on 3 10 097

Add the following sequence 3 10 097 : Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD)

Table references Element name note
3 10 096 VII L2 common information part The common information shared for CLD, CTP, OCA, WVV and WVI
0 08 011 Meteorological feature Set to 11 to specify we are speaking about cloud clear
0 33 007 Per cent confidence Pixel clear-sky confidence
0 08 011 Meteorological feature Set to cancel
0 08 072 Pixel type Cloudy flag
0 08 029 Pixel surface type Surface type

VII L2 CTP

Details on 3 10 098

Add the following sequence 3 10 098 : Metop-SG Visible Infrared Imager (VII), level 2 Cloud Top Pressure (CTP)

Table references Element name note
3 10 096 VII L2 common information part The common information shared for CLD, CTP, OCA, WVV and WVI
0 13 094 Log 10 of Cloud optical thickness Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Pressure at cloud top
0 20 014 Height of top of cloud Height at cloud top
0 08 093 Measurement uncertainty significance Total uncertainty
0 13 094 Log 10 of Cloud optical thickness Error on Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Error on pressure at cloud top
0 20 014 Height of top of cloud Error on Height at cloud top
0 08 093 Measurement uncertainty significance Cancel total uncertainty

VII L2 OCA

Details on 3 10 099

Add the following sequence 3 10 099 : Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA)

Table references Element name note
3 10 096 VII L2 common information part The common information shared for CLD, CTP, OCA, WVV and WVI
0 25 144 Number of visual channels used in processing From 0 to 11
0 25 145 Number of infrared channels used in processing From 5 to 9
0 20 139 Final cloud model assumption Final cloud model assumption: 0=clear ,1=single layer water,2=single layer ice,3=2-layers (water/ice),4=single layer ash,5=2-layers ash
0 08 029 Pixel surface type Surface type
1 20 002 Repeat 20 descriptors 2 times Loop other first layer or/and second layer
0 08 002 Vertical significance (surface observations) On first iteration : 21 First instrument detected cloud layer, for second iteration : 22 Second instrument detected cloud layer
0 13 094 Log 10 of Cloud optical thickness Cloud optical thickness of single or top layer in Log10
0 20 016 Pressure at top of cloud Pressure at cloud top
0 12 001 Temperature Temperature at cloud top
2 02 131 Add 3 to scale
2 01 139 Add 11 to width
0 20 131 Effective radius of cloud hydrometeors Cloud particle effective radius
2 01 000 Cancel
2 02 000 Cancel
0 20 014 Height of top of cloud Height at cloud top
0 08 093 Measurement uncertainty significance Total uncertainty
0 13 094 Log 10 of Cloud optical thickness Error on cloud optical thickness in log 10
0 20 016 Pressure at top of cloud Error on Pressure at top cloud
2 02 131 Add 3 to scale
2 01 139 Add 11 to width
0 20 131 Effective radius of cloud hydrometeors Error on cloud particle effective radius
2 01 000 Cancel
2 02 000 Cancel
0 20 014 Height of top of cloud Error on height at cloud top
0 08 093 Measurement uncertainty significance Cancel total uncertainty

VII L2 WVI/WVV

Details on 3 10 100

Add the following sequence 3 10 100 : Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV)

Table references Element name note
3 10 096 VII L2 common information part The common information shared for CLD, CTP, OCA, WVV and WVI
2 02 129 Add 1 to scale
2 01 132 Add 4 to width
0 13 060 Total accumulated precipitation
2 01 000 Cancel
2 02 000 Cancel
0 33 118 Relative error percentage Relative error on total accumulated precipitation water
0 33 119 Forecast error percentage Relative difference between retrieved TPW and forecast TPW

Acronyms :

ADC | Analog-to-digital converter
ASCAT | Advanced Scatterometer (190 in Common Code Table C-8)
DEM | Digital elevation model
EOP | Earth orientation parameters
EPS-SG | EUMETSAT Polar System – Second Generation
EUMETSAT | European Organisation for the Exploitation of Meteorological Satellites
FOVs | Fields of view
IASI-NG | Infrared Atmospheric Sounding Interferometer - New Generation (231 in Common Code Table C-8)
ICI | Ice Cloud Imager (236 in Common Code Table C-8)
IERS | International Earth Rotation Service
L0 | Level 0 (raw data)
L2 | Level 2 (geophysical data)
LOS | Line of sight
LSM | Landform segmentation model
MR | Main reflector
MWI | Microwave Imager (237 in Common Code Table C-8)
MWS | Microwave Sounder (233 in Common Code Table C-8)
NAVATT | Navigation and attitude data
NEDT | Nose equivalent delta temperature
NEdT | Nose equivalent delta temperature
NEΔT | Nose equivalent delta temperature
OBCT | On board calibration target
PCS | Principle component scores
PRT | Platinum resistance thermometer
RAD | Radiances
RFI | Radio frequency interference
RMS | Root mean square
SCA | Scatterometer (238 in Common Code Table C-8)
SVR | Resampled at 12.5 km swath grid
SZF | Resampled at full resolution
THM | Thermistor
TM | Telemetry
TWV | IASI-NG temperature and humidity profiles
YPR | Yaw, pitch, and roll
YSM | Yaw steering mode

Comments

No response

Requestor(s)

EUMETSAT :

  • Simon Elliott
  • Antoine MERLE

Stakeholder(s)

ECMWF
WMO
ECCODES

Publication(s)

New entries in : Table B
New entries in : Table D
New entries in : Flag Table
New entries to existing flag table

Expected impact of change

MEDIUM

Collaborators

EUMETSAT scientists

References

No response

Validation

No response

@antoinemerle antoinemerle added this to the FT2023-2 milestone Apr 25, 2023
@amilan17
Copy link
Member

amilan17 commented Apr 26, 2023

https://github.com/wmo-im/CCT/wiki/Teleconference.25.26.April.2023 notes:

  • Antoine introduced the proposal;

  • TWV review

    • (Marijana) asked for clarifications on several elements; The "in bounds check" might fit somewhere else, e.g. significance table [..?..] [ACTION] Antoine double check how many fields for "Satellite instruments" are needed and add notes;
    • (Sergio) questions about pressures and the relationship to air temperature[..?..] (Antoine) it's for describing the range of pressure; [ACTION] Antoine will add significance descriptors (Sergio) maybe Integrated water vapour be moved to after pressure and before air temperature? (Antoine) no, it needs to be outside (Marijana) it would be useful to add "layer number" in the descriptor [ACTION] Antoine will research this option.
    • (Sybille or Marijana) Missing maybe a descriptors' significance for air temperature humidity to explicitly says what it corresponds too (max temp).
  • CLD review

    • (Antoine) everything is the same up to as TWV until atmospheric profile
    • (Marjana) Add a significance on the replications of the 2 n_cld to explicit explain what those 2 level correspond too , Antoine will get close to the EUM scientist to find that info and if needed will create a new code table for that one
  • SFC overview

    • (Sibylle) Pressure should be from category 10, not cat 07, because this is the measurement of the pressure. Cat 07 is to indicate pressure level at a location and here it's not the location, but the measurement; [ACTION] Antoine change Pressure sequence

[ACTION] Anna coordinate a side meeting to discuss further details with Antoine, Simon, Marijana, Sibylle, Sergio and other invitees as desired after Antoine indicates that it's available.

Edit : Antoine fixed a typo in the list displayed in markdown langage.
Edit 2 : Antoine added so miss reco on CLD and TWV

@amilan17
Copy link
Member

amilan17 commented May 2, 2023

https://github.com/wmo-im/CCT/wiki/Teleconference.2&3May.2023 notes:
Antoine presented GHG and CO, but work is still in progress on feedback received during last meeting.
@amilan17 will try to set up mtg for 15 May week.

@antoinemerle
Copy link
Contributor Author

antoinemerle commented May 15, 2023

TWV review

(Marijana) asked for clarifications on several elements; The "in bounds check" might fit somewhere else, e.g. significance table [..?..] [ACTION] Antoine double check how many fields for "Satellite instruments" are needed and add notes;

  • Add explaination to why do we have 3 times sat identification
  • "in bounds check" : Will add a note in colomn D
  • "in bounds first guess chedk" : Will add a note in colomn D

(Sergio) questions about pressures and the relationship to air temperature

  • Antoine changed the sequence after the discussion with the EUM scientists. only 101 pressure not layers, no need to add a descriptor number layer because the pressure is there for that

CLD review

(Antoine) everything is the same up to as TWV until atmospheric profile
(Marjana) Add a significance on the replications of the 2 n_cld to explicit explain what those 2 level correspond too , Antoine will get close to the EUM scientist to find that info and if needed will create a new code table for that one

  • Antoine : I could have use class 10 but one was even better 0 20 016 : Pressure at top of cloud with exactly the same scale ref and width as 07
  • : add also CLoud phase 0 20 056 in front of the ice/liquid water path to add more information to what type of mass content we are getting.

SFC overview

(Sibylle) Pressure should be from category 10, not cat 07, because this is the measurement of the pressure. Cat 07 is to indicate pressure level at a location and here it's not the location, but the measurement;

  • Antoine change Pressure code from category 07 to 10 : new code is 0 10 004
  • Antoine also move the vertical significance in order to save some bits in the sequence

@amilan17
Copy link
Member

amilan17 commented May 16, 2023

https://github.com/wmo-im/CCT/wiki/16.May.2023.satellite.issues/ notes:
Antoine provided an overview of recent changes based on feedback and described in comment above.

@antoinemerle actions:

  • add descriptors for 0 15 21 Integrated mass density that indicates the pressure layer for GHG[?]
  • update first order statistic code
  • change name of 3 10 096 from "temperature" to "pressure"
  • will try to provide BUFR sample data for the entries discussed so far before 6 June; noting that Sibylle will only be able to validate in July;
  • Sergio will also try to validate before and after his June holidays;

@antoinemerle
Copy link
Contributor Author

antoinemerle commented May 17, 2023

Following the meeting of 16/05/2023 :

Hi @marijanacrepulja and @amilan17 : I updated the proposition according to the presentation I made yesterday.
You can now see in the current status the correction applied.


GHG

📝 summary

(Marjana) : for GHG Replicate 1 descriptors 5 times , how the users can know?
(Antoine) : corresponding to different layers
( Marjana) :maybe using a pressure descriptors or altitude would be needed
(Antoine) : I will investigate on our side and speak to our scientist.

🚀 action :

  • Antoine to contact scientist and make a decision for that : adding pressure descriptor ?

TWV

📝 : summary

(Sybille) : TWV check the code for first order statistics maybe we can use another one (for TWV, CLD, SFC..etc)

🚀 action :

  • : Antoine needs to investigate on that possibility to use this 008093 : Measurement uncertainty significance instead of using 0 08 023 : First-order statistics and to add a new element 15 as Algorithm uncertainty

VII L2 CTP

📝 : summary

(Sybille) : Later in the proposition CTP for VII, title says temperature but there is no temperature....
(Simon) : Yes that's a typo from our side, it is pressure

🚀 action :

  • : Antoine will correct the propositions and official document and title to pressure

@marijanacrepulja
Copy link
Collaborator

Hi @antoinemerle and @SimonElliottEUM

I had a chat with scientist and it would be beneficial to have following descriptors in the templates for O3, CO, SO2:

  • Information about cloud cover, descriptor 020010
  • Information about cloud top height, descriptor 020016
  • Information about surface type/emissivity, descriptor 013040
  • The pressure values (in Pa) at the layer boundaries
  • Averaging kernels informations, descriptors 015043, 015044 and related pressure 010004

We would appreciate if you can add the pressure [Pa] of the plume alitude in addition to 0 07 XXA 'Height of the plume' for SO2 BUFR template.

Many thanks

@antoinemerle
Copy link
Contributor Author

antoinemerle commented May 24, 2023

Hi @marijanacrepulja,

Just to let you know that I asked those questions to our scientists, and I am waiting their feedback.
I am going to update the sequences accordingly to what they can provide to us.

Only one quick question concerning the last bullet point :

Averaging kernels informations, descriptors 015043, 015044 and related pressure 010004
Can you describe what you are excepting there exactly ? I am 100% sure that I understood your request.

Thanks for your feedback and review, @marijanacrepulja !

Best,

@marijanacrepulja
Copy link
Collaborator

Hi @antoinemerle and @SimonElliottEUM

Many thanks for feedback.

As for averaging kernels and related pressure, we would need as described below:

015043      numberOfAveragingKernelLayers
102000      102000 Replicate 2 descriptors     
031001      delayedDescriptorReplicationFactor  
010004      nonCoordinatePressure   
015044      averagingKernelValue   

@antoinemerle
Copy link
Contributor Author

Hi @marijanacrepulja ,

On GHG

  • I updated GHG according to your comments

on O3, CO and SO2 :

In the case of IASI-NG, unlike IASI, the products are multiple files and not just one, so not all the variables are present in all the products. Unfortunately we don't have what you request in the given product, however it can be find in :

  • Information about cloud cover : % coverage of cloud : Available in IAS-02-CLD
  • Information about cloud top height : Pressure at cloud top : Available in IAS-02-CLD
  • Information about surface type : type of surface (land, sea, ice..etc) : Variables land_fraction and ice_fraction are available in several IASI-02 data such as IASI-02-TWV land_fraction

Still under investigation

  • The pressure values (in Pa) at the layer boundaries (it may be found as static variable)
  • Averaging kernels information, descriptors 015043, 015044 and related pressure 010004 : The closest concept we have to an averaging kernel are the of the sensitivity matrix eigenvectors, I am still investigating on that one.

Only for SO2 :

Still under investigation but I decided to add the pressure because it is likely to be available as static variable.

@antoinemerle
Copy link
Contributor Author

HI @SibylleK,

Following your comments on : TWV.

Antoine needs to investigate on that possibility to use this 008093 : Measurement uncertainty significance instead of using 0 08 023 : First-order statistics and to add a new element 15 as Algorithm uncertainty

First : THanks a lot for the comment and by investigating I found out it was not for TWV but on VII L2 products.
Here is the impacted part of the sequence :

Table references Element name note
0 13 165 log 10 of Cloud optical thickness Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Pressure at cloud top
0 20 014 Height of top of cloud Temperature at cloud top
0 08 023 First order statistics Specifying we are speaking about : Error / algorithm uncertainty on the following descriptors :
0 13 165 log 10 of Cloud optical thickness Error on Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Error on Pressure at cloud top
0 20 014 Height of top of cloud Error on Temperature at cloud top
0 08 023 First order statistics Missing

It is a good remark, and I would be happy to avoid creating a new line in this code table if we can use a different descriptor.
Basically what the scientists explained to me is :
the first iteration is about the geophysical quantity retrieved and then we have its associated algorithm uncertainty also retrieved.

Maybe we can use your descriptors and use the value 2 Random component of uncertainty or maybe another descriptors would be even better ? What do you think ?

Thanks a million for your help @SibylleK

@marijanacrepulja
Copy link
Collaborator

marijanacrepulja commented Jun 5, 2023

Hi @antoinemerle and @SimonElliottEUM

Many thanks for the feedback.

As for GHG

Could you please investigate is it possible to include informations for the a -priori and the average kernels.

  1. We would need for the a -priori
    Pressure
    SignificandOfVolumetricMixingRatio

  2. We would need the Error in the integratedMassDensity

  3. We need pressure at the top and at the bottom of the layer for the
    0 15 021 | Integrated mass density

@amilan17
Copy link
Member

amilan17 commented Jun 6, 2023

https://github.com/wmo-im/CCT/wiki/Teleconference.6.7.June.2023 notes:
Antoine presented the remaining products;

  • @antoinemerle will confirm if pressure is in the middle of the column;
  • need add a specific code for algorithm uncertainty in 008023 or other table?
  • @antoinemerle share sample date in the WMO SharePoint (to be included)

@antoinemerle
Copy link
Contributor Author

antoinemerle commented Jun 6, 2023

Dear @amilan17 ,

Following the meeting 06/06/2023 : and last comment

  • @antoinemerle : Update the VII/WVI sequence (typo in the github proposal)
  • @antoinemerle : make a decision on how to deal with "Algorithm uncertainty" ==>After discussing with the science team here in EUMETSAT they definitely request to use the suggestion solution made by SYbille which is using 0 08 093 to measure the ucnertainty. changing the sequence accordingly.
  • @antoinemerle : typo in the WMO note column : 0 20 014 | Height of top of cloud | Temperature at cloud top. it should be 0 20 014 | Height of top of cloud | Height at cloud top
  • @antoinemerle to push the issue on the Github before end of June
  • @antoinemerle to provide the sample data for IASI L2 and VII L2
  • @antoinemerle : For GHG and SO2 : need to explain if the specific pressure is the pressure average between the top and bottom of the column/plume (from @marijanacrepulja )
  • @amilan17 to give the URL of the SharePoint where Antoine can drop the data

thanks,

@antoinemerle
Copy link
Contributor Author

antoinemerle commented Jun 6, 2023

@amilan17 ,

Dear @amilan17 : bellow and in the initial request I updated the code to be the following :

TABLE B

F X Y Element name Unit Scale Reference value Bit width Instrument
0 12 083 Temperature difference between observed and computed K 2 -10000 16 IASI-NG L2
0 10 073 Height of plume m 2 0 21 IASI-NG L2
0 08 071 Type of cost estimation Code table 0 0 3 IASI-NG L2
0 12 169 Brightness temperature difference K 3 -100000 18 IASI-NG L2
0 15 007 Estimated amount of chemical constituent DU 2 0 16 IASI-NG L2
0 33 116 First guess in bounds check Flag Table 0 0 17 IASI-NG L2
0 33 117 In bounds check Flag Table 0 0 17 IASI-NG L2
0 40 080 Cloud test results for IASI-NG level 2 Flag Table 0 0 5 IASI-NG L2
0 40 075 Retrieval error covariance matrix for humidity in principal component domain Numeric 0 0 21 IASI-NG L2
0 40 076 Retrieval error covariance matrix for temperature in principal component domain Numeric 0 0 21 IASI-NG L2
0 25 144 Number of visual channels used in processing Numeric 0 0 6 VII-02-OCA
0 25 145 Number of infrared channels used in processing Numeric 0 0 6 VII-02-OCA
0 20 139 Final cloud model assumption Code table 0 0 4 VII-02-OCA
0 13 094 log 10 of Cloud optical thickness Numeric 3 -2000 14 VII-02-CTP,VII-02-OCA
0 33 118 Relative error percentage Percentage 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage Percentage 0 -100 8 VII-02-WVV, VII-02-WVI

TABLE D

New element Element name
3 10 087 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (TWV, CLD, SFC, GHG)
3 10 088 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV)
3 10 089 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud detection (CLD)
3 10 090 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC)
3 10 091 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG)
3 10 092 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2)
3 10 093 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_)
3 10 094 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) / level 2 ozone total column, ozone profile (O3_)
3 10 095 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)
3 10 096 Metop-SG Visible Infrared Imager (VII), level 2 common part for VII L2 CLD, CTP, OCA, WVI, WVV
3 10 097 Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD)
3 10 098 Metop-SG Visible Infrared Imager (VII), level 2 Cloud top pressire (CTP)
3 10 099 Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA)
3 10 100 Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV)

can you confirm this is okay for you thanks a million !

antoinemerle added a commit that referenced this issue Jun 26, 2023
…new products : related to IASI L2 and VII L2 Products (Issue 161 : #161)
@antoinemerle
Copy link
Contributor Author

Hi @amilan17,

I just pushed all the changes on the dedicated branch : 03b301f

Can you provide me the Slideshare URL in order to allow me to drop the sample files there ?

Thanks a million !

@antoinemerle
Copy link
Contributor Author

antoinemerle commented Jun 27, 2023

HI @amilan17 ,

Here are all the data generated corresponding to this issue and to the changes applied to this branch.
I already put version 40 for the Master table code in the test data

Note : Due to the Github's restrictions of not being able to upload more than 25MB I had to split the VII L2 in 3 parts... sorry for the inconvenience....

IASI 2 :

VII L2 :

@marijanacrepulja
Copy link
Collaborator

Hi @antoinemerle,

Many thanks for the samples provided.

I believe the units should be corrected for the descriptors

0 33 118 Relative error percentage Percentage 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage Percentage 0 -100 8 VII-02-WVV, VII-02-WVI

Instead of Percentage should be %

antoinemerle added a commit that referenced this issue Jul 5, 2023
As suggested by @marijanacrepulja (#161 (comment)) : fixing the following


Updating 033118 and 03119 from `percentage` to `%`
@antoinemerle
Copy link
Contributor Author

Hi, @marijanacrepulja ,

Thanks for spotting it!
Updating 033118 and 03119 from percentage to % in de70ce8

@SibylleK
Copy link
Contributor

BUFR-issue161-EPS-SG-Level2_DWDreadbufrxOutput.tar.zip

Using the BUFR tables of the branch (and link version 41 to 40, as the version in the example is 40), the examples could be read with the DWD BUFR reading software.
But only the first 100 Subsets of each BUFR are read, to be able to upload the output here.
The Output of the BUFR seems to be Ok.
(But I don't know whether the values are correct. Please feel free to check:-))

There are no examples with 310090 (IASI SFC?) and 310098 (VII L2 CTP is encoded with 310097 as the CLD) in the provided test-BUFRs.
All other sequences are included in the examples and the BUFR are readable.

From my point of view the validation of the sequences 310087, 310088, 310089, 310091, 310092, 310093, 310094, 310095, 310096, 310097, 310099, 310100 including table B entries and code flag table entries is done.

@antoinemerle can you provide samples for 310090 and 310098?

@amilan17
Copy link
Member

https://github.com/wmo-im/CCT/wiki/Teleconference.13.July.2023 notes:

@antoinemerle address comments in PR, provide two more samples for antoine and @SibylleK validate within a week;

@antoinemerle
Copy link
Contributor Author

antoinemerle commented Jul 18, 2023

Hi @SibylleK ,

Again, apologies for not attending the meeting the other day and for not having replied to you before the meeting ! 😢

  • Indeed for 310090 called IASI-NG L2 SFC product, I missed that one during my last delivery
  • However, for 310098 it seems it was present in the First_part_VIIL2.zip ?

In any case, to avoid more troubles and confusion : please find below the two product for your review 😃

Thanks again for your time to decode and review my product @SibylleK ! 👍

amilan17 added a commit that referenced this issue Jul 19, 2023
@SibylleK
Copy link
Contributor

Thanks @antoinemerle for the new examples with the two remaining sequences.

Here is the output of the DWD software:
W_XX-EUMETSAT-Darmstadt,SAT,SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N____.bin.DWDreadbufrxOutput.zip
W_XX-EUMETSAT-Darmstadt,SAT,SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B____.bin.DWDreadbufrxOutput100.zip

I only read the first 100 subset of the first BUFR of the provided CTP BUFR examples.
But this should be enough for a successful validation.

Therefore, from my point of view, validation is done.

@SimonElliottEUM
Copy link
Contributor

SimonElliottEUM commented Jul 19, 2023 via email

@amilan17
Copy link
Member

@antoinemerle  are IASI-NG L2 GHG and IASI-NG L2 SO2 still under investigation for pressure?

@antoinemerle
Copy link
Contributor Author

HI @amilan17 ,

At the moment, this information is still pending on our side. The scientists were not able to give a clear answer to that question and are still investigating.

The way I propose is to let the comment related to pressure for SO2 and GHG empty at the moment and when we will have this info, I will submit request to add the correct note in the corresponding "comment" column.

What do you think @amilan17 ?

Thanks !

Best,

@antoinemerle
Copy link
Contributor Author

Hi @SibylleK ,

Thank you for your work and dedication on that Task ! Much appreciated !

Cheers,

Thanks @antoinemerle for the new examples with the two remaining sequences.

Here is the output of the DWD software: W_XX-EUMETSAT-Darmstadt,SAT,SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N____.bin.DWDreadbufrxOutput.zip W_XX-EUMETSAT-Darmstadt,SAT,SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B____.bin.DWDreadbufrxOutput100.zip

I only read the first 100 subset of the first BUFR of the provided CTP BUFR examples. But this should be enough for a successful validation.

Therefore, from my point of view, validation is done.

@amilan17 amilan17 moved this to Ready for FT Approval Procedure in BUFR4 Amendments Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
5 participants