Grib files

Tide, wind, and wave data is collected from a .grb file automatically downloaded from https://openskiron.org/. These are files in the grib1 format, and are accessed using the NETCDF Java Library from https://www.unidata.ucar.edu. Which is a project of University Corporation for Atmospheric Research, and creates a common interface for a variety of scientific filetypes including both grib1 and grib2. This is called The NETCDF (NETwork Common Data Form) interface.

This was found the be the most supported and easiest to use way to access grib files. However any new ways of accessing grib file data are possible by implementing the gribReader interface, and changing the conf.kt to support it in the config.yaml config file.

NETCDF Grib Reader

We approached the task with an iterative process to refine our approach until we achieved success. In navigating the project, we encountered the challenge of complex documentation, which posed difficulties in comprehension. The grib reader is not guaranteed to work with grib files from different sources as this was only able to be tested with these ones.

The docs for the NETCDF Library are found here https://docs.unidata.ucar.edu/netcdf-java/.

NETCDF Files contain variables, which can change and so are found using regex. These variables are stored as large multidimensional arrays, the indexes for these arrays are found in separate "dimension" variable. Each variable contains information about how the data is stored.

Interpolation

The grib files we were using were of low resolution, so the grib reader, if there is missing data for a certain point will interpolate it from surrounding the data, the algorithm used is described below.

val res = variable.read(origin, shape).reduce().getDouble(0)
if (res.isNaN()) {
      var i = 1
      var resList: List<Double>
      do {
          resList = trySurrounding(variable, origin, shape, latDim, lonDim, i)
          i++
      } while (resList.isEmpty())
      return resList.average()
}
return res

private fun trySurrounding(
    variable: Variable,
    origin: IntArray,
    shape: IntArray,
    latDim: Int,
    lonDim: Int,
    i: Int,
): List<Double> {
    val above = origin.copyOf()
    val below = origin.copyOf()
    val right = origin.copyOf()
    val left = origin.copyOf()

    above[latDim] += i
    below[latDim] -= i
    right[lonDim] += i
    left[lonDim] -= i

    val surroundingValues =
        arrayOf(above, below, right, left).map { x ->
            variable.read(x, shape).reduce(0).getDouble(0)
        }.filter { x ->
            !x.isNaN()
        }
    return surroundingValues
    }

What this function does is if the result of reading the variable where necessary is NaN (Not a number) it will try every datapoint surrounding the variable to check if they are also NaN. If this is the case it continues to expand the "search range". Once values which are filled are found, every value of equal distance to the missing point will be averaged.

To visualise this we show trying to attempt to get a point x

X is NaN so it searches the values of surrounding squares

These are also surrounding so it continues to expand

It repeats until it finds data points which are not null

Once it has, it will take the average of the found points, shown here in green