🥇 Should the new format be disruptive or not with the legacy one ?

[gmaze]

Following discussions on the list of possible format and wether to include indexed database or not #39 (comment), #39 (comment), #39 (comment) ...
There could be a divergence between a DAC perspective with extensive knowledge of database management and the average user perspective without such expertise that requires much higher level means of accessing the data than a SQL-like query.

Whether a new format should be disruptive to a legacy system depends on various factors:

• the goals of the change,
• and the tolerance for disruption within our (i) ADMT/DACs organisation and (ii) user community

There is also the need to consider the capabilities of the existing system to support the disruption, but within this WG, we shall take for granted that there will be a new system based on a cloud infrastructures (i.e. academic infrastructures such as NOAA, EOSC, Galaxy and commercial infrastructure such as Amazon, Google, Azure) and that the associated migration capabilities of the ADMT players will be available.

Goals of the change
If the goal of introducing a new Argo data format is to significantly improve efficiency, functionality, or user experience, some level of disruption to the legacy system/format may be unavoidable. In such cases, a disruption might be necessary to achieve the desired improvements.

Tolerance for Disruption within our ADMT/DACs organisation
Some may prioritize maintaining smooth operations and minimizing disruptions to users or workflows, while others may prioritize innovation and are willing to accept short-term disruptions for long-term gains.
This WG does not have to answer this, it shall limit its work to give decision making data to the ADMT.

Tolerance for Disruption within our user community
I guess the Argo data user community players differ in their tolerance for disruption. But I believe it is the role of the GDAC/ADMT to provide a well planned transition phase between the 2 formats and associated systems to help mitigate the negative effects of disruption. This might involve phased implementation, user training and support, parallel operation of old and new systems for a period.
With regard to this WG, we shall take this into account in our definition of objectives and metrics.

[dirkslawinski]

As there is some lead time and there are very longstanding workflows that are build around NetCDF a disruptive approach/format will hinder uptake. Running concurrent workflows will be a fact of life until full acceptance of whatever format is agreed up. So it is not so much a question of disruptive but of acceptance with in agreed upon time frame ... whatever is agreed up.