[Feature] Support module definition change in BanyanDB storage.

[wu-sheng]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Currently, we don't do column validation(name, type, existing) during booting, and would not upgrade the column update when the latest and existing are compatible.

We should consider adding this. Elasticsearch storage implementation would be a good reference.

Use case

No response

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-sheng]

@wankai123 Considering #12442, let's add a database version number concept in OAP codebase. Once we have this in place, we have a chance to do incompatible upgrade through two group.

e.g. banyandb_db_schema_v = 1, and we need some incompatible upgrade, introduced a new kind of index, but the query logic is consistent. We could do banyandb_db_schema_v=2(compatible with banyandb_db_schema_v=1).

Then, when we apply definition change through create new schema_v suffix groups.

[wu-sheng]

@wankai123 Considering #12442, let's add a database version number concept in OAP codebase. Once we have this in place, we have a chance to do incompatible upgrade through two group.

e.g. banyandb_db_schema_v = 1, and we need some incompatible upgrade, introduced a new kind of index, but the query logic is consistent. We could do banyandb_db_schema_v=2(compatible with banyandb_db_schema_v=1).

Then, when we apply definition change through create new schema_v suffix groups.

@wankai123 #12442 has been moved to 0.9, so OAP will consider that for 10.3 only.