Migrations with older builds

[hmaarrfk]

Comment:

One of the big challenges with libprotobuf is that it always has quite a big lag with the tensorflow migration.

Often this isn't insurmountable, but i've been getting environments lately that fail to solve because libprotobuf is held back.

Would it be appropriate to build out a few packages with some older protobuf versions too?

[h-vetinari]

Taken as-is, that would essentially mean remigrating a lot of packages for older abseil & grpc & protobuf, as tensorflow is still stuck on a grpc/protobuf that was built with the old abseil. It's not impossible, but quite the churn, because it touches 50+ feedstocks.

Also worth noting that tensorflow is really the only relevant outlier here, and has already missed 3 migrations, so we'd have to rebuild lots of packages for completely obsolete combinations.

An alternative might be to rebuild grpc 1.54 + protobuf 3.21 (where tensorflow is currently stuck) with newer abseil. Or rather, we'd have to do that in any case, in case we want to piggy-back protobuf 3.21 onto the already pending next protobuf-migration.

[h-vetinari]

In more detail, this is our migration history:

abseil	grpc	protobuf	comment
20230125	1.54	3.21	tensorflow is here
20230125	1.56	4.23.3
20230802	1.57	4.23.4
20230802	1.58	4.24.3	rest of the ecosystem is here
20230802	1.59	4.24.4	migration ready to go

[hmaarrfk]

ok hopefully the model for zaber-motion can be used for others to build multiple configurations upon request.

[h-vetinari]

ok hopefully the model for zaber-motion can be used for others to build multiple configurations upon request.

I think anything having been rebuilt only for newest abseil will conflict (which can easily happen if it came after another migration that you need, like say libjpeg-turbo), unless we rebuild grpc 1.54 & libprotobuf 3.21 for newest abseil.

[hmaarrfk]

ok hopefully the model for zaber-motion can be used for others to build multiple configurations upon request.

I think anything having been rebuilt only for newest abseil will conflict (which can easily happen if it came after another migration that you need, like say libjpeg-turbo), unless we rebuild grpc 1.54 & libprotobuf 3.21 for newest abseil.

This is a good point. For now, it hasn't been a problem. lets hope it stays that way, i want to avoid the churn.