Many Xenial entries are being dropped

[smits]

I just noticed that many Ubuntu Xenial entries are being dropped. Looking at the ROS metrics: https://metrics.ros.org/packages_linux.html this will impact a significant portion of the community? Is this intentional as a way to force these users to newer distros or to force them forking rosdep?

[knorth55]

this PR #31552 is breaking many dependencies...
Our robot is still running in Indigo and Kinetic.
We want to upgrade but we have no time to do it.

[nuclearsandwich]

Is this intentional as a way to force these users to newer distros or to force them forking rosdep?

There was no nefarious or explicit intent to break things or push people into upgrading. The changes to the rosdep db were made during infrastructure bootstrap of a new platform with a cleaner-than-we-found-it maintenance policy.

The ROS team is preparing for the upcoming release of Ubuntu 22.04 Jammy which is the supported Ubuntu distribution for the upcoming ROS 2 Humble Hawksbill release. In order to work on ROS 2 on Jammy (and Debian Bullseye), we need to update the rosdep database. Keys which are defined for all Debian or Ubuntu distributions should be valid for all currently supported distributions. As Debian and Ubuntu phase out Python 2, many Python 2 packages are no longer being packaged in Ubuntu Focal and Jammy, or Debian Buster and Bullseye. As a result, there were several cases such as #31554 and #31552 where I needed to expand the collapsed key (i.e. ubuntu: [python-lxml]) key to explicitly exclude Jammy.

When doing so, I made sure that expanded entries were present for currently supported distributions that have the package available. Since community support for Ubuntu Xenial ended in April, I did not add entries for it in order to keep the expanded definition in its minimally valid form.

Based on the reports from #31570 #31580 and in this issue I've opened #31613 and will be either holding or updating the currently in-review pull requests which make similar changes to temporarily limit further changes affecting Ubuntu Xenial.

However I do not think it is sustainable for the ROS team to indefinitely maintain an accurate rosdep database that includes unsupported platforms. So I'd like to work with the affected parties to settle on an alternative.

@cottsay's recommendation to use rosdep sources from an earlier commit remains the best option I see for a static Kinetic project and we can add a few release policies to the rosdep db project to tag just before dropping support for a platform so that there is a named target for running rosdep on unsupported platforms using the last set of official sources to support it.

Since rosdep allows multiple sources, this also means that projects which may be actively developing and introducing new dependencies on an unsupported platform may add additional custom sources to expand upon the snapshot of the official rosdep db without requiring any modifications to it. I mentioned in this comment that I think this approach represents the minimum amount of maintenance work required overall even if it shifts some of that maintenance work onto the projects using unsupported distributions.

[knorth55]

The python-numpy package is not present in Ubuntu Jammy. You can see the package listed for focal and bionic but it is not present in jammy (link 404s) because the python2 version of the package has been dropped. If we left the key defined for all Ubuntu distributions then attempts to use rosdep for packages defining those platforms would result in package installation failures rather than correctly reporting that the key has no definition for that platform.

python-numpy key is not found in Ubuntu Jammy, rosdep cannot install it in Jammy and raise key errors.
So I think there is no necessity to remove the existing xenial key from ros/rosdistro, but is my understanding incorrect?
Is there difference between the situation rosdep cannot find python-numpy key in ros/rosdistro, and the situation that rosdep cannot install python-numpy resolved from the key python-numpy.
For me, it seems both two are catched as install error by rosdep.

I understand that adding new keys for xenial should not be acceptable.
Tagging is the gread idea, I think.

[tfoote]

Dropping Xenial rules at this point is expected and is not something that we should be avoiding. If someone was to take the time to clean up all the xenial rules today and simplify all the rules which xenail requires a special case for I would likely approve such as PR in my role as a reviewer.

We officially supported Xenial until it's EOL date which was well over half a year ago. And the Xenial support window was clearly advertised as being 5 years ending in April 2021. The reason that we have support periods is to scope the amount of work that we have to do. Maintaining the rosdep database is a non-trivial effort that we do to support the entire community.

We no longer support Xenial so that we can spent that effort on current and upcoming distros. In this case as we're making the updates for Jammy we are taking the opportunity to clean up old rules on unsupported platforms that will safe effort in the future. We could do a policy of never removing them, but that would end up with a weird state of things partially work, but can't be expected to work in all cases and so really it would just be a bad experience and people wouldn't really know if it's supported or not.

Along similar lines the packages in the apt-repository are subject to removal at any point now too. We do not want to continue host them on our high bandwidth CDN network and have our data requirements infinitely grow. We've provided continuing facilities because it was easier to not turn them off immediately and gives users a small grace period but if users take advantage of that extra period doesn't mean that we shouldn't turn it off as previously announced.

It does sound a little harsh, but we should not be spending effort to support users who are choosing to use an unsupported OS with an unsupported version of ROS. If the user makes this choice not to migrate forward, it is there responsibility to provide all the necessary infrastructure that was previously provided by the support platform.

We obviously do not want to leave users who cannot upgrade for whatever reason in the lurch. We have the snapshot repositories for people who want to be able to reproduce old package installations. And we make tags of the rosdistros for all syncs now. I'd recommend those working on xenial to consider using the latest kinetic tag. If a user needs new or updated rules for any other reason. The rosdep database can be forked for their own purposes or augmented with additional rules and it won't slow down the preparations for the next supported release and ongoing improvements for the currently supported releases.

[smits]

@tfoote , @nuclearsandwich I fully understand and support your arguments to move on. Before I close the issue I would like to double check if I got the proper resolution correct.

We create a self-maintained rosdep list file (10-ros-kinetic.list) with the following contents (afaiu it needs to start with a number smaller than 20 to make sure that its definitions will overlay on the default definitions) :

yaml https://raw.githubusercontent.com/ros/rosdistro/kinetic/2021-05-11/rosdep/base.yaml
yaml https://raw.githubusercontent.com/ros/rosdistro/kinetic/2021-05-11/rosdep/python.yaml
yaml https://raw.githubusercontent.com/ros/rosdistro/kinetic/2021-05-11/rosdep/ruby.yaml

Can we still rely on https://raw.githubusercontent.com/ros/rosdistro/master/index-v4.yaml to provide the released kinetic packages, or is it safer to add it in the above list file as well?

On top of that we should update our apt sources files to point to snapshots.ros.org instead?
deb http://packages.ros.org/ros/ubuntu/ xenial main --> deb http://snapshots.ros.org/kinetic/2021-05-11/ubuntu/ xenial main

[tfoote]

Yeah, the rosdep sources like that look correct. You may end up wanting to have your own lines added too.

Can we still rely on https://raw.githubusercontent.com/ros/rosdistro/master/index-v4.yaml to provide the released kinetic packages, or is it safer to add it in the above list file as well?

I don't think that we've ever talked about a migration path for the rosdistro cache and have at least put in the EOL status so it's probably be best recommendation to keep the production one for now. It's not a large resource sink so it's likely easier to keep it there than officially migrate it.

On top of that we should update our apt sources files to point to snapshots.ros.org instead?
deb http://packages.ros.org/ros/ubuntu/ xenial main --> deb http://snapshots.ros.org/kinetic/2021-05-11/ubuntu/ xenial main

Yes definitely switch over to use the snapshots. This is what we do in the official final docker images too. They will disappear from the main repository at some point in the not too distant future. Note that the snapshot repository also has a different gpg key too when you set it up.

[tfoote]

As I don't think that there's anything we're still tracking on this I'm going to close it out. @smits Please let us know if you hit any corner cases that are problematic on using the snapshots.