Bintray will be gone at the end of April

[cielavenir]

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

crystal-lang install.sh still relies on bintray, so I suppose it should be fixed as soon as possible.

I think some of you already started working on it, but I don't see any trackers of it.

(sorry if the category is wrong, but it could be considered as install.sh's issue...)

[j8r]

Alternatives are Launchpad and Open Build Service.

[straight-shoota]

Unfortunately, the sunset of Bintray cought us at a really bad time, while being busy with the big 1.0 release and leadership changes. And we had even just completed migrating to bintray a couple of monts ago 😞 So unfortunately, I don't think there has been any work on a replacement yet. But obviously, this requires attention now.

@j8r Launchpad is for Ubuntu/APT only, right? So it wouldn't be a replacement for the RPM repository?

Open Build Service seems more versatile.

[jhass]

OBS is fine despite maybe not the most intuitive thing to use, at least when I tried a couple of years ago.

However integrating distribution and building might not be a big must here. I would consider building a solution that can generate repositories using Github Actions (or perhaps Circle), maybe something that can run in a docker container to be pretty independent. Then we can upload those repositories just to any CDN solution. This should give some flexibility and independence for exchanging either the build service or the CDN service.

[straight-shoota]

OBS is fine despite maybe not the most intuitive thing to use, at least when I tried a couple of years ago.

Definitely. I've taken a look at it a couple of weeks ago. Documentation is plenty, but there's a lot to grasp. And documentation is mostly focused on RPM packages, so while others are still supported, it's not always easy to spot the specific differences.

I would consider building a solution that can generate repositories using Github Actions (or perhaps Circle), maybe something that can run in a docker container to be pretty independent.

I think we essentially have this already in distribution-scripts as it was used before bintray. A generalized solution would be better. And dedicated repo hosting is also better than plain file hosting service because then we'd need to do the management ourselves.

[jhass]

What exactly is "the management" here?

Both RPM and APT repositories are designed to be plain file structures, historically so to allow easy FTP mirroring.

Note that I'm talking about repository generation above, "building" might have been a misleading term :)

[straight-shoota]

Repository management. Configuring package metadata etc. So I guess that's what you include in building? I assumed building would be just packaging the .deb etc. That's what we already did ourselves in the old process.

[jhass]

Yeah, I realize as much. Sorry I wasn't very clear indeed. So yes, my point is that if we make repository generation/maintenance a distinct step we can just use any CDN/filehost to host the generated repository.

[jhass]

• https://www.aptly.info/ looks interesting here for Debian based.
• https://github.com/knazarov/mkrepo looks interesting too, handling both.

[petr-fischer]

Crystal Bintray channel looks dead right now.

Updating repos...
Err:2 https://dl.bintray.com/crystal/deb all InRelease                                                                       
  502  Bad Gateway [IP: 35.156.125.116 443]
Hit:3 http://archive.ubuntu.com/ubuntu groovy InRelease                                                                      
...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
W: Failed to fetch https://dl.bintray.com/crystal/deb/dists/all/InRelease  502  Bad Gateway [IP: 35.156.125.116 443]
W: Some index files failed to download. They have been ignored, or old ones used instead.

[Blacksmoke16]

@petr-fischer https://status.bintray.com/incidents/4mzph0y8l6hk.

[straight-shoota]

I suppose an alternative would be to stop providing our own distribution packages. With the 1.0 release I'd expect we should be able to get Crystal into the official repositories of the respective package managers.
For most distribtuions, propagating new releases wouldn't be as quick through these channels. But we would still offer generic linux release builds. Homebrew and Snappack are available for faster release propagation.

The biggest problem with this approach is the time contraint. We need a replacement soon. And adding crystal packages to official repositories might not be done quickly enough. That's just a wild guess, I don't have any experience with the process.

[jhass]

We certainly have no way of getting it into the current Debian stable release for example. I can't imagine the story is any different with RHEL. For just two major examples.

[straight-shoota]

Yeah, that's what I fear as well.

As an unrelated intermediary step, we could maybe set up a redirect proxy for our package repositories on crystal-lang.org. So that https://dl.crystal-lang.org/crystal/deb redirects to https://dl.bintray.com/crystal/deb. Then we could already change the URLs on the install instructions. Later we just need to update the redirect target for a seamless migration at least for everyone who uses the installer/installation instructions from now on.

[straight-shoota]

This might be another option for self-building: https://github.com/pacur/pacur

[mattfox]

I'd like to conditionally offer my service, Balto Repo, for consideration. It supports Debian and will add RPM soon. You wouldn't have to worry about either the repository metadata or the hosting.

The condition would be- do you have a ballpark idea of the monthly network transfer Crystal uses at Bintray right now? I just wouldn't want to offer something and not be able to deliver if it turns out you do 50 TB/month, for example. Alternatively you could subscribe to a paid plan, of course.

[j8r]

@mattfox You could charge additional fees per additional GB, like some cloud providers do.

[straight-shoota]

@mattfox Thanks for your proposal. I haven't seen that before. The time frame of "RPM soon" would be particularly interesting. Can it realistically be expected before May 1st?
Unfortunately, public information on your website on how exactly this works is pretty scarse. I've only managed to get some insights after creating an account with test project and repo. So as far as I understand, this service basically offers repository hosting and manages metadata for that. We need to build the packages ourselves and upload them, right?
That could probably work as we already have package build workflows and it would be better than the plain-file hosting we had before bintray. But OBS for example has the benefit of being a single solution for the entire package management, including the packaging process itself. It can just grab the generic linux build and push it into packages for APT, RPM etc.

[mattfox]

Yes, RPM is expected before May 1st. And improved documentation after that...

For an example, Balto hosts the Debian repo for Helm, the package manager for Kubernetes- https://helm.baltorepo.com/ . And here's all you do to upload your package- just a curl command: https://github.com/BaltoRepo/helm-linux-packages/blob/master/push.sh#L19

You're correct about what Balto offers. It's for hosting the repositories, not for helping build the .deb and other packages. It's aimed at Bintray and not OBS. From the comments above it sounds like you need a simpler solution soon, not a better solution later. ;)

[mattfox]

@mattfox You could charge additional fees per additional GB, like some cloud providers do.

Certainly that's how it works for paid subscriptions. For an open source project with 16k GitHub stars I want to be more careful about offering an exception to the normal open source transfer max of 10 GB.

But actually I see Crystal has sponsors and commercial backing so a paid plan would be reasonable and make the transfer a non-issue.

[beta-ziliani]

@mattfox if my calculations are correct 10GB/month should be more than enough for some while. What's the space limit you offer for an OS project?

[mattfox]

What's the space limit you offer for an OS project?

"Reasonable". 😀 Storage is so inexpensive that unless your package is huge, or you're adding new builds very frequently, it just doesn't matter.

[rubyFeedback]

Just one comment on that part here:

I suppose an alternative would be to stop providing our own distribution packages.

I just used the generic download for linux thingy and it worked. I think you don't need
to generate the distribution-specific parts, because distributions such as debian or
archlinux always have people who love packaging everything and their cat into
a package as-is. :P

[straight-shoota]

We did some more in-depth evaluation of the available options and decided to go with Open Build Service. It helps to reduce our internal tooling for packaging and promises to add more value in the long run.
Building packages is now completely handled by OBS, it just takes the tarball with the static binaries and sources as input.

The new home for Crystal's deb and rpm packages is now at:

https://build.opensuse.org/project/show/devel:languages:crystal

Installation instructions are available at:

https://software.opensuse.org//download.html?project=devel%3Alanguages%3Acrystal&package=crystal

If you want to help, please check the new packages and report any issues.

I'll update the instructions at https://crystal-lang.org/install and the installer script next.

Currently, only the most recent release 1.0 is supported. I don't think there's much reason to publish older releases.
Nightlies packages will be made available as soon as nightly builds are up and running again (see crystal-lang/distribution-scripts#90).

With OBS we can easily split a separate shards package and have options to add Arch Linux and containers (Docker image) in the future.

[fdr]

I'll give it a shot. Any chance you can turn on aarch64 builds?

[straight-shoota]

Thanks!

Sorry, we still need to boostrap aarch64 in the dist pipeline. But once that's done, it should be a piece of cake to distribute packages with OBS.

[fdr]

I tried it, it seems to work pretty much effortlessly and as I would have expected. Let me (well, everyone I guess) know when you are going to commit to the new repo so I can ship the change more permanently. Will keep you posted if anything out of the ordinary appears.

[straight-shoota]

The new installation instructions and installer script are online: https://crystal-lang.org/2021/04/30/new-apt-and-rpm-repositories.html

Let's hope the transition goes smoothly 🤞

Please try the new repositories and report any issues you encounter.

[fdr]

@straight-shoota broken link in the post, https://build.opensuse.org/package/show/devel:languages:crystal doesn't exist. But other links work.

[Erisa]

For me, running the script failed since it seemed to call apt update which failed because Bintray is now returning 403 to all requests:

erisa@choco:~$ curl -fsSL https://crystal-lang.org/install.sh | sudo bash
Hit:2 http://de.archive.ubuntu.com/ubuntu focal InRelease
Err:3 https://dl.bintray.com/crystal/deb all InRelease
  403  Forbidden [IP: 3.125.168.216 443]
Hit:4 http://ppa.launchpad.net/jonathonf/zfs/ubuntu focal InRelease
Hit:5 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:6 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:7 http://de.archive.ubuntu.com/ubuntu focal-security InRelease
Hit:1 https://kryptco.github.io/deb kryptco InRelease
Hit:8 https://download.docker.com/linux/ubuntu focal InRelease
Reading package lists... Done
E: Failed to fetch https://dl.bintray.com/crystal/deb/dists/all/InRelease  403  Forbidden [IP: 3.125.168.216 443]
E: The repository 'https://dl.bintray.com/crystal/deb all InRelease' is no longer signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.`

I worked around this by deleting /etc/apt/sources.list.d/crystal.list and then rerunning the installation, which setup the new repo and upgraded my crystal package to the latest version.

[straight-shoota]

The script runs apt-get update when wget or gpg are missing because those are necessary for installing the signing key. That's the only explanation I can think of why update would be run before overriding the repo location.
I suppose this could be fixed by removing the repo configuration before that in the script as well.

[luaVolk]

It seems like the instructions for openSuse haven't been updated yet.

https://crystal-lang.org/install/on_opensuse/

[straight-shoota]

There's a PR pending to fix this: crystal-lang/crystal-website#139

[straight-shoota]

With #10684 merged, packages for nightly builds are back.
The packages can be installed from https://build.opensuse.org/package/show/devel:languages:crystal:nightly/crystal or with the installer script from https://crystal-lang.org/install.sh with --channel=nightly argument.

Builds for CentOS and RHEL nightly packages are currently broken because of a weird segfault when executing the packaged crystal binary. Users of these distributions should be able to use any other RPM package (for example from one of the Fedora repositories). The build instructions are identical, but something breaks with CentOS and RHEL 🤷