Checklist of steps towards 0.5 release [candidates]

[tkelman]

[minor edits / cross referencing welcome here, but copy-paste your changes before submitting in case of multiple people trying to simultaneously make edits and be careful not to overwrite changes made by others - for additions, or deletions?, leave comments below and we'll discuss before adding to the master list]

Timeline for things that need to happen in a specific order:

• finish the remaining breaking/feature items on the 0.5.0 milestone
  1. SSH support in Pkg: SSH support in Pkg #16041, PR at LibGit2: build libgit2 with libssh2 support #17391 - assignees = Art, Keno, and myself
  2. safe non-traditional array indexing: Safe non-traditional array indexing #16973, (possibly stale?) PR at Make unvetted size throw an error for arrays with non-1 indexing #17228 - assignees = Tim, Jeff, core team for feedback
• announce feature freeze by changing VERSION from 0.5.0-dev to 0.5.0-pre feature freeze master #17503
  • master becomes (temporarily) locked for anything other than bugfixes (preferably release-critical ones, anything not absolutely critical can potentially wait until backporting for 0.5.1), doc fixes or test additions
• create release-0.5 branch, change master VERSION from 0.5.0-pre to 0.6.0-dev to reopen master for breaking changes
  • this could happen immediately after the feature freeze step, but unless progress towards breaking 0.6 changes is inconveniently blocked here we might want to wait a short time
• at this point commits should not be made directly to the release-0.5 branch without prior discussion, the branch will operate in release mode:
  1. bugfixes should be made as usual via PR's to master first, wherever applicable there
  2. if you think something is a backport candidate, ask about it with a comment, we will have some discussion and decide yes or no on backporting during the RC process
  3. a contributor will mark the issue/PR with the backport pending 0.5 label, or ping the @juliabackports user in a comment (which sends a message to https://groups.google.com/forum/#!forum/julia-backports which we will monitor) if there's a needed commit to master without an associated issue/PR number (we should almost always avoid doing that)
  4. backporting changes should be done as follows:
     • git checkout release-0.5
     • git pull
     • git checkout -b $USER/backport-foo
     • git cherry-pick -x -e SHA
     • Edit the commit message to add "ref #..." so the cross-reference link shows up to the associated issue/PR.
     • Open a pull request targeted against the release-0.5 branch (there's a github dropdown menu on the open PR page) so CI runs. Please add [release-0.5] to the title of the PR so it's more obvious that the PR is against the release branch and not master.
     • If we have a lot of these, I will handle the backporting myself with a series of rollup pull requests to lessen the CI load.
• when the last RC1-critical cleanup items (see below) are finished, we tag v0.5.0-rc1, build binaries, and announce to the mailing lists
  • automatic windows code signing is not working right now on the buildbots, so this will have to be done manually on a windows machine by someone who has access to the certificates - probably me
  • any users and package authors who've been waiting to test their code and update their packages for 0.5 should do so
  • the most recent RC will be available under Travis language: julia as a julia: 0.5 option and AppVeyor with an 0.5-latest url, but release will not change until 0.5.0 final is tagged
• repeat the last few steps in the process, targeting 1 RC per week, until we're satisfied with being ready to tag v0.5.0 final.
  • I'll probably handle most of the backports at this stage, and run PackageEvaluator on the proposed changes to make sure we don't introduce regressions between successive RC's
• tag v0.5.0 final, celebrate, 🎆 🍻 etc, get 0.6 going and onwards towards 1.0, world domination
  • bugfix backport 0.5.x releases will happen at a target of one per month, as we've been doing
  • change VERSION on release-0.5 branch to 0.5.1-pre

Things that can happen at any time but need to be done (help welcome!) before RC1:

• remaining NEWS and doc additions for Stringapalooza, threads, IOContext [Doc] Documentation for IOContext required #16763 and all other breaking changes, deprecations, and major feature additions and performance changes that users should know about (e.g. startup time on Windows, LLVM version, debugger availability, iterator traits HasLength() is the default iteratorsize for all custom iterators? Even if they don't define length? (Breaks Collect) #15977, docs for collect() are incorrect #17439, etc)
• deprecations still not in NEWS like deprecate print_escaped, print_unescaped, and print_joined #16603 and many others (done in NEWS: move breaking changes earlier, add a few deprecations #17779)
• make sure CI, buildbots, and tests in both source builds and binary installs are all happy with the final set of changes (linux binaries need rpath errors #17602 (comment) fixed prior to 0.5.0 final to get rid of a warning when libgfortran is not installed system-wide)
• make sure make release-candidate passes and checklist will all work, and keep it that way (doctests etc)
• fix final important straggler bugs, e.g. Pkg.checkout issues, deciding what to do about ARM/Power breakage (maybe use a different LLVM version but only on those architectures, if that's the best fix)

Things that can start now but will go on between RC1 and v0.5.0 final:

• fixing broken packages! see http://pkg.julialang.org/pulse.html, there's a lot of work to do to bring the ecosystem to a state where it works as well on 0.5 as it does on 0.4 right now
• identifying and fixing any release-blocking bugs that we don't know about yet and/or can't fix before we decide to tag RC1
• review parts of the documentation that aren't tested by doctests and make sure they're up to date (moved from rc1 to ongoing)
• review issues and PR's (both open and closed) marked with needs-docs and needs-tests and rectify any we can (moved from rc1 to ongoing)

The later pieces of this have been my policy for release management ever since the 0.3.x backports and 0.4 RC's, which has been working pretty well from my impression. Feedback on this is welcome, and I'll be turning these items into a formal written release/backporting policy document going forward.

[davidanthoff]

I would like to see #17422 added to the 0.5.0 milestone.

[davidanthoff]

I'd like to see #17423 added to the 0.5.0 milestone.

(I'm adding all of these as separate comments so that people can like, dislike individually).

[eschnett]

I would like to see eschnett/SIMD.jl#12 added to the 0.5.0 milestone. Although reported against the SIMD package, this seems to be an issue with setindex! for arrays in Base.

[timholy]

See the final line of #16125 (comment) (I'm not sure what the right thing to do is, but I suspect we should do something).

[tkelman]

@timholy are you responding to someone or making a separate point? I think something could be added to PkgDev for ambiguity checking. Or PkgEval, or the default Travis script.

@eschnett looks like you closed that in your package. just in terms of how github works, something needs to have an issue or pr in this repo to be added to the milestone.

[davidanthoff]

I'd like to see #17438 added to 0.5.0.

[timholy]

I meant that if it needs to be a change to Pkg itself (e.g., add another keyword to test, with the default behavior...to warn?...to error?...to not even check?), we should do that before RC because it could have substantial impact on people who are busily updating their packages.

If the change will be applied elsewhere, then no sweat.

[tkelman]

Doing it in Pkg.test would get more people running it right away, so there would be a big benefit to doing it there. I don't think anyone was planning on working on anything related to ambiguity, but it would be a good idea. Maybe warn by default with a kwarg to disable (or set to an error) would be good?

[timholy]

detect_ambiguities needs to be passed a module name. What's the right strategy? I'm guessing it's rather error-prone to assume there is a module with the same name as the package?

Or make the keyword be a list of modules that the user wants to test? It will thus require opt-in (which has its strengths and weaknesses).

[tkelman]

We should move this to its own issue or placeholder for a PR (edit: done by Tim, https://github.com/JuliaLang/julia/issues/17444). Doesn't the package loading machinery already assume the module and package name are the same?

[eschnett]

@tkelman We found and corrected the bug (which was in SIMD after all).

[tmptrash]

I'd like to see #17386 issue fixed in 0.5.0

[tmptrash]

Also #15017, #14495 and #16746 issues...

[mauro3]

@tmptrash: best reference issues & PRs with just typing: #17386. That way one sees the number and hovering the mouse-pointer over the link will show the issue title. Thus your issues are: #17386, #15017, #14495, and #16746.

[davidanthoff]

I'd like to see #17481 fixed in 0.5.0.

[ChrisRackauckas]

@tmptrash your issues should be addressed but I don't see them as release blocking. They all seem to be featured around Tasks which, while it's important to fix, it's not like people can't really run Julia v0.5 without them fixed.

The remaining issues should focus on things like making sure the new package manager works correctly on Windows (@davidanthoff 's issues) or making sure that the Rmath.jl change works (as of right now it breaks all of the stats libraries, some plotting packages, some optimization packages, etc.). These issues are hard for (at least Windows users) to NOT run into.

[tmptrash]

I agree, that some of them are not blockers. Should i remove them from the list? On the other hand #15017 is a critical, because it produces crashes. I have waiting for this fix since february and as i understand, it has already fixed in a brunch. Anyway i can agree with your desicion ;) In this #17386 issue, the part of language (profiler) doesn't work. So maybe it also a little bit critical :)

P.S. I'm new in julia and not so deep understand all the process, sorry.

[ChrisRackauckas]

Just because it produces crashes doesn't make it critical. #15017 produces crashes if you use eval in a loop with Tasks. That's a pretty specific case which I have not seen in any package. In fact, it's been an issue for quite awhile, and the vast majority of Julia simply avoided doing it. So I don't think that large parts of Julia v0.5 are unusable without fixing it, and that seems like a good working definition of what's release blocking.

Since there's already a fixed branch for it, it'll get tested and probably put into a master build soon. However, I don't see a major release being delayed for it.

[tmptrash]

Okay, what about #17386?

[timholy]

Same thing. You are clearly pushing into new territory with Tasks, which is great, but it doesn't affect 99.9% of all users of the profiler.

[ChrisRackauckas]

That again looks like the same issue: Tasks with eval in a loop. Can you come up with a way of creating this error without using Tasks with eval in a loop? If not, then it's not really critical because unless you have this exact use case (which, try to find any Julia package which does this) you don't get this error.

I think it's because eval is not thread safe, yet you're using it with a parallelization construct that requires some kind of safety (though I am not really sure). It sounds dangerous to mix the two. Yes, a good fix for it should definitely come at least by the time threading is introduced as non-experimental (which I think is by v1.0?), but it's not part of what Julia v0.5 is really trying to do.

[tmptrash]

I don't need real threads. The idea of green threads is a best solution for my task, because in reality there is only one system thread and i don't need to synchronize data between green threads. This is one of reasons, why i chose julia for my research. Anyway, i'm waiting for v0.4.7, where this issue should be fixed.

[tkelman]

if it's already been fixed on master, then it certainly doesn't need to be mentioned here

[martinholters]

If the bump to 0.6.0-dev is not the first commit after a tag, the "build number" won't be reset. We might want to fix that first (or along with the version bump, at least). See #17434 (comment) for an idea.

[omus]

#17107 should be added to the 0.5 milestone. It's only an issue with an example in the manual but it should be addressed before release.

[tkelman]

Is there a separate discussion around this, possibly for divvying up work for those who would be willing to update others packages?

Not yet, and I'm not sure whether this repo is the best place to do so. Mailing list? Gitter? Placeholder repo in some organization?

A decent number of packages may be working okay on master but need tags, so in those cases some prodding to release a new version that passes on 0.5 may be in order. It wouldn't be too hard to modify PkgEval to test against master of everything if someone's interested in trying that and seeing how the results differ.

[ChrisRackauckas]

@randyzwitch as Tony Kelman said, for most largely used packages (PyPlot, Juno, all of JuliaStats, etc.) you'll probably notice that people already have it working on v0.5-rc0 but you just need to checkout master (likely on many dependencies as well). It would be best to work it out via Gitter or however else you can get in contact with the main developers for this reason. For example, to get Juno working on v0.5-rc0, you just have to checkout master on like 8-9 different repos. So the issue is mostly about tagging the new versions.

The only major issue would be to get everyone to root out the deprecations. If you use something like PyPlot, you get a wall of deprecation warnings, usually the string deprecations. It works but is off-putting. Someone should use PkgEval test and go around to fix the deprecations in the large projects, or at least the ones that many other packages depend on.

[josefsachsconning]

Just want to make it very clear that I echo @davidanthoff 's request above.
#17418 (comment)

[tkelman]

As an update here, I'm preparing a branch full of cherry-picks of backport pending PR's for RC2, which I'll be opening a PR for when it's ready to run nanosoldier and pkgeval on. (It will look a lot like master, minus a handful of things that aren't backport candidates.)

edit: #17960

[tkelman]

rc2 will be tagged momentarily and uploaded as soon as it's built has been tagged and uploaded, just need to announce

[josefsachsconning]

Is rc2 actually literally a release candidate, unlike rc0 which was understood not to be, but more like a beta release?

[tkelman]

rc2 is missing one major thing at the moment, which is proxy support - that exists on master but the build system with curl is rough around the edges and will need some work to get backported into rc3.

[davidanthoff]

I'd like to nominate #17146 (or #17995, they might be dups) for 0.5.0. They corrupt repo state and I don't think 0.5.0 should ship with bugs on such a common code path that messes up local repositories.

[tkelman]

RC3 backports are at #18156. This includes curl and should work with proxies, so will be the first release candidate that we could seriously consider promoting to 0.5.0 final. We'll decide after between a few days and a week of testing on RC3 whether to put out an RC4 or go straight to final.

[tkelman]

See #18276 for RC4 backports. It doesn't look like many more bugfixes are going to happen right now, so anything that doesn't have a fix materialize soon will be 0.5.1 material.

[mortenpi]

I am wondering, should the issue described in #18353 get a NEWS mention (in short: non-obvious behaviour when passing redefined functions to map, quadgk etc. due to #265)?

I ran into this the moment I used 0.5 for interactive numeric work, and it took a moment to figure out why things were behaving strangely, since it works without problems on 0.4.

[Ken-B]

There doesn't seem to be a step in the release process for adding release-0.5 to readthedocs.org.

At the moment on docs.julialang.org you have versions release-0.3, release-0.4, stable and latest. For me it's not clear which one is currently for v0.5. Thank you.

[tkelman]

Yes, good question @Ken-B. I have never touched readthedocs myself, @StefanKarpinski who has admin access there to create a new release option?

[ViralBShah]

I now have admin access and enabled the release-0.5 branch and the 0.5 docs are live: http://docs.julialang.org/en/release-0.5/

[ViralBShah]

What needs to be done for the stable branch on readthedocs?

[tkelman]

final few backports are at #18546

[davidanthoff]

Is there going to be a RC5 for this stuff?

[tkelman]

I'll post links to the binaries that are going to run through pkgeval, but the diff there is pretty small inside src and base.

[tkelman]

will be tagging, building and uploading today