State of jasper

[jubalh]

Debian and Ubuntu dropped jasper from their repositories in 2016 because of the security issues.
In 2018 there were several more CVEs discovered.
They have been submitted in one bug report and some of them as individual bugreports.

An effort was made in fixing these bug via a pull request which did not receive any feedback since Oct 6, 2017 to this day. I commented on it to ping the maintainer, no respone.
However one of the commits that were included in that PR were by hand again without mentioning the original commit/author: 573a6e4. The coresponding issue #142 was also not closed.

Later another effort was made by posting fixes to several of the bugs here, located on gist.github.com.
AFAIK it were the patches used by the Debian LTS team.

They also received no feedback.

I asked the maintainer what was the reason for this, and he replied that he has limited time. Which is understandable of course. And that some pull requests are not real fixes but only mask the problem. Which is also fine but would be good to have commented on publicly on the pull request since now he himself didn't remember which PRs were the faulty ones.

I created pull request #197 #198 #199 so that it is easier for the jasper maintainer to just merge them and not have to go through the pastes. #200 was opened as a better solution for one of them.

CVE-2017-13748 was adressed in #159 in 2017. CVE-2018-9055 was adressed in #204 in 2019. Some more fixes in #158.
Neither received any feedback.

Today there are 56 open issues, and 7 open pull requests.
openSUSE ships 2.0.16 with a few patches for CVEs on top.
Fedora ships 2.014 with some patches for CVEs but not fixing all the ones that have fixes available.
Debian and Ubuntu removed jasper.
Gentoo ships 2.0.16 with no patches.
Arch ships 2.0.16 with one patch.

With 2.0.16 (12 March 2019) the tarball was not put to the usual location, the changelog and website were not updated. Website still mentions JasPer 2.0.14 (2017) as latest release. See #202 for the report.

Having limited time for an open source project is fine and understandable. But I think some parts of how things were handled could have been done better. I also think that it seems like at one point there were several people trying to help and contribute to jasper but stopped doing so because there was no reaction to their efforts.
I asked several months ago if it would be possible to at least now add more maintainers to the project. People who are still interested in helping jasper. Or assign some students from University of Victoria for this task. Even if not for active development/bug fixing then at least for sorting issues and taking care of pull requests but did not get a reply to this.

So currently openSUSE is also trying to reduce dependencies on jasper, and might drop jasper from its repos too.

My strategy was to try to help revive jasper, commenting on issues and creating pull requests.
Now I'm trying to convince other projects that rely on JasPer for JPEG-2000 support to switch to openjpeg which seems currently to be better maintained, so that when we remove jasper we don't loose JPEG-2000 support in those programs. Debian people did this for DigiKam in the past too. I'm also changing openSUSE packages that can choose between the two to use openjpeg instead.
An alternative would be that upstream decides to add more maintainers and fixes bugs or that the community forks jasper and merges the current pull requests and tries to fix more.

I'm creating this issue in hope of finding more people interested in jasper so that we could find a solution together.

[SoapGentoo]

@jubalh I agree, and I have just started the process of dropping jasper from Gentoo. Consumers should move to openjpeg-2, which is maintained. Thanks for the wake up call.

[crocket]

@SoapGentoo But, you masked media-libs/jasper without fixing ebuilds that depend on jasper.

> equery d media-libs/jasper
 * These packages depend on media-libs/jasper:
dev-qt/qtimageformats-5.12.3 (jpeg2k ? media-libs/jasper)
media-libs/gegl-0.3.34 (jpeg2k ? >=media-libs/jasper-1.900.1)
media-libs/gegl-0.4.14 (jpeg2k ? >=media-libs/jasper-1.900.1)
media-libs/libraw-0.18.13 (jpeg2k ? >=media-libs/jasper-1.900.1-r6[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_riscv_lp64d(-)?,abi_riscv_lp64(-)?,abi_s390_32(-)?,abi_s390_64(-)?])
media-libs/netpbm-10.76.00 (jpeg2k ? media-libs/jasper)
x11-libs/gdk-pixbuf-2.38.1 (jpeg2k ? media-libs/jasper[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_riscv_lp64d(-)?,abi_riscv_lp64(-)?,abi_s390_32(-)?,abi_s390_64(-)?])

[jubalh]

you masked media-libs/jasper without fixing ebuilds that depend on jasper.

Which should be reported on the Gentoo bugzilla or directly fixed in their tree. But not reported here :-)

[mdadams]

@jubalh @SoapGentoo I am sorry for the situation with JasPer, but I have no one to help me with the project and I also have no funding for this project either. Moreover, I do not do any research work related to JPEG 2000. Although I have been told by many people that they find JasPer useful in their commercial products, no company wants to actually provide funding for the maintenance/development of the code. So, I just do the best that I can (in my own extremely limited spare time), which is admittedly not very satisfactory. But I don't see any alternatives, unless someone can provide support for this project.

[SoapGentoo]

@mdadams I understand that, but one approach would be to transfer it into a "JasPer trust" or round table, in order to increase the maintainer/gatekeeper bus factor. I commend @jubalh for trying to find a community-oriented solution at this point.

[Justinzobel]

@SoapGentoo that would be a great plan.

[SoapGentoo]

Gentoo has removed jasper two weeks ago.

[Justinzobel]

@SoapGentoo an unfortunate necessity I imagine. Just a pity. I just wish someone could pick it up, fork it and provide the necessary patches as many things still rely on jasper.

[jubalh]

It's not just about picking it up. Finding solutions for some of the issues is not easy.

[Justinzobel]

Then it's about getting downstream projects moving on from abandoned software.

[jubalh]

Which is what we are doing here since a few weeks ;-)
I wrote to all downstream projects that use jasper and have packages in openSUSE.

You are late to the party ;)

[Justinzobel]

Thank you then for writing to them all.

[Justinzobel]

Do you mind linking the issues?

[jubalh]

Do you mind linking the issues?

Some of them have issues in the projects bugtrackers others were communicated via private mail.
You can start looking at https://bugzilla.suse.com/show_bug.cgi?id=1130404
And then click on the bugs mentioned after Depends on.

Usually I documented there the link to the bugtrackers. But if it was a private mail I only made a private comment which not everybody can read.

[jubalh]

We are making progress.

Two months ago openSUSE removed Jasper from its repositories too.

Today OpenCV merged a patch to use OpenJPEG instead of Jasper for JPEG2000 support.

Nevertheless I opened another pull request which fixes CVE-2018-9154.
There are still 9 pull requests open.

@mdadams would you consider taking a look at merging them?

[jubalh]

So I would like to try it on more time.

Do you think we can work together here to improve the state of the project?
You could add collaborators to your repo so that more people can merge pull requests and review things. I would volunteer to do so.

If even more people would like to joind or you don't want any responsibility at all or don't want to host it on your private account we could also create an organization on GitHub and transfer the repository there.

If you send us the source for the website currently hosted at https://www.ece.uvic.ca/~frodo/jasper/ we could also use GitHub Pages to host the site on GitHub.

With more people spending time on this there is a better chance that this thing keeps on going.
Like I said, I would volunteer to help, and maybe others join too.
What do you think @mdadams ?

[jubalh]

@mdadams another week has passed.

While I totally get that you probably are quite busy, I absolutely don't get why you won't allow the project to live without need of you immediate action.

Yes, you created the jasper-software organization on GitHub (after we created jasper-maint..) but you are the only owner of the organization. @MaxKellermann and I can't do much there. So we are totally blocked to do anything there.
Also the old repo is under your personal account, so we couldn't even transfer it. So we are blocked again..

All CVEs (and more) got fixed. Code got cleaned up pretty nicely. We tried to organize and comment on existing issues, making all the work transparent and easy to follow.

So all the technical work is done. But more organizational work is needed.
Release notes to be written, a release put out. So that finally after many years, people actually can use a safe version of JasPer again.

We would still like to help here, but you mentioned your opposition of our fork. Saying it needs to be done in a way that you feel comfortable with.

Well, I'm not sure what you are comfortable with and with what not. I only see that you cling to this project in a way blocking all progress.

Let us just take the easiest route now:
Join our jasper-maint organization, so you stay in control of JasPer. (Which we already sent before but the invitation expired because you didn't accept it in time)
Write a note in the Readme of your repo pointing to the new repo saying it is the new upstream.

This means the least amount of work for you. And the quickest way to a new JasPer for the rest of the world ;)

[SoapGentoo]

@mdadams please make jasper usable for the community by transferring it. Otherwise more and more distributions and packagers will remove it from their repositories.

[jubalh]

If it won't be transferred soonish we will just start creating releases (and continue maintaining) at our fork: https://github.com/jasper-maint/jasper

In that scenario openSUSE will definitely switch to that fork.

[jubalh]

@mdadams I hope you can find some time to respond.

If you won't respond within a week we will:

• Write a ChangeLog
• Create a new release (2.0.17 or 3.0.0 @MaxKellermann any preference?)
• Still use the name "jasper"
• Notify potential users about our release and project

[MaxKellermann]

I say 2.0.17, because the release is supposed to be an ABI-compatible drop-in for 2.0.16.
JasPer's ABI is very less-than-optimal because it uses the notoriously slow (and badly named) int_fast32_t. Breaking the ABI would have lots of advantages (like jasper-maint/jasper@52cb87e). Also, JasPer exposes too many internals in its API/ABI. I have lots of ideas and good reasons for breaking the ABI, but I'm not sure if we'll ever do that. There may never be JasPer version 3 with an optimized ABI.

[jubalh]

I say 2.0.17, because the release is supposed to be an ABI-compatible drop-in for 2.0.16.

Agreed. Though in the past it seems .so name bump and ABI incompatibly wasn't related (although of course it should) ;) https://abi-laboratory.pro/index.php?view=timeline&l=jasper

[mdadams]

I say 2.0.17, because the release is supposed to be an ABI-compatible drop-in for 2.0.16.
JasPer's ABI is very less-than-optimal because it uses the notoriously slow (and badly named) int_fast32_t. Breaking the ABI would have lots of advantages (like jasper-maint/jasper@52cb87e). Also, JasPer exposes too many internals in its API/ABI. I have lots of ideas and good reasons for breaking the ABI, but I'm not sure if we'll ever do that. There may never be JasPer version 3 with an optimized ABI.

@MaxKellermann and @jubalh
I have manually merged all of the changes from your repository (as of earlier today) into a new jasper-maint branch, with two minor modifications (namely, the README and reviving one deleted file). I also tagged the commit with version-2.0.17 as Max suggested. If there is a more optimal way that I could have used to pickup your changes with some additional metadata but still make a few modifications before propagating the result to my repository, please let me know and I can do that. I wasn't sure if I could do this by forking and merging and still apply some changes before propagating the result to the original repository. So, I did what I knew would work so as not to delay things more. I also sent an email to each of you with some additional details and discussion that is probably better handed via email.

[jubalh]

@mdadams I answered to all your mails now but didn't get any reply.
So I'll move the discussion here again just to have it transparent for everybody.

If we want to merge the project together we should be quick. Otherwise people create PRs to one or the other and things might diverge..

For example you received: #218
While we received: jasper-maint/jasper#45

And since you last reviewed our changes I also merged: jasper-maint/jasper#44

So please let's hurry to come to a conclusion together.

Like I wrote in the mail your changes in 3b9c307 are not good.
They add all our changes as one single commit. But we had something over 100 commits, which contained also detailed descriptions in the commit messages and references to issues.

Also then just tagging a release without clearing up the state of the project and not having a proper changelog is also not good in my opinion.

You mentioned that you merged all our changes but two into that commit. It's very hard to find out which these were.
For this reason you should have merged so that each commit stays one commit, and then did two revert commits for the commits which you don't agree with.

So please now do this.

• merge our changes
• git revert cbcaf65 (and add a commit message describing why its reverted)
• git revert (second thing you didnt like)
• push this to master

Then we can discuss how we will go about the ChangeLog. Since this release fixes all known CVEs for JasPer it's important to highlight this to the user.

If you are not that familiar with git, please tell me the two commits you don't like. Give me write access to this repo and I'll do it for you.
But let's do it soon.

[theta682]

@jubalh, @MaxKellermann: I see the version 2.0.17 was released (from a local jasper-maint branch, not master) with all commits from jasper-maint/jasper repository and two commits on top. Now we have official release which includes all your hard work.
However, I agree with you that this project needs a collaborative team that can fast fix security vulnerabilities. I like @MaxKellermann ideas to improve performance.

[theta682]

@jubalh, @MaxKellermann thank you for great work!

[osamu620]

In terms of compliance with the international standard, PR jasper-maint/jasper#45 is very important. Now the JPEG group is looking for a long-term way how to maintain reference software packages. Depending on future discussions and if Jasper does not comply with the standard, it may become necessary to drop Jasper from the official reference software.

[jubalh]

@mdadams good! The work on the now new jasper-maint branch (apparently the one where you tagged last is now jasper-maint-old) looks much better! Up to 50bd3ad.

08db440 could have been done in two commits and using git revert. But its no big deal.

I think we should wait with tagging until we are sure all is fine in general :-)

I see the version 2.0.17 was released (from a local jasper-maint branch, not master) with all commits from jasper-maint/jasper repository and two commits on top.

I also think we should tag on master only :-)

In terms of compliance with the international standard, PR jasper-maint/jasper#45 is very important.

Yep! We won't forget to review/merge it.

Now the JPEG group is looking for a long-term way how to maintain reference software packages. Depending on future discussions and if Jasper does not comply with the standard, it may become necessary to drop Jasper from the official reference software.

I think slowly we are gaining ground and JasPer will survive. Which means we will try to comply with the standard and will review pull requests (once all the current confusion has settled). So no worries :-)

[jubalh]

I merged all our commits (as of today) from https://github.com/jasper-maint/jasper to the master branch of https://github.com/jasper-software/jasper.

I then reverted the note about the fork in the README: f74f882
Updated the links in the README from mdadams to jasper-software 0663cdc

Added a (not the nicest) changelog: eaa7abb containing mentions of all the CVEs we fixed.
Certainly this can be improved but I hope it is okay for now.

And I bumpbed the version to 2.0.19 and tagged it: 7d8cfd8

My goal is now to move jasper-maint/jasper#45 from jasper-maint to jasper-software.
Then hoping @MaxKellermann and @mdadams can review this PR. So it can be part of the next release.

Then I plan to archive/disable pull requests/issues on jasper-maint and add a note that we merged here in jasper-software with the original author. I hope this reduced confusion for users.

https://repology.org/project/jasper/versions shows that distributions also are confused already.
In the Arch AUR they use the 2.0.17 already even though it was tagged two times on different branches/commits. Not good.

Slackware and CRUX already use a 2.0.18.

We need to align all this and make it more visible to users whats going on behind the scenes.

So I took the liberty to do the changes mentioned above.
From here we can continue to work on JasPer together in one repository under the lead of @mdadams.

@mdadams you told us you wanted to review our changes first before merging them on master. But the confusion that we now have needed correction. This is why I did the above steps. Otherwise we end up in a huge confused mess.

You can still review all our changes and then use git rever commit-hash to rever what you don't like. Preferable only after we discussed it together. You can then still create a new release (tagged on master) that is the way you want it.
But having a 2.0.17 with our changes out in public for people to use it and not having them on master because you first want to review it, makes little sense to me.

I hope now all projects switch to 2.0.19 and we can continue to work all together on JasPer.

[jubalh]

@thoger maybe you can inform the Fedora/RedHat maintainer
@apoleon AFAIK you worked on JasPer for Debian LTS, maybe you want to add it back into Debian?
@ncopa maybe you want to update the Alpine package
@anthraxx maybe you want to update the Arch package
@SoapGentoo I created https://bugs.gentoo.org/734284

Submission to add JasPer 2.0.19 again to the official openSUSE repo.

I informed Digikam that there is no need to replace JasPer anymore.

[SoapGentoo]

@jubalh thanks, I'll look into readding jasper back to Gentoo.

[ncopa]

@jubalh Impressive work you have done here. Thank you!

[mdadams]

@jubalh and @MaxKellermann:
Thank you for your hard work, Michael and Max.

I'm sorry about the tag issue. This was my fault. Since 2.0.17 was used to tag different commits at different points in time, it might be best to delete this tag because any references to it might not really be referencing what people think it references. At least if the tag were deleted, people would know that something is wrong, rather than unknowingly referencing the wrong commit.

[mdadams]

@jubalh: Okay, I understand your rationale for merging the changes now. It makes sense, given the circumstances. I'm sorry for the botched 2.0.17 tag. I can review the merged changes after the fact. Perhaps, merging sooner will also help to consolidate the various JasPer efforts more quickly, which would only be a good thing.

[Justinzobel]

Oh damn I missed this issue's birthday, happy birthday issue for 26 days ago!

[jubalh]

All our fixes and improvements have been merged from jasper-maint/jasper into jasper-software/jasper.

The last pull request moved from jasper-maint/jasper#45 to #221 for further review and discussion.

I put a note in the about section at https://github.com/jasper-maint/jasper and archived the repository.

Current state of this project:

• all known CVEs fixed.
• 3 people with commit rights (bottleneck solved)
• more contributors
• distributions notified

[anthraxx]

The 2.0.17 is already release in some distros. Please don't simply delete a tag or do things like re-tagging. If you want to correct this please release an appropriate followup version 2.0.18.

[jubalh]

@anthraxx maybe you can start by at least reading the last couple of comments in this thread...

[anthraxx]

@jubalh I'm following the whole thread in fact since it has been born. I'm just saying its in my humble opinion not a good idea to delete tags. It can be annotated in the release description that its a broken version or something.
The last sentences clearly say what i tried to summarize, please never re-tag tags as you yourself said "it was tagged two times on different branches/commits".

[jubalh]

@anthraxx You are saying what we have said since many comments. Plus the addition to not delete the tag.
And then you say we should create a follow up tag "2.0.18" when we have already released 2.0.19. The reason for 19 was also discussed in this thread.

[anthraxx]

@jubalh I think you are not really reading it in a way it was meant -- and/or maybe I just wrote it badly in a non unambiguous way. That was meant as an example what should be done instead of re-tagging. Yes, it may have been just a +1, but I wanted to add a written feedback from another distro packager, consider it more like a +1 ACK but with the difference I would not delete tags even if they are faulty.
I'm following here because we were also in the middle of nuking jasper out of existence in our distro

[jubalh]

Buildroot has been updated to 2.0.19 too: buildroot/buildroot@d0f7b24