Distro packages

[pombredanne]

We should provide and eventually maintain distribution packages as RPMs and Debian packages.

Beside the basic packaging, this would mean also contributing the package to the distro and addressing some issues to make this particular such as:
#469

[v27u27]

Hello Sir, as per my understanding, are you asking to make debain and RPMs distribution package for the scan code toolkit, so that linux users can directly install this toolkit onto their system ?, Please correct me if i am wrong anywhere.

[pombredanne]

@v27u27 Yes this is exactly that. Note that eventually there could be some tools to help build skeletons for these.

[sudeepb02]

Hello @pombredanne,
I'd like to work on this issue as my GSoC project'17 as it is listed here GSoC 2017 nexb/aboutcode wiki

[pombredanne]

Great! Check also #469 and http://Appimage.org that would be a great candidate for Linux packaging.

[sudeepb02]

@pombredanne Thanks. I'll go through the Appimage documentation and start working on it ASAP.

[shivakrishna9]

Hi @pombredanne I would like to give my first PR for this issue as part of gsoc ideas list.

[pombredanne]

@shivakrishna9 sure, that's great.

[shivakrishna9]

@pombredanne please clarify if this project is assigned to me so that I will be able to completely dedicate my time to this. Thanks.

[pombredanne]

@shivakrishna9 I cannot assign to you anything for now:

1. the GSoC has not even started in earnest. You have not suggested a plan for your project submission. And then you may or not be selected. This will happen in April... I cannot tell anything now.

2. I do not know you. I will get to know you through PRs. This is part of our selection process.

So if you want to submit some PRs (which is a requirement if you plan to submit a project application with our org) this is great. I will welcome this and review it then and when you submit this.

[sudeepb02]

@pombredanne I went through the AppImage documentation for distro packages and made an AppImage for a applications. But I see that most of the work here is testing rather than coding.
I was able to run the AppImage successfully on my system but running it on different distros raise some issues which needs the source to be changed.
So can I take this up as a project for GSoC? As the duration for GSoC is 3 months, I'll also create AppImages for AttributeCode and TraceCode

[pombredanne]

@sudeepb02 Great! Can you point me to a public place where you pushed this?
Now this is a GSOC project alright, but to be complete it should cover in order of priorities:

1. Fix De-vendor prebuilt binaries to ease packaging for Linux distros #469 and this will require quite a few code changes such that its works in scancode proper and the scancode-toolkit-contrib too.

2. Create a build loop that handles all the automated compilation of the natives deps on linux/mac and windows . There is some elements of this in https://github.com/pombreda/thirdparty and https://github.com/pombreda/thirdparty-manylinux/ for the python side. And the native code is listed here: https://github.com/nexB/scancode-thirdparty-src and there are some build notes to help.

3. Once we have this in place, create a proper AppImage AND a test suite to validate this is working on may linuxes (may be using OpenSuse OBS, openshift, Travis, circle, etc.) possibly using container images for this

4. create also proper Debian and RPM packages. Create tests for many distros to smoke check the installation. Submit the packages for inclusion in the distros.

5. also create packages and/or ports for other places: e.g. homebrew, archlinux, Guix, FreeBSD

6. also create packages for other languages wrapping ScanCode to run it in a subprocess and get the results back for JavaScript, Java, Ruby. Create proper NPM, Maven and Gem packages and push these to their respective repos . Consider also creating a Maven plugin to scan while you build. Consider other build plugins for similar tools, e.g. Jenkins.

7. do similar things for other tools such as AboutCodeMgr and TraceCode.

8. to 4. are must have. 5. to 7. are bonuses if time permits .

The creation and tests for all this must be automated end-to-end... so I guess this can fill up three months alright. But if this is not the case, there will be for sure a lot of things in that domain that can fill any voids and slack...

[pombredanne]

@sudeepb02 so my point is that you should not underestimate the amount of work needed for what is expected here! Creating the package may end up being a tiny part of the effective work and there is quite a bit of code needed for #469 and to create build and tests scripts that are stable and highly automated.

[sudeepb02]

@pombredanne Can you please explain the second point in some detail

Create a build loop that handles all the automated compilation of the natives deps on linux/mac and windows

I've not tried any automatic building tools but I guess Travis CI would be used for this?

[zjesko]

Hi @pombredanne
Wanted to check the status of this issue. I was really keen on working on it as my GSOC 2020 project. Moreover, would you think it is a good idea to maybe develop a python API for scancode as well so that the users can simply do pip install scancode to use this.

[zjesko]

Hi @pombredanne

Is there more work required on this front or this project has been wrapped up?

[pombredanne]

@akshatcx ... @aj4ayushjain has been working on it ... Ayush what's the status on your side? How could Akshat help there?

[aj4ayushjain]

@pombredanne @akshatcx
I have done the debian(unofficial)package -scancode tool in python2 along with docker image.
So u can help me out with these things:-

1.There is major roadBlock which need's to be sorted to ease packaging and further issues (naming of the python plugins).
2. If u know about official rules and regulations of debian u can help me getting an official debian package or might be get an fedora/rpm package.

[zjesko]

@pombredanne @aj4ayushjain
Sure I would be glad to work on that. Can we maybe discuss this in detail over something like slack?
Meanwhile @pombredanne , is there a particular area of focus where I can start helping out to gain familiarity with the scancode codebase

[Abhishek-Dev09-zz]

@akshatcx
We don't use slack , we are on gitter. You can discuss here here . First you need to start using it to get familiar to know how it works.

[n1khilmane]

@pombredanne @akshatcx
I have done the debian(unofficial)package -scancode tool in python2 along with docker image.
So u can help me out with these things:-

1.There is major roadBlock which need's to be sorted to ease packaging and further issues (naming of the python plugins).
2. If u know about official rules and regulations of debian u can help me getting an official debian package or might be get an fedora/rpm package.

Hey @aj4ayushjain @pombredanne , Regarding the second task mentioned here , how much work is done on getting an official debian package? or rpm/fedora? how can i help in that ?

[aj4ayushjain]

@n1khilmane
This could be pulled off by anybody being an official debian maintainer easily and if you are not one then i guess you need to work on that bit and then publish the package on debian publicly but with the current constraints it's tough though.

[xsuchy]

FYI Scancode-toolkit is packaged as an RPM in official Fedora repos for some time. See https://src.fedoraproject.org/rpms/scancode-toolkit