Merge pull request whipper-team#411 from ddevault/py3

Python 3 port

From now on whipper's codebase will be compatible only with Python 3.
NOTE: This pull request introduces a regression: more details in whipper-team#424.

Special thanks to @ddevault for kickstarting the porting effort!

.travis.yml

@@ -3,7 +3,7 @@ sudo: required
 
 language: python
 python:
-  - "2.7"
+  - "3.5"
 virtualenv:
   system_site_packages: false
 

Dockerfile

@@ -1,11 +1,11 @@
 FROM debian:buster
 
 RUN apt-get update \
-  && apt-get install -y autoconf cdrdao curl eject flac git libiso9660-dev \
-  libsndfile1-dev libtool locales make pkgconf python-gobject-2 \
-  python-musicbrainzngs python-mutagen python-pip python-requests \
-  python-ruamel.yaml python-setuptools sox swig \
-  && pip install pycdio==2.1.0
+  && apt-get install -y autoconf cdrdao curl eject flac gir1.2-glib-2.0 git libiso9660-dev \
+  libsndfile1-dev libtool locales make pkgconf python3-gi \
+  python3-musicbrainzngs python3-mutagen python3-pip python3-requests \
+  python3-ruamel.yaml python3-setuptools sox swig \
+  && pip3 install pycdio==2.1.0
 
 # libcdio-paranoia / libcdio-utils are wrongfully packaged in Debian, thus built manually
 # see https://github.com/whipper-team/whipper/pull/237#issuecomment-367985625
@@ -44,7 +44,7 @@ RUN echo "LC_ALL=en_US.UTF-8" >> /etc/environment \
 # install whipper
 RUN mkdir /whipper
 COPY . /whipper/
-RUN cd /whipper && python2 setup.py install \
+RUN cd /whipper && python3 setup.py install \
   && rm -rf /whipper \
   && whipper -v
 

README.md

@@ -8,14 +8,12 @@
 [![GitHub Issues](https://img.shields.io/github/issues/whipper-team/whipper.svg)](https://github.com/whipper-team/whipper/issues)
 [![GitHub contributors](https://img.shields.io/github/contributors/whipper-team/whipper.svg)](https://github.com/whipper-team/whipper/graphs/contributors)
 
-Whipper is a Python 2.7 CD-DA ripper based on the [morituri project](https://github.com/thomasvs/morituri) (_CDDA ripper for *nix systems aiming for accuracy over speed_). It started just as a fork of morituri - which development seems to have halted - merging old ignored pull requests, improving it with bugfixes and new features. Nowadays whipper's codebase diverges significantly from morituri's one.
+Whipper is a Python 3 (3.5+) CD-DA ripper based on the [morituri project](https://github.com/thomasvs/morituri) (_CDDA ripper for *nix systems aiming for accuracy over speed_). It started just as a fork of morituri - which development seems to have halted - merging old ignored pull requests, improving it with bugfixes and new features. Nowadays whipper's codebase diverges significantly from morituri's one.
 
 Whipper is currently developed and tested _only_ on Linux distributions but _may_ work fine on other *nix OSes too.
 
 In order to track whipper's latest changes it's advised to check its commit history (README and [CHANGELOG](#changelog) files may not be comprehensive).
 
-We've nearly completed porting the codebase to Python 3 (Python 2 won't be supported anymore in future releases). If you would like to follow the progress of the port e/o help us with it, please check [pull request #411](https://github.com/whipper-team/whipper/pull/411).
-
 ## Table of content
 
 - [Rationale](#rationale)
@@ -27,8 +25,7 @@ We've nearly completed porting the codebase to Python 3 (Python 2 won't be suppo
 - [Building](#building)
   1. [Required dependencies](#required-dependencies)
   2. [Fetching the source code](#fetching-the-source-code)
-  3. [Building the bundled dependencies](#building-the-bundled-dependencies)
-  4. [Finalizing the build](#finalizing-the-build)
+  3. [Finalizing the build](#finalizing-the-build)
 - [Usage](#usage)
 - [Getting started](#getting-started)
 - [Configuration file documentation](#configuration-file-documentation)
@@ -123,33 +120,34 @@ If you are building from a source tarball or checkout, you can choose to use whi
 
 Whipper relies on the following packages in order to run correctly and provide all the supported features:
 
-- [cd-paranoia](https://www.gnu.org/software/libcdio/), for the actual ripping
+- [cd-paranoia](https://github.com/rocky/libcdio-paranoia), for the actual ripping
   - To avoid bugs it's advised to use `cd-paranoia` versions ≥ **10.2+0.94+2-2**
   - The package named `libcdio-utils`, available on Debian and Ubuntu, is affected by a bug (except for Debian testing/sid): it doesn't include the `cd-paranoia` binary (needed by whipper). For more details see: [#888053 (Debian)](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=888053), [#889803 (Debian)](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=889803) and [#1750264 (Ubuntu)](https://bugs.launchpad.net/ubuntu/+source/libcdio/+bug/1750264).
 - [cdrdao](http://cdrdao.sourceforge.net/), for session, TOC, pre-gap, and ISRC extraction
 - [GObject Introspection](https://wiki.gnome.org/Projects/GObjectIntrospection), to provide GLib-2.0 methods used by `task.py`
 - [PyGObject](https://pypi.org/project/PyGObject/), required by `task.py`
-- [python-musicbrainzngs](https://github.com/alastair/python-musicbrainzngs), for metadata lookup
-- [python-mutagen](https://pypi.python.org/pypi/mutagen), for tagging support
-- [python-setuptools](https://pypi.python.org/pypi/setuptools), for installation, plugins support
-- [python-requests](https://pypi.python.org/pypi/requests), for retrieving AccurateRip database entries
+- [musicbrainzngs](https://pypi.org/project/musicbrainzngs/), for metadata lookup
+- [mutagen](https://pypi.python.org/pypi/mutagen), for tagging support
+- [setuptools](https://pypi.python.org/pypi/setuptools), for installation, plugins support
+- [requests](https://pypi.python.org/pypi/requests), for retrieving AccurateRip database entries
 - [pycdio](https://pypi.python.org/pypi/pycdio/), for drive identification (required for drive offset and caching behavior to be stored in the configuration file).
   - To avoid bugs it's advised to use the most recent `pycdio` version with the corresponding `libcdio` release or, if stuck to old pycdio versions, **0.20**/**0.21** with `libcdio` ≥ **0.90** ≤ **0.94**. All other combinations won't probably work.
 - [ruamel.yaml](https://pypi.org/project/ruamel.yaml/), for generating well formed YAML report logfiles
 - [libsndfile](http://www.mega-nerd.com/libsndfile/), for reading wav files
 - [flac](https://xiph.org/flac/), for reading flac files
 - [sox](http://sox.sourceforge.net/), for track peak detection
+- [git](https://git-scm.com/) or [mercurial](https://www.mercurial-scm.org/)
+  - Required either when running whipper without installing it or when building it from its source code (code cloned from a git/mercurial repository).
 
 Some dependencies aren't available in the PyPI. They can be probably installed using your distribution's package manager:
 
-- [cd-paranoia](https://www.gnu.org/software/libcdio/)
+- [cd-paranoia](https://github.com/rocky/libcdio-paranoia)
 - [cdrdao](http://cdrdao.sourceforge.net/)
 - [GObject Introspection](https://wiki.gnome.org/Projects/GObjectIntrospection)
 - [libsndfile](http://www.mega-nerd.com/libsndfile/)
 - [flac](https://xiph.org/flac/)
 - [sox](http://sox.sourceforge.net/)
 - [git](https://git-scm.com/) or [mercurial](https://www.mercurial-scm.org/)
-  - Required either when running whipper without installing it or when building it from its source code (code cloned from a git/mercurial repository).
 
 PyPI installable dependencies are listed in the [requirements.txt](https://github.com/whipper-team/whipper/blob/master/requirements.txt) file and can be installed issuing the following command:
 
@@ -164,22 +162,9 @@ git clone https://github.com/whipper-team/whipper.git
 cd whipper
 ```
 
-### Building the bundled dependencies
-
-Whipper uses and packages a slightly different version of the `accuraterip-checksum` tool:
-
-You can edit the install path in `config.mk`
-
-```bash
-cd src
-make
-sudo make install
-cd ..
-```
-
 ### Finalizing the build
 
-Install whipper: `python2 setup.py install`
+Install whipper: `python3 setup.py install`
 
 Note that, depending on the chosen installation path, this command may require elevated rights.
 
@@ -232,7 +217,7 @@ The configuration file is stored in `$XDG_CONFIG_HOME/whipper/whipper.conf`, or
 
 See [XDG Base Directory
 Specification](http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html)
-and [ConfigParser](https://docs.python.org/2/library/configparser.html).
+and [ConfigParser](https://docs.python.org/3/library/configparser.html).
 
 The configuration file consists of newline-delineated `[sections]`
 containing `key = value` pairs. The sections `[main]` and
@@ -271,7 +256,7 @@ To make it easier for developers, you can run whipper straight from the
 source checkout:
 
 ```bash
-python2 -m whipper -h
+python3 -m whipper -h
 ```
 
 ## Logger plugins
@@ -298,8 +283,10 @@ Whipper searches for logger plugins in the following paths:
 On a default Debian/Ubuntu installation, the following paths are searched by whipper:
 
 - `$HOME/.local/share/whipper/plugins`
-- `/usr/local/lib/python2.7/dist-packages/whipper/plugins`
-- `/usr/lib/python2.7/dist-packages/whipper/plugins`
+- `/usr/local/lib/python3.X/dist-packages/whipper/plugins`
+- `/usr/lib/python3.X/dist-packages/whipper/plugins`
+
+Where `X` stands for the minor version of the Python 3 release available on the system.
 
 ### Official logger plugins
 
@@ -336,7 +323,7 @@ Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301  USA
 Make sure you have the latest copy from our [git
 repository](https://github.com/whipper-team/whipper). Where possible,
 please include tests for new or changed functionality. You can run tests
-with `python -m unittest discover` from your source checkout.
+with `python3 -m unittest discover` from your source checkout.
 
 ### Developer Certificate of Origin (DCO)
 
@@ -410,7 +397,7 @@ gzip whipper.log
 
 And attach the gzipped log file to your bug report.
 
-Without `WHIPPER_LOGFILE` set, logging messages will go to stderr. `WHIPPER_DEBUG` accepts a string of the [default python logging levels](https://docs.python.org/2/library/logging.html#logging-levels).
+Without `WHIPPER_LOGFILE` set, logging messages will go to stderr. `WHIPPER_DEBUG` accepts a string of the [default python logging levels](https://docs.python.org/3/library/logging.html#logging-levels).
 
 ## Credits
 

misc/offsets.py

@@ -6,13 +6,12 @@
 
 import sys
 
-import BeautifulSoup
+from bs4 import BeautifulSoup
 
-handle = open(sys.argv[1])
+with open(sys.argv[1]) as f:
+    doc = f.read()
 
-doc = handle.read()
-
-soup = BeautifulSoup.BeautifulSoup(doc)
+soup = BeautifulSoup(doc)
 
 offsets = {}  # offset -> total count
 
@@ -50,18 +49,17 @@
 
 # now format it for code inclusion
 lines = []
-line = 'OFFSETS = "'
+line = 'OFFSETS = ("'
 
 for offset in offsets:
     line += offset + ", "
     if len(line) > 60:
-        line += "\" + \\"
         lines.append(line)
-        line = '          "'
+        line = '           "'
 
 # get last line too, trimming the comma and adding the quote
 if len(line) > 11:
-    line = line[:-2] + '"'
+    line = line[:-2] + '")'
     lines.append(line)
 
 print("\n".join(lines))

setup.py

@@ -8,7 +8,7 @@
     maintainer=['The Whipper Team'],
     url='https://github.com/whipper-team/whipper',
     license='GPL3',
-    python_requires='>=2.7,<3',
+    python_requires='>=3.5',
     packages=find_packages(),
     setup_requires=['setuptools_scm'],
     ext_modules=[

src/accuraterip-checksum.c

@@ -147,7 +147,13 @@ static PyMethodDef accuraterip_methods[] = {
 	{ NULL, NULL, 0, NULL },
 };
 
-PyMODINIT_FUNC initaccuraterip(void)
+static struct PyModuleDef accuraterip_module = {
+	.m_base = PyModuleDef_HEAD_INIT,
+	.m_name = "accuraterip",
+	.m_methods = accuraterip_methods,
+};
+
+PyMODINIT_FUNC PyInit_accuraterip(void)
 {
-	Py_InitModule("accuraterip", accuraterip_methods);
+	return PyModule_Create(&accuraterip_module);
 }

whipper/command/cd.py

@@ -37,8 +37,8 @@
 SILENT = 0
 MAX_TRIES = 5
 
-DEFAULT_TRACK_TEMPLATE = u'%r/%A - %d/%t. %a - %n'
-DEFAULT_DISC_TEMPLATE = u'%r/%A - %d/%A - %d'
+DEFAULT_TRACK_TEMPLATE = '%r/%A - %d/%t. %a - %n'
+DEFAULT_DISC_TEMPLATE = '%r/%A - %d/%A - %d'
 
 TEMPLATE_DESCRIPTION = '''
 Tracks are named according to the track template, filling in the variables
@@ -137,7 +137,7 @@ def do(self):
         if getattr(self.options, 'working_directory', False):
             os.chdir(os.path.expanduser(self.options.working_directory))
         if hasattr(self.options, 'output_directory'):
-            out_bpath = self.options.output_directory.decode('utf-8')
+            out_bpath = self.options.output_directory
             # Needed to preserve cdrdao's tocfile
             out_fpath = self.program.getPath(out_bpath,
                                              self.options.disc_template,
@@ -295,10 +295,9 @@ def handle_arguments(self):
         self.options.output_directory = os.path.expanduser(
             self.options.output_directory)
 
-        self.options.track_template = self.options.track_template.decode(
-            'utf-8')
+        self.options.track_template = self.options.track_template
         validate_template(self.options.track_template, 'track')
-        self.options.disc_template = self.options.disc_template.decode('utf-8')
+        self.options.disc_template = self.options.disc_template
         validate_template(self.options.disc_template, 'disc')
 
         if self.options.offset is None:
@@ -323,7 +322,7 @@ def handle_arguments(self):
 
     def doCommand(self):
         self.program.setWorkingDirectory(self.options.working_directory)
-        self.program.outdir = self.options.output_directory.decode('utf-8')
+        self.program.outdir = self.options.output_directory
         self.program.result.offset = int(self.options.offset)
         self.program.result.overread = self.options.overread
         self.program.result.logger = self.options.logger
@@ -336,13 +335,11 @@ def doCommand(self):
         if os.path.exists(dirname):
             logs = glob.glob(os.path.join(dirname, '*.log'))
             if logs:
-                msg = ("output directory %s is a finished rip" %
-                       dirname.encode('utf-8'))
+                msg = ("output directory %s is a finished rip" % dirname)
                 logger.debug(msg)
                 raise RuntimeError(msg)
         else:
-            logger.info("creating output directory %s",
-                        dirname.encode('utf-8'))
+            logger.info("creating output directory %s", dirname)
             os.makedirs(dirname)
 
         # FIXME: turn this into a method
@@ -366,7 +363,7 @@ def _ripIfNotRipped(number):
             logger.debug('ripIfNotRipped: path %r', path)
             trackResult.number = number
 
-            assert isinstance(path, unicode), "%r is not unicode" % path
+            assert isinstance(path, str), "%r is not str" % path
             trackResult.filename = path
             if number > 0:
                 trackResult.pregap = self.itable.tracks[number - 1].getPregap()
@@ -385,7 +382,7 @@ def _ripIfNotRipped(number):
 
                 logger.info('verifying track %d of %d: %s',
                             number, len(self.itable.tracks),
-                            os.path.basename(path).encode('utf-8'))
+                            os.path.basename(path))
                 if not self.program.verifyTrack(self.runner, trackResult):
                     logger.warning('verification failed, reripping...')
                     os.unlink(path)
@@ -403,7 +400,7 @@ def _ripIfNotRipped(number):
                         extra = " (try %d)" % tries
                     logger.info('ripping track %d of %d%s: %s',
                                 number, len(self.itable.tracks), extra,
-                                os.path.basename(path).encode('utf-8'))
+                                os.path.basename(path))
                     try:
                         logger.debug('ripIfNotRipped: track %d, try %d',
                                      number, tries)

whipper/command/image.py

@@ -45,7 +45,6 @@ def do(self):
         runner = task.SyncRunner()
 
         for arg in self.options.cuefile:
-            arg = arg.decode('utf-8')
             cueImage = image.Image(arg)
             cueImage.setup(runner)
 

whipper/command/mblookup.py

@@ -17,7 +17,7 @@ def add_arguments(self):
 
     def do(self):
         try:
-            discId = unicode(self.options.mbdiscid)
+            discId = str(self.options.mbdiscid)
         except IndexError:
             print('Please specify a MusicBrainz disc id.')
             return 3
@@ -29,7 +29,7 @@ def do(self):
             print('- Release %d:' % (i + 1, ))
             print('    Artist: %s' % md.artist.encode('utf-8'))
             print('    Title:  %s' % md.title.encode('utf-8'))
-            print('    Type:   %s' % unicode(md.releaseType).encode('utf-8'))  # noqa: E501
+            print('    Type:   %s' % str(md.releaseType).encode('utf-8'))  # noqa: E501
             print('    URL:    %s' % md.url)
             print('    Tracks: %d' % len(md.tracks))
             if md.catalogNumber:

whipper/command/offset.py

@@ -177,7 +177,7 @@ def _arcs(self, runner, table, track, offset):
         logger.debug('ripping track %r with offset %d...', track, offset)
 
         fd, path = tempfile.mkstemp(
-            suffix=u'.track%02d.offset%d.whipper.wav' % (
+            suffix='.track%02d.offset%d.whipper.wav' % (
                 track, offset))
         os.close(fd)