Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite in Python 3.7 #7

Merged
merged 92 commits into from
Jun 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
a84a215
readme: Remove extra parenthesis
Mar 22, 2020
4e0d635
wrapper: Update qvm-convert-pdf into Python 3
Mar 24, 2020
7ef5b33
wrapper: Add logging and trim options
Mar 25, 2020
9668bfb
wrapper: Prepare for multiple file support
Mar 25, 2020
a014fb3
wrapper: Remove unneeded main() try block
Mar 25, 2020
fd9175f
wrapper: Remove logging
Mar 26, 2020
0a7cfae
client: Update to Python 3
Mar 26, 2020
4d88ecf
wrapper, client: Re-add logging
Mar 29, 2020
588a9e2
wrapper: PEP 8
Mar 29, 2020
516cd21
client: Add qrexec wrapper functions
Mar 29, 2020
552d2f4
client: PEP 8
Mar 29, 2020
f6c90fb
server: Refactor to Python 3
Mar 29, 2020
8a8283d
client, server: Add documentation
Mar 29, 2020
5cfaab7
server: Make check_output() return a String instead of a bytestring
Mar 29, 2020
ec754fc
wrapper: Add documentation
Mar 29, 2020
b8b41da
Add .py extensions and update install commands
Mar 30, 2020
150793c
readme: Update usage example
Mar 31, 2020
21407f1
wrapper: Fix leftover debugging command
Mar 31, 2020
3d941e8
wrapper: Trim down argument parsing
Mar 31, 2020
c6831f5
wrapper: Move logging config to global namespace
Mar 31, 2020
2857aa4
client, server: Fix stream flushing
Mar 31, 2020
8d5bc16
client: Handle EOFErrors if server suddenly dies
Mar 31, 2020
f937480
server: Avoid errors if subprocess command fails
Mar 31, 2020
dd4abde
server, client: Grammar
Mar 31, 2020
82b7e73
client: Use OG stdout file descriptor
Mar 31, 2020
e43c071
server: Fix location of STDOUT flush call
Mar 31, 2020
098ef25
wrapper: Simplify argument parsing
Apr 1, 2020
88acf01
wrapper: Add non-file check
Apr 1, 2020
7279415
wrapper, client, server: Qualify variable names
Apr 2, 2020
a4b70cb
server, client: Remove unecessary documentation
Apr 2, 2020
574e63c
readme: Update language
Apr 2, 2020
54e25fe
Add support for multiple files
Apr 2, 2020
3fd5912
Update docs
Apr 2, 2020
672ad1a
client, server: Replace recv() with recvline_b()
Apr 3, 2020
7913920
client: Remove redundant check condition
Apr 18, 2020
c6f6be4
client, server: Add custom exceptions
Apr 18, 2020
8178f1d
client, server: Update utility functions
Apr 18, 2020
f66da6e
client: Combine image dimension functions
Apr 18, 2020
4d1ac9e
client, server: Update initial phase
Apr 18, 2020
f4291c3
client, server: Update to asyncio and new model
Apr 18, 2020
5979f9f
client, server: Move to pdf-converter directory
Apr 18, 2020
23a45e5
meta: Remove extra file
Apr 18, 2020
456ac9a
tests: Moved to pdf-converter/tests
Apr 18, 2020
a0cffae
meta: Remove extra file
Apr 18, 2020
d7dade5
meta: Remove wrapper
Apr 18, 2020
ee7420d
client: Fix error handling for readexactly()
Apr 19, 2020
e057643
client: Remove extra byte for send()
Apr 19, 2020
1c4f61d
client: Remove unneeded PID prefix for tmpdir
Apr 19, 2020
523aab8
client: Remove unused recv()
Apr 19, 2020
8dd8d6f
server: Remove uneeded quoting in identify command
Apr 19, 2020
11d610a
Rename source directory
Apr 20, 2020
8905550
client: Start batch processing code
Apr 23, 2020
8c0054c
client: Add CLI parameter handling
Apr 25, 2020
89ebda3
client: Update error handling
Apr 25, 2020
06bafaf
server: Remove recv_pdf()
Apr 26, 2020
1fa078b
server: Update error handling
Apr 26, 2020
7c9a21f
server: Update sending process
Apr 26, 2020
dc4742b
client: Replace click.echo()
Apr 26, 2020
4e5e87e
client, server: Add Representation & *File objects
Apr 27, 2020
3fd4636
server: Remove nonexistent PageError
Apr 27, 2020
4e6b286
client, server: Remove unused imports
Apr 28, 2020
24dc64f
client, server: Rename unsanitized file class
Apr 28, 2020
225780e
client: Add --archive, --dry-run, and --in-place
May 2, 2020
9a61502
client: Implement --in-place
May 2, 2020
2a5d8b9
client: Add initial version of tqdm output
May 4, 2020
924824f
client: Replace namedtuple with dataclass
May 19, 2020
7243999
client, server: Update UI, design, and error handling
May 19, 2020
f05f4d5
client: Fix lower bound for batch size
May 19, 2020
21a6314
client: Update shebang to 3.7
May 19, 2020
5ac1bf0
setup: Add tqdm dependency
May 19, 2020
4cfb5e9
client: Update docstring
May 19, 2020
35524b7
client, server: Replace shellcheck with pylint
May 28, 2020
32dd0d2
travis: Add pylint dependency
May 28, 2020
2718653
travis: fix paths and try without --exit-code
May 28, 2020
4e3fee3
travis: Re-add --exit-code because i was right the first time
May 28, 2020
0b1e80c
travis: Attempt to fix pylint
May 31, 2020
467a7c3
travis: remove --exit-code
May 31, 2020
deb6c87
travis: Update imports and pylint exceptions
May 31, 2020
61437ca
client: Use python3 shebang
Jun 14, 2020
5772fdf
server: Handle non-ASCII output from pdfinfo(1)
Jun 14, 2020
c00e7a1
server: Prevent CPU hogging by PNG tasks
Jun 16, 2020
8e32abb
server: Rename batch entry variables
Jun 16, 2020
bcbaf9e
client: Implement bulk saves and remove reps appropriately
Jun 16, 2020
ef41913
client: Exit with 1 on error
Jun 16, 2020
2abacda
meta: Copyright info
Jun 16, 2020
9b655af
pylint: Add bad-continuation to .pylintrc
Jun 19, 2020
c87e61b
client: Simplify image appending
Jun 19, 2020
5eab363
client: Fix output spacing
Jun 19, 2020
f1d35f2
client: Handle out of space error
Jun 20, 2020
379659b
client: Add support for older tqdm versions
Jun 21, 2020
64cc14f
pylint: Add expression-not-assigned
Jun 21, 2020
1ee08f7
makefile: Resolve makefile conflict
Jun 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .pylintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
[MASTER]
persistent=no
ignore=qubespdfconverter/tests

[MESSAGES CONTROL]
disable=
bad-continuation,
bare-except,
blacklisted-name,
deprecated-method,
duplicate-code,
expression-not-assigned,
file-ignored,
fixme,
invalid-name,
locally-disabled,
locally-enabled,
missing-docstring,
protected-access,
too-few-public-methods,
unused-argument,
wrong-import-order
17 changes: 9 additions & 8 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
sudo: required
dist: bionic
language: generic
install: git clone https://github.com/QubesOS/qubes-builder ~/qubes-builder
script: ~/qubes-builder/scripts/travis-build
language: python
python:
- '3.7'
install:
- git clone https://github.com/QubesOS/qubes-builder ~/qubes-builder
- pip install --quiet -r ci/requirements.txt
script:
- python3 -m pylint --rcfile=.pylintrc qubespdfconverter
- ~/qubes-builder/scripts/travis-build
env:
- DIST_DOM0=fc25 USE_QUBES_REPO_VERSION=4.0 USE_QUBES_REPO_TESTING=1
- DISTS_VM=fc30 USE_QUBES_REPO_VERSION=4.0 USE_QUBES_REPO_TESTING=1
Expand All @@ -11,8 +17,3 @@ env:
- DISTS_VM=buster USE_QUBES_REPO_VERSION=4.0 USE_QUBES_REPO_TESTING=1
- DIST_DOM0=fc31 USE_QUBES_REPO_VERSION=4.1 USE_QUBES_REPO_TESTING=1
- DISTS_VM=bullseye USE_QUBES_REPO_VERSION=4.1 USE_QUBES_REPO_TESTING=1

jobs:
include:
- script:
- shellcheck qpdf-convert-client qpdf-convert-server
62 changes: 4 additions & 58 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,68 +19,13 @@
#
#

RPMS_DIR=rpm/
VERSION := $(shell cat version)

help:
@echo "Qubes addons main Makefile:" ;\
echo "make rpms <--- make rpms and sign them";\
echo; \
echo "make clean <--- clean all the binary files";\
echo "make update-repo-current <-- copy newly generated rpms to qubes yum repo";\
echo "make update-repo-current-testing <-- same, but for -current-testing repo";\
echo "make update-repo-unstable <-- same, but to -testing repo";\
echo "make update-repo-installer -- copy dom0 rpms to installer repo"
@exit 0;

rpms: rpms-vm

rpms-dom0:
rpmbuild --define "_rpmdir rpm/" -bb rpm_spec/qpdf-converter-dom0.spec
rpm --addsign rpm/x86_64/qubes-pdf-converter-dom0*$(VERSION)*.rpm

rpms-vm:
rpmbuild --define "_rpmdir rpm/" -bb rpm_spec/qpdf-converter.spec
rpm --addsign rpm/x86_64/qubes-pdf-converter*$(VERSION)*.rpm

update-repo-current:
for vmrepo in ../yum/current-release/current/vm/* ; do \
dist=$$(basename $$vmrepo) ;\
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter*$(VERSION)*$$dist*.rpm $$vmrepo/rpm/ ;\
done
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter-dom0-*$(VERSION)*.rpm ../yum/current-release/current/dom0/rpm/

update-repo-current-testing:
for vmrepo in ../yum/current-release/current-testing/vm/* ; do \
dist=$$(basename $$vmrepo) ;\
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter*$(VERSION)*$$dist*.rpm $$vmrepo/rpm/ ;\
done
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter-dom0-*$(VERSION)*.rpm ../yum/current-release/current-testing/dom0/rpm/

update-repo-unstable:
for vmrepo in ../yum/current-release/unstable/vm/* ; do \
dist=$$(basename $$vmrepo) ;\
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter*$(VERSION)*$$dist*.rpm $$vmrepo/rpm/ ;\
done
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter-dom0-*$(VERSION)*.rpm ../yum/current-release/unstable/dom0/rpm/

update-repo-template:
for vmrepo in ../template-builder/yum_repo_qubes/* ; do \
dist=$$(basename $$vmrepo) ;\
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter*$(VERSION)*$$dist*.rpm $$vmrepo/rpm/ ;\
done

update-repo-installer:
ln -f $(RPMS_DIR)/x86_64/qubes-pdf-converter-dom0-*$(VERSION)*.rpm ../installer/yum/qubes-dom0/rpm/

build:
make manpages -C doc

install-vm:
make install -C doc
install -D qvm-convert-pdf $(DESTDIR)/usr/bin/qvm-convert-pdf
install -D qpdf-convert-client $(DESTDIR)/usr/lib/qubes/qpdf-convert-client
install -D qpdf-convert-server $(DESTDIR)/usr/lib/qubes/qpdf-convert-server
install -D qubespdfconverter/client.py $(DESTDIR)/usr/bin/qvm-convert-pdf
install -D qubespdfconverter/server.py $(DESTDIR)/usr/lib/qubes/qpdf-convert-server
install -d $(DESTDIR)/etc/qubes-rpc
ln -s ../../usr/lib/qubes/qpdf-convert-server $(DESTDIR)/etc/qubes-rpc/qubes.PdfConvert
install -D qvm-convert-pdf.gnome $(DESTDIR)/usr/lib/qubes/qvm-convert-pdf.gnome
Expand All @@ -90,7 +35,8 @@ install-vm:
install -m 0644 qvm-convert-pdf.desktop $(DESTDIR)/usr/share/kde4/services

install-dom0:
python2 setup.py install -O1 --root $(DESTDIR)
python3 setup.py install -O1 --root $(DESTDIR)

clean:
rm -rf debian/changelog.*
rm -rf pkgs
40 changes: 19 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,30 @@
Qubes PDF Converter
====================

Qubes PDF converter is a [Qubes](https://qubes-os.org) Application, which
utilizes Qubes flexible qrexec (inter-VM communication) infrastructure and
Disposable VMs to perform conversion of potentially untrusted (e.g. maliciously
malformed) PDF files into safe-to-view PDF files.

This is done by having the Disposable VM perform the complex (and potentially
buggy) rendering of the PDF in question) and sending the resulting RGB bitmap
(simple representation) to the client AppVM. The client AppVM can _trivially_
verify the received data are indeed the simple representation, and then
construct a new PDF out of the received bitmap. Of course the price we pay for
this conversion is loosing any structural information and text-based search in
the converted PDF.

More discussion and introduction of the concept has been described in the
original article
Qubes PDF converter is a [Qubes](https://qubes-os.org) Application that
utilizes Disposable VMs and Qubes' flexible qrexec (inter-VM communication)
infrastructure to securely convert potentially untrusted PDF files into
safe-to-view PDF files.

This is done by having a Disposable VM render each page of a PDF file into a
very simple representation (RGB bitmap) that (presumably) leaves no room for
malicious code. This representation is then sent back to the client AppVM which
then constructs an entirely new PDF file out of the received bitmaps.

More discussion of the concept has been described in the original article
[here](http://blog.invisiblethings.org/2013/02/21/converting-untrusted-pdfs-into-trusted.html).

Usage
------

[user@varia ~]$ qvm-convert-pdf test.pdf
-> Sending file to remote VM...
-> Waiting for converted samples...
-> Receving page 8 out of 8...
-> Converted PDF saved as: ./test.trusted.pdf
-> Original file saved as /home/user/QubesUntrustedPDFs/test.pdf
[user@domU ~]$ qvm-convert-pdf file1.pdf file2.pdf file3.pdf
:: Sending files to Disposable VMs...

file1.pdf...done
file2.pdf...fail
file3.pdf...done

Total Sanitized Files: 2/3

Authors
---------
Expand Down
6 changes: 6 additions & 0 deletions ci/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# WARNING: those requirements are used only for travis-ci.org
# they SHOULD NOT be used under normal conditions; use system package manager
click
pillow
pylint
tqdm
1 change: 0 additions & 1 deletion debian/qubes-pdf-converter.install
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
usr/lib/qubes/qpdf-convert-client
usr/lib/qubes/qpdf-convert-server
etc/qubes-rpc/qubes.PdfConvert
usr/bin/qvm-convert-pdf
Expand Down
27 changes: 14 additions & 13 deletions doc/qvm-convert-pdf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,28 @@ QVM-CONVERT-PDF(1)

NAME
====
qvm-convert-pdf - converts a potentially untrusted pdf to a safe-to-view pdf
qvm-convert-pdf - converts potentially untrusted PDFs to a safe-to-view PDF

SYNOPSIS
========
| qvm-convert-pdf <pdf to convert>
| qvm-convert-pdf <PDF to convert ...>

DESCRIPTION
===========

Qubes PDF converter is a Qubes Application, which utilizes Qubes flexible qrexec
(inter-VM communication) infrastructure and Disposable VMs to perform conversion
of potentially untrusted (e.g. maliciously malformed) PDF files into
safe-to-view PDF files.
Qubes PDF converter is a Qubes Application that utilizes Qubes' flexible qrexec
(inter-VM communication) infrastructure and Disposable VMs to securely convert
potentially untrusted (e.g. maliciously malformed) PDF files into safe-to-view
PDF files.

This is done by having the Disposable VM perform the complex (and potentially
buggy) rendering of the PDF in question) and sending the resulting RGB bitmap
(simple representation) to the client AppVM. The client AppVM can _trivially_
verify the received data are indeed the simple representation, and then
construct a new PDF out of the received bitmap. Of course the price we pay for
this conversion is loosing any structural information and text-based search in
the converted PDF.
This is done by having a Disposable VM render each page of a PDF file into a
very simple representation (RGB bitmap) that (presumably) leaves no room for
malicious code. This representation is then sent back to the client AppVM which
then constructs an entirely new PDF file out of the received bitmaps.

Of course, the price we pay for this conversion is an increase in file size and
the loss of any structural information or text-based search in the converted
PDF.

AUTHORS
=======
Expand Down
150 changes: 0 additions & 150 deletions qpdf-convert-client

This file was deleted.

Loading