Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm backend LTO docs and testing #8942

Merged
merged 3 commits into from
Jul 15, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion site/source/docs/optimizing/Optimizing-Code.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ In addition to the above, the following tips can help to reduce code size:

- Use :ref:`the closure compiler <emcc-closure>` on the non-compiled code: ``--closure 1``. This can hugely reduce the size of the support JavaScript code, and is highly recommended. However, if you add your own additional JavaScript code (in a ``--pre-js``, for example) then you need to make sure it uses `closure annotations properly <https://developers.google.com/closure/compiler/docs/api-tutorial3>`_.
- `Floh's blogpost on this topic <http://floooh.github.io/2016/08/27/asmjs-diet.html>`_ is very helpful.
- Use :ref:`llvm-lto <emcc-llvm-lto>` when compiling from bitcode to JavaScript: ``--llvm-lto 1``. This can break some code as the LTO code path is less tested.
- Make sure to use gzip compression on your webserver, which all browsers now support.
- You can move some of your code into the `Emterpreter <https://github.com/emscripten-core/emscripten/wiki/Emterpreter>`_, which will then run much slower (as it is interpreted), but it will transfer all that code into a smaller amount of data.

Expand All @@ -90,6 +89,20 @@ The following compiler settings can help (see ``src/settings.js`` for more detai
- The ``ENVIRONMENT`` flag lets you specify that the output will only run on the web, or only run in node.js, etc. This prevents the compiler from emitting code to support all possible runtime environments, saving ~2KB.
- You can use ``ELIMINATE_DUPLICATE_FUNCTIONS`` to remove duplicate functions, which C++ templates often create. (This is already done by default for wasm, in ``-O1`` and above.)

LTO
===

Link Time Optimization (LTO) lets the compiler do more optimizations, as it can inline across separate compilation units, and even with system libraries. The :ref:`main relevant flag <emcc-llvm-lto>` is ``--llvm-lto 1`` at link time.

Separately from that flag, the linker must also receive LLVM bitcode files in order to run LTO on them. With fastcomp that is always the case; with the LLVM wasm backend, object files main contain either wasm or bitcode. The linker can handle a mix of the two, but can only do LTO on the bitcode files. You can control that with the following flags:

- The ``-flto`` flag tells the compiler to emit bitcode in object files, but does *not* affect system libraries.
- The ``-s WASM_OBJECT_FILES=0`` flag also tells the compiler to emit bitcode in object files (like ``-flto``), and also to emit bitcode in system libraries.

Thus, to allow maximal LTO opportunities with the LLVM wasm backend, build all source files with ``-s WASM_OBJECT_FILES=0`` and link with ``-s WASM_OBJECT_FILES=0 --llvm-lto 1``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is OK as a documentation of the existing status quo, but it's still pretty awkward and there are a lot of combinations of flags/behavior, not all of which are not useful. So it would be nice to simplify or line up with native a bit better. For example WASM_OBJECT_FILES is a legacy and it controls a lot of code that we can rip out once fastcomp is gone (we should be able to do what native platforms do and use lld to drive the LTO pipeline rather than driver code). Also if the only delta from -flto to NATIVE_OBJECT_FILES is that it also affects system libraries, then the name should reflect that, and it should only be required/allowed at link time. And --llvm-lto is a meaningless name when everything is done with LLVM.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed about simplifying this stuff later, yes.


Note that older versions of LLVM had bugs in this area. With the older fastcomp backend LTO should be used carefully.

Very large codebases
====================

Expand Down
11 changes: 11 additions & 0 deletions tests/test_other.py
Original file line number Diff line number Diff line change
Expand Up @@ -8262,6 +8262,17 @@ def test_wasm_backend_lto(self):
def test_wasm_backend_lto_libcxx(self, *args):
run_process([PYTHON, EMXX, path_from_root('tests', 'hello_libcxx.cpp')] + ['-s', 'WASM_OBJECT_FILES=0'] + list(args))

@no_fastcomp('wasm backend lto specific')
def test_lto_flags(self):
for flags, expect_bitcode in [
([], False),
(['-s', 'WASM_OBJECT_FILES=0'], True),
(['-flto'], True),
]:
run_process([PYTHON, EMCC, path_from_root('tests', 'hello_world.cpp')] + flags + ['-c', '-o', 'a.o'])
seen_bitcode = Building.is_bitcode('a.o')
self.assertEqual(expect_bitcode, seen_bitcode, 'must emit LTO-capable bitcode when flags indicate so (%s)' % str(flags))

def test_wasm_nope(self):
for opts in [[], ['-O2']]:
print(opts)
Expand Down