Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test analyzer with asan and ubsan #96

Merged
merged 3 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,16 @@ jobs:
with:
submodules: "true"

- name: Generate a timestamp
run: echo "TIMESTAMP=$(date +%Y%m%d%H%M%S)" >>"$GITHUB_ENV"

- uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.ccache
key: native
key: native-${{ env.TIMESTAMP }}
restore-keys: native-

- name: Configure ccache
run: echo "CCACHE_DIR=$HOME/.ccache" >>"$GITHUB_ENV"
Expand All @@ -63,11 +67,12 @@ jobs:
run: PATH=/usr/lib/ccache:$PATH ./ci
--skip-repair
-DVALGRIND_CONFIGURE_FLAGS=--disable-dependency-tracking
--sanitize

- name: Smoke test
run: python3 -m pip install --user dist/wheelhouse/*.whl &&
memtrace record true &&
memtrace report >/dev/null
./python3-asan "$(command -v memtrace)" record true &&
./python3-asan "$(command -v memtrace)" report | tail

- name: Show and reset ccache stats
run: ccache --show-stats --verbose --zero-stats
Expand All @@ -90,12 +95,16 @@ jobs:
with:
submodules: "true"

- name: Generate a timestamp
run: echo "TIMESTAMP=$(date +%Y%m%d%H%M%S)" >>"$GITHUB_ENV"

- uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.ccache
key: cross-${{ matrix.arch }}-${{ matrix.python }}
key: cross-${{ matrix.arch }}-${{ matrix.python }}-${{ env.TIMESTAMP }}
restore-keys: cross-${{ matrix.arch }}-${{ matrix.python }}-

- name: Pull the Docker image
run: ./in-docker
Expand Down
11 changes: 8 additions & 3 deletions .github/workflows/ci.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,16 @@ name: Continuous integration
with:
submodules: "true"

- name: Generate a timestamp
run: echo "TIMESTAMP=$(date +%Y%m%d%H%M%S)" >>"$GITHUB_ENV"

- uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.ccache
key: {{ cache_key }}
key: {{ cache_key }}-{% raw %}${{ env.TIMESTAMP }}{% endraw %}
restore-keys: {{ cache_key }}-
{%- endmacro %}

{%- macro steps_post(artifact_name) %}
Expand Down Expand Up @@ -73,11 +77,12 @@ jobs:
run: PATH=/usr/lib/ccache:$PATH ./ci
--skip-repair
-DVALGRIND_CONFIGURE_FLAGS=--disable-dependency-tracking
--sanitize

- name: Smoke test
run: python3 -m pip install --user dist/wheelhouse/*.whl &&
memtrace record true &&
memtrace report >/dev/null
./python3-asan "$(command -v memtrace)" record true &&
./python3-asan "$(command -v memtrace)" report | tail

- name: Show and reset ccache stats
run: ccache --show-stats --verbose --zero-stats
Expand Down
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ target_compile_options(
-Wconversion
-Wdate-time
-Wformat-security
-Wno-maybe-uninitialized
-pedantic
-Werror
-fstack-protector-strong
Expand Down
13 changes: 12 additions & 1 deletion ci
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ def main():
"--wheel-dir",
default=os.path.join(basedir, "dist", "wheelhouse"),
)
parser.add_argument("--sanitize", action="store_true")
args, build_args = parser.parse_known_args()
if args.host != platform.machine():
build_args.extend(
Expand All @@ -150,6 +151,8 @@ def main():
f"-DCMAKE_SYSTEM_PROCESSOR={args.host}",
)
)
if args.sanitize:
build_args.append("-DCMAKE_CXX_FLAGS=-fsanitize=address,undefined")
build = os.path.join(basedir, "build", f"{platform.system()}-{args.host}")
build_prefix = "" if args.host_python is None else "build-"
cross_prefix = "" if args.host_python is None else "cross-"
Expand All @@ -166,7 +169,15 @@ def main():
*build_args,
],
)
run_in_venv(venv, ["python", "-m", "unittest", "discover"])
run_in_venv(
venv,
[
"./python3-asan" if args.sanitize else "python3",
"-m",
"unittest",
"discover",
],
)
build_wheel(
venv,
args.build_type,
Expand Down
52 changes: 52 additions & 0 deletions python3-asan
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
from argparse import ArgumentParser
import os
import subprocess


def main():
parser = ArgumentParser()
parser.add_argument("-m")
args, rest = parser.parse_known_args()

script = """\
from os import environ
del environ["LD_PRELOAD"]
"""
if args.m:
script += f"""\
import {args.m}.__main__
"""
else:
script += """\
import sys
assert sys.argv[0] == "-c"
del sys.argv[0]
with open(sys.argv[0]) as fp:
exec(compile(fp.read(), sys.argv[0], "exec"))
"""

cc = os.environ.get("CC", "cc")
# https://github.com/google/sanitizers/issues/934
libstdcxx = (
subprocess.check_output([cc, "-print-file-name=libstdc++.so"]).strip().decode()
)
libasan = (
subprocess.check_output([cc, "-print-file-name=libasan.so"]).strip().decode()
)

# https://stackoverflow.com/questions/77894856
argv = ["setarch", "--addr-no-randomize", "python3", "-c", script, *rest]
os.execvpe(
argv[0],
argv,
{
**os.environ,
"ASAN_OPTIONS": "detect_leaks=0",
"LD_PRELOAD": os.pathsep.join((libasan, libstdcxx)),
},
)


if __name__ == "__main__":
main()
4 changes: 0 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,6 @@
entry_points={
"console_scripts": [
"memtrace=memtrace.cli:main",
"memtrace-analyze=memtrace.analysis:main",
"memtrace-index=memtrace.index:main",
"memtrace-stats=memtrace.stats:main",
"memtrace-ud=memtrace.ud:main",
],
},
)