Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dnsdist: Add meson support #14724

Merged
merged 23 commits into from
Feb 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
9176d81
dnsdist: Add meson support
rgacogne Sep 23, 2024
758c61a
dnsdist: Fix a typo in the CDB m4 file
rgacogne Sep 25, 2024
245fee6
dnsdist: Fix a few "unused parameter" warnings
rgacogne Sep 25, 2024
e8e6f46
dnsdist: Fix detection of a specific nghttp2 feature
rgacogne Sep 25, 2024
7579c48
Fix fuzzing targets with meson
rgacogne Sep 27, 2024
cd00d04
dnsdist: Fix building with meson without all options enabled
rgacogne Sep 27, 2024
738e8a5
dnsdist: Apply Otto's patch (thanks!)
rgacogne Sep 27, 2024
c0e1848
dnsdist: Remove duplicated meson directories and files
rgacogne Oct 14, 2024
c2f2f83
dnsdist: Fix a lot of "unused parameter" warnings
rgacogne Oct 14, 2024
a55ac2b
dnsdist: Re-enable "unused parameter" warnings
rgacogne Oct 14, 2024
73689fb
dnsdist: Add meson-based builds to the CI
rgacogne Oct 14, 2024
2405ae7
build-and-test-all: Install meson for dnsdist
rgacogne Oct 14, 2024
56add5c
tasks: Properly convert sanitizers to meson
rgacogne Oct 15, 2024
cc1ec80
dnsdist: Fix meson build without LMDB
rgacogne Oct 15, 2024
a95b899
dnsdist: Fix duplicated dnslabeltext.cc
rgacogne Oct 15, 2024
41cd3ec
dnsdist: Fix more "unused parameters" warnings
rgacogne Oct 17, 2024
b29e68a
dnsdist: Fix building without dnstap support enabled
rgacogne Oct 17, 2024
c34e19f
dnsdist: Correctly set env vars up for meson
rgacogne Jan 20, 2025
19298bc
dnsdist: Fix additional LDFLAGS in tasks.py
rgacogne Jan 20, 2025
b155625
dnsdist: Fix remaining build issues for YAML with meson
rgacogne Jan 24, 2025
9360407
dnsdist: Fix compilation errors w/o DoQ and DoH3, plus clang-tidy war…
rgacogne Jan 28, 2025
fd1c0a9
dnsdist: Fix formatting and a clang-tidy warning
rgacogne Feb 3, 2025
cced151
dnsdist: Fix unused parameter warnings
rgacogne Feb 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 28 additions & 17 deletions .github/workflows/build-and-test-all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ jobs:
needs: get-runner-container-image
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
sanitizers: [asan+ubsan, tsan]
features: [least, full]
builder: [autotools, meson]
exclude:
Expand Down Expand Up @@ -216,11 +216,14 @@ jobs:
needs: get-runner-container-image
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
builder: [autotools, meson]
sanitizers: [asan+ubsan, tsan]
features: [least, full]
exclude:
- sanitizers: tsan
features: least
- sanitizers: tsan
builder: meson
container:
image: "${{ needs.get-runner-container-image.outputs.id }}:${{ needs.get-runner-container-image.outputs.tag }}"
env:
Expand Down Expand Up @@ -250,27 +253,33 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.ccache
key: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-ccache-${{ steps.get-stamp.outputs.stamp }}
restore-keys: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-ccache-
key: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-${{ matrix.builder}}-ccache-${{ steps.get-stamp.outputs.stamp }}
restore-keys: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-${{ matrix.builder}}-ccache-
- run: inv install-lld-linker-if-needed
working-directory: ./pdns/dnsdistdist/
- run: inv ci-install-meson
if: ${{ matrix.builder == 'meson' }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-install-rust ${{ env.REPO_HOME }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-build-and-install-quiche ${{ env.REPO_HOME }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-autoconf
if: ${{ matrix.builder == 'autotools' }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-dnsdist-configure ${{ matrix.features }}
- run: inv ci-dnsdist-configure ${{ matrix.features }} ${{ matrix.builder }} dnsdist-${{ env.BUILDER_VERSION }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-make-distdir
if: ${{ matrix.builder == 'autotools' }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-dnsdist-configure ${{ matrix.features }}
- run: inv ci-dnsdist-make-bear
- run: inv ci-dnsdist-run-unit-tests
- run: inv ci-dnsdist-configure ${{ matrix.features }} ${{ matrix.builder }} dnsdist-${{ env.BUILDER_VERSION }}
if: ${{ matrix.builder == 'autotools' }}
- run: inv ci-dnsdist-make-bear ${{ matrix.builder }}
- run: inv ci-dnsdist-run-unit-tests ${{ matrix.builder }}
- run: inv generate-coverage-info ./testrunner $GITHUB_WORKSPACE
if: ${{ env.COVERAGE == 'yes' && matrix.sanitizers != 'tsan' }}
if: ${{ env.COVERAGE == 'yes' && matrix.sanitizers != 'tsan' && matrix.builder == 'autotools'}}
- name: Coveralls Parallel dnsdist unit
if: ${{ env.COVERAGE == 'yes' && matrix.sanitizers != 'tsan' }}
if: ${{ env.COVERAGE == 'yes' && matrix.sanitizers != 'tsan' && matrix.builder == 'autotools' }}
uses: coverallsapp/github-action@v2
with:
flag-name: dnsdist-unit-${{ matrix.features }}-${{ matrix.sanitizers }}
Expand All @@ -279,12 +288,14 @@ jobs:
allow-empty: true
fail-on-error: false
- run: inv ci-make-install
if: ${{ matrix.builder == 'autotools' }}
- run: ccache -s
- run: echo "normalized-branch-name=${{ inputs.branch-name || github.ref_name }}" | tr "/" "-" >> "$GITHUB_ENV"
- name: Store the binaries
if: ${{ matrix.builder == 'autotools' }}
uses: actions/upload-artifact@v4 # this takes 30 seconds, maybe we want to tar
with:
name: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-${{ env.normalized-branch-name }}
name: dnsdist-${{ matrix.features }}-${{ matrix.sanitizers }}-${{ matrix.builder}}-${{ env.normalized-branch-name }}
path: /opt/dnsdist
retention-days: 1

Expand Down Expand Up @@ -526,7 +537,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
sanitizers: [asan+ubsan, tsan]
dist_name: [debian]
pdns_repo_version: ['48']
container:
Expand Down Expand Up @@ -572,7 +583,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
sanitizers: [asan+ubsan, tsan]
dist_name: [debian]
pdns_repo_version: ['48']
container:
Expand Down Expand Up @@ -621,7 +632,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
sanitizers: [asan+ubsan, tsan]
threads: [1, 2, 3, 4, 8]
mthreads: [2048]
shards: [1, 2, 1024]
Expand Down Expand Up @@ -668,7 +679,7 @@ jobs:
runs-on: ubicloud-standard-8-ubuntu-2404
strategy:
matrix:
sanitizers: [ubsan+asan] # TSAN disabled for now
sanitizers: [asan+ubsan] # TSAN disabled for now
threads: [8]
mthreads: [2048]
shards: [1024]
Expand Down Expand Up @@ -712,7 +723,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
sanitizers: [ubsan+asan, tsan]
sanitizers: [asan+ubsan, tsan]
container:
image: "${{ needs.get-runner-container-image.outputs.id }}:${{ needs.get-runner-container-image.outputs.tag }}"
env:
Expand All @@ -736,7 +747,7 @@ jobs:
- name: Fetch the binaries
uses: actions/download-artifact@v4
with:
name: dnsdist-full-${{ matrix.sanitizers }}-${{ env.normalized-branch-name }}
name: dnsdist-full-${{ matrix.sanitizers }}-autotools-${{ env.normalized-branch-name }}
path: /opt/dnsdist
- run: inv install-clang-runtime
- run: inv install-dnsdist-test-deps $([ "$(. /etc/os-release && echo $VERSION_CODENAME)" = "bullseye" ] && echo "--skipXDP=True")
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,12 +169,12 @@ jobs:
if: matrix.product == 'dnsdist'
working-directory: ./pdns/dnsdistdist/
run: |
inv ci-dnsdist-configure full
inv ci-dnsdist-configure full autotools build-dir
- name: Build dnsdist
if: matrix.product == 'dnsdist'
working-directory: ./pdns/dnsdistdist/
run: |
inv ci-dnsdist-make-bear
inv ci-dnsdist-make-bear autotools
- run: ln -s ../../.clang-tidy.full .clang-tidy
if: matrix.product == 'dnsdist'
working-directory: ./pdns/dnsdistdist/
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/coverity.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
working-directory: ./pdns/dnsdistdist/
- run: inv ci-build-and-install-quiche ${{ env.REPO_HOME }}
working-directory: ./pdns/dnsdistdist/
- run: inv ci-dnsdist-configure full
- run: inv ci-dnsdist-configure full autotools build-dir
working-directory: ./pdns/dnsdistdist/
- run: inv coverity-make
working-directory: ./pdns/dnsdistdist/
Expand Down
3 changes: 3 additions & 0 deletions ext/lmdb-safe/lmdb-safe.hh
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,9 @@ private:
// * so let's go back
}
#else /* ifndef DNSDIST */
(void)key;
(void)data;
(void)op;
return rc;
#endif
}
Expand Down
24 changes: 18 additions & 6 deletions fuzzing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,15 @@ By default the targets are linked against a standalone target,
`standalone_fuzz_target_runner.cc`, which does no fuzzing but makes it easy
to check a given test file, or just that the fuzzing targets can be built properly.

This behaviour can be changed via the `LIB_FUZZING_ENGINE` variable, for example
by setting it to `-lFuzzer`, building with clang by setting `CC=clang CXX=clang++`
before running the `configure` and adding `-fsanitize=fuzzer-no-link` to `CFLAGS`
and `CXXFLAGS`. Doing so instructs the compiler to instrument the code for
efficient fuzzing but not to link directly with `-lFuzzer`, which would make
the compilation tests done during the configure phase fail.
This behaviour can be changed via:
- either the `LIB_FUZZING_ENGINE` variable when building with `./configure`
- or the `-Dfuzzer_ldflags` option when building with `meson`

For example, setting `LIB_FUZZING_ENGINE` to `-lFuzzer`, then building with clang
by setting `CC=clang CXX=clang++` before running the `configure`, and adding
`-fsanitize=fuzzer-no-link` to `CFLAGS` and `CXXFLAGS`, instructs the compiler
to instrument the code for efficient fuzzing but not to link directly with
`-lFuzzer`, which would make the compilation tests done during the configure phase fail.

Sanitizers
----------
Expand Down Expand Up @@ -86,6 +89,15 @@ LIB_FUZZING_ENGINE="/usr/lib/clang/11.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a"
make -C pdns -j2 fuzz_targets
```

or, if you are using `meson` to build the authoritative server instead of `./configure`:

```
env CC=clang CXX=clang++ \
CFLAGS=-fsanitize=fuzzer-no-link CXXFLAGS=-fsanitize=fuzzer-no-link \
meson setup .. -Dfuzz-targets=true -Dfuzzer_ldflags=/usr/lib/clang/18/lib/linux/libclang_rt.fuzzer-x86_64.a -Db_sanitize=address,undefined
ninja
```

Now you're ready to run one of the fuzzing targets.
First, copy the starting corpus:

Expand Down
19 changes: 14 additions & 5 deletions meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -413,10 +413,6 @@ conditional_sources = {
'sources': [src_dir / 'lua-record.cc'],
'condition': dep_lua_records.found(),
},
'standalone-fuzz-target-runner': {
'sources': [src_dir / 'standalone_fuzz_target_runner.cc'],
'condition': get_option('fuzz-targets'),
},
}

foreach name, info: conditional_sources
Expand Down Expand Up @@ -928,6 +924,15 @@ if get_option('unit-tests')
endif

if get_option('fuzz-targets')
fuzz_extra_sources = []
fuzzer_ldflags = []
# https://github.com/harfbuzz/harfbuzz/pull/2549/files
if get_option('fuzzer_ldflags') == ''
fuzz_extra_sources += src_dir / 'standalone_fuzz_target_runner.cc'
else
fuzzer_ldflags += get_option('fuzzer_ldflags')
endif

fuzz_targets = [
'moadnsparser',
'packetcache',
Expand All @@ -940,7 +945,9 @@ if get_option('fuzz-targets')
foreach target: fuzz_targets
source_file = src_dir / 'fuzz_' + target.underscorify() + '.cc'
tools += {
'fuzz-target-' + target: { 'main': source_file }
'fuzz-target-' + target: { 'main': source_file,
'link-args': fuzzer_ldflags,
'files-extra': fuzz_extra_sources }
}
endforeach
endif
Expand Down Expand Up @@ -968,6 +975,7 @@ foreach tool, info: tools
export_dynamic = 'export-dynamic' in info ? info['export-dynamic'] : false
files_extra = 'files-extra' in info ? info['files-extra'] : []
deps_extra = 'deps-extra' in info ? info['deps-extra'] : []
link_args = 'link-args' in info ? info['link-args'] : []

set_variable(
var_name,
Expand All @@ -977,6 +985,7 @@ foreach tool, info: tools
config_h,
files_extra,
export_dynamic: export_dynamic,
link_args: link_args,
dependencies: [
deps,
libpdns_common,
Expand Down
1 change: 1 addition & 0 deletions meson_options.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ option('unit-tests', type: 'boolean', value: false, description: 'Build and run
option('unit-tests-backends', type: 'boolean', value: false, description: 'Build and run backend unit tests')
option('reproducible', type: 'boolean', value: false, description: 'Reproducible builds (for distro maintainers, makes debugging difficult)')
option('fuzz-targets', type: 'boolean', value: false, description: 'Enable fuzzing targets')
option('fuzzer_ldflags', type: 'string', value: '', description: 'Linker flags used for the fuzzing targets (a path to the libFuzzer static library, for example)')
option('verbose-logging', type: 'boolean', value: false, description: 'Enable verbose logging')
option('experimental-pkcs11', type: 'feature', value: 'disabled', description: 'PKCS11 support')
option('experimental-gss-tsig', type: 'feature', value: 'disabled', description: 'GSS-TSIG support')
Expand Down
1 change: 1 addition & 0 deletions pdns/dnsdistdist/NOTICE
1 change: 0 additions & 1 deletion pdns/dnsdistdist/base32.hh

This file was deleted.

4 changes: 4 additions & 0 deletions pdns/dnsdistdist/bpf-filter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -900,6 +900,9 @@ uint64_t BPFFilter::getHits(const ComboAddress& requestor)

BPFFilter::BPFFilter(std::unordered_map<std::string, MapConfiguration>& configs, BPFFilter::MapFormat format, bool external)
{
(void)configs;
(void)format;
(void)external;
}

void BPFFilter::addSocket(int)
Expand Down Expand Up @@ -972,6 +975,7 @@ bool BPFFilter::supportsMatchAction(MatchAction action) const
}
return d_mapFormat == BPFFilter::MapFormat::WithActions;
#endif /* HAVE_EBPF */
(void)action;
return false;
}

Expand Down
4 changes: 2 additions & 2 deletions pdns/dnsdistdist/configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ m4_pattern_forbid([^_?PKG_[A-Z_]+$], [*** pkg.m4 missing, please install pkg-con
LT_PREREQ([2.2.2])
LT_INIT([disable-static])

CFLAGS="-g -O3 -Wall -Wextra -Wshadow -Wno-unused-parameter -fvisibility=hidden $CFLAGS"
CXXFLAGS="-g -O3 -Wall -Wextra -Wshadow -Wno-unused-parameter -Wmissing-declarations -Wredundant-decls -fvisibility=hidden $CXXFLAGS"
CFLAGS="-g -O3 -Wall -Wextra -Wshadow -fvisibility=hidden $CFLAGS"
CXXFLAGS="-g -O3 -Wall -Wextra -Wshadow -Wmissing-declarations -Wredundant-decls -fvisibility=hidden $CXXFLAGS"

AC_SUBST([pdns_configure_args], ["$ac_configure_args"])
AC_DEFINE_UNQUOTED([DNSDIST_CONFIG_ARGS],
Expand Down
Loading
Loading