Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve unit testing (Part 2) #3385

Draft
wants to merge 15 commits into
base: develop
Choose a base branch
from

Conversation

falbrechtskirchinger
Copy link
Contributor

@falbrechtskirchinger falbrechtskirchinger commented Mar 9, 2022

This is part 2 of the unit test improvements.

The original plan of adding build information output has been scrapped after the code had ballooned into a 4000 line monstrosity.

Instead, this PR addresses some issues that were discussed (remove magic keywords from the unit tests) and adds a very minimal version of what the build information output would have provided by extending JSON meta and printing that at the start of each unit test or via the print_meta target.

For example, most CI targets will now print the following for each enabled C++ standard:

print_meta_cpp11
{
    "version": "3.10.5",
    "platform": "linux",
    "compiler": {
        "family": "clang",
        "version": "15.0.0 ",
        "c++": "201103",
        "libc++": {
            "family": "GNU libstdc++",
            "version": 20210427
        }
    },
    "config": {
        "JSON_DIAGNOSTICS": false,
        "JSON_DISABLE_ENUM_SERIALIZATION": false,
        "JSON_HAS_CPP_11": true,
        "JSON_HAS_CPP_14": false,
        "JSON_HAS_CPP_17": false,
        "JSON_HAS_CPP_20": false,
        "JSON_HAS_CPP_23": false,
        "JSON_HAS_EXPERIMENTAL_FILESYSTEM": false,
        "JSON_HAS_FILESYSTEM": false,
        "JSON_HAS_THREE_WAY_COMPARISON": false,
        "JSON_NOEXCEPTION": false,
        "JSON_NO_IO": false,
        "JSON_USE_IMPLICIT_CONVERSIONS": true,
        "JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON": false,
        "NLOHMANN_JSON_ABI_STRING": "json_v3_10_5"
    }
}

To Do

  • Document meta() changes.
  • Convert boolean macros.

@coveralls
Copy link

coveralls commented Mar 9, 2022

Coverage Status

Coverage remained the same at 100.0% when pulling 35f6a61 on falbrechtskirchinger:topic/test-improvements-part2 into 98d70d4 on nlohmann:develop.

@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch 4 times, most recently from 1a35528 to f4d0546 Compare March 10, 2022 17:31
@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch from f4d0546 to 69a2784 Compare May 9, 2022 18:07
@falbrechtskirchinger falbrechtskirchinger changed the title Improve unit testing (Part 2) Improve unit testing (Part 2) and fuzz testing May 9, 2022
@falbrechtskirchinger
Copy link
Contributor Author

falbrechtskirchinger commented Jul 19, 2022

Note to self: Add ABI string from #3590 to meta().

Edit: Done.

@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch 5 times, most recently from cd146ae to bd05c62 Compare August 1, 2022 11:09
@nlohmann
Copy link
Owner

nlohmann commented Aug 1, 2022

This starts to become a very large PR. Please consider breaking it into smaller pieces, such as the whole fuzzer part.

@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch from bd05c62 to b3a29f5 Compare August 1, 2022 11:52
@falbrechtskirchinger
Copy link
Contributor Author

This starts to become a very large PR. Please consider breaking it into smaller pieces, such as the whole fuzzer part.

Yes. I'm just using this for CI. Locally I already have a separate branch for the fuzzing stuff.

@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch from b3a29f5 to 3e30e1f Compare August 1, 2022 12:18
@falbrechtskirchinger falbrechtskirchinger changed the title Improve unit testing (Part 2) and fuzz testing Improve unit testing (Part 2) Aug 1, 2022
@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch 3 times, most recently from d927dfc to 89211f0 Compare August 2, 2022 10:46
Add and use the following CMake function:

  json_test_add_standard_keyphrases(
      PHRASES <args>...
      CXX_STANDARDS <args>...)

  Create a mapping between C++ standard versions and key phrases.

  json_test_add_test_for() will search for these key phrases and build
  tests for associated C++ standard versions.
Use "logical" ordering for use with ordered_json.
Add a target to print JSON meta for all tested C++ standard versions.
@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch from 8a27a68 to dbf568a Compare August 3, 2022 20:36
@falbrechtskirchinger falbrechtskirchinger force-pushed the topic/test-improvements-part2 branch from dbf568a to 35f6a61 Compare August 3, 2022 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants