Extend/Update (public) APIs with std::span overloads

[reneme]

Alongside existing std::vector<T, Alloc> and C-style ptr-length APIs we're gradually introducing overloads taking a std::span from C++20. This ticket is an attempt to create/maintain a list of interfaces that should get such overloads.

Public APIs and Interfaces

• RandomNumberGenerator (std::span (and more) for Botan::RandomNumberGenerator #3195)
• Buffered_Computation (Chore: Buffered_Computation can deal with generic containers #3294, std::span for Buffered_Computation #3681)
• Symmetric_Algorithm (Chore: Symmetric_Algorithm can deal with generic containers #3297)
• Encoding
  • std::span/std::string_view for hex_encode/decode #3349
  • std::span/std::string_view for base64 codec #3350
  • base32 (std::span, std::string_view for base32, base58 #3399)
  • base58 (std::span, std::string_view for base32, base58 #3399)
  • DER/BER
• TLS (Client, Callbacks, ...) (Public TLS API uses std::span #3363)
• Key Derivation Function (std::span for Key Derivation Function #3398)
• Message Authentication Code (std::span for MessageAuthenticationCode #3397)
• Cipher Mode
  • Cipher_Mode (std::span for Botan::Cipher_Mode #3392)
  • StreamCipher (for set_iv(), write_keystream()) (std::span for Stream_Cipher #3395)
  • AEAD_Mode (Chore: AEAD_Mode can deal with generic containers #3317)
  • BlockCipher (std::span for Botan::BlockCipher #3394)
• pubkey.h (and friends) (std::span for Public Key Operations #3400)
  • c'tors of Public Key types
• BigInt (eg. c'tors, ::encode, ::binary_encode, ...)
  • Public Interface ([std::span] BigInt - use std::span<> in some internal implementations #3855)
  • Internal implementations (big_rand.cpp, big_ops*.cpp, big_code.cpp)
• Compression_Algorithm/Decompression_Algorithm
• ec_group.h
• HashFunction (std::span for Buffered_Computation #3681)
• mem_ops.h (Range-based mem_ops #3715)

Internal APIs and Interfaces

• EMSA
• Buffered_Computation (std::span for Buffered_Computation #3681)
• SymmetricAlgorithm (std::span for SymmetricAlgorithm::key_schedule() #3684)
• Cipher_Mode
  • TLS_CBC_HMAC_AEAD_Mode
• BlockCipher (encrypt_n(), decrypt_n()) ([std::span] Pass std::span buffers into Block_Cipher Implementations #3870)
  • Those methods are public API and they are virtual methods! Hence, we cannot easily override them without breaking API changes.
  • Also reconsider this patch: Chore: Modernize copy_out_be/le #3885 (comment)
  • Tweakable_Block_Cipher (set_tweak()) ([std::span] Tweakable_Block_Cipher #3872)
• MessageAuthenticationCode (std::span for MessageAuthenticationCode internals #3713)
• MDx_HashFunction (Refactor: Remove MDx_HashFunction #3705)
• StreamCipher (cipher_bytes(), generate_keystream(), set_iv_bytes())
• Header: pk_ops.h
  • Encryption (Use span in PK operations and avoid needless secure_vector #4239)
  • Decryption (Use span in PK operations and avoid needless secure_vector #4239)
  • Verification (Use span in PK operations and avoid needless secure_vector #4239)
  • Signature (Use span in PK operations and avoid needless secure_vector #4239)
  • Key_Agreement (Use span in PK operations and avoid needless secure_vector #4239)
  • KEM_Encryption (API modernization PK_KEM_Encryptor/Decryptor #3611)
  • KEM_Decryption (API modernization PK_KEM_Encryptor/Decryptor #3611)
• SQL_Database (has methods returning std::pair<uint8_t*, size_t>)
• load_be / store_be ([std::span] {load,store}_{le,be} #3707, Refactor: Final iteration on load/store #3869)
• ct_utils.h (Convenience overloads for CT::poison() #4197)

[randombit]

Another one, from #3411

      Kyber_PublicKey(const AlgorithmIdentifier& alg_id,
-                      const std::vector<uint8_t>& key_bits);
+                      std::span<const uint8_t> key_bits);

Would be nice to do this for all of the key types.

(This might require some work on BER_Decoder though ...)

[crazydef]

It would be nice if Botan detected the C++ version in the headers and didn't expose unsupported/new features to C++ compilers that didn't support them.

I am currently trying to integrate Botan into a codebase that can't be built with C++20 (because other libraries we're using don't support it yet) and having to wrap everything is becoming a royal pain in the ass.

[randombit]

@crazydef Don't do that. Use Botan2, which is still supported and probably will be for years, and is C++11.

Or (for some functionality) you can use the C89 API in ffi.h

[lieser]

because other libraries we're using don't support it yet

Do this libraries really break if compiled with C++20? I would expect most older code to work without a problem if compiled with a newer C++ standard.

[crazydef]

@lieser C++17 deprecated a lot of stuff and removed even more. One particular library I have to deal with still only supports C++14 because of the mess that is C++17.

[CharlesJQuarra]

I'm just trying to build from tag 3.2.0, and I'm seeing the #include <span> breaking compilation for C++17. make is building with the following options:

g++ -fPIC -fvisibility=hidden -fstack-protector -m64 -pthread -std=c++17 -D_REENTRANT -O3 -DBOTAN_IS_BEING_BUILT -Wall -Wextra -Wpedantic -Wstrict-aliasing -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual -Wzero-as-null-pointer-constant -Wnon-virtual-dtor -Wold-style-cast -Wsuggest-override -Wshadow -Wextra-semi -I build/include -isystem build/include/external

As I'm sure you are aware, #include <span> is only supported for C++20. Not sure if this is somehow supported as an extension by other compilers but on g++ 8.4.0 it is certainly not.

[randombit]

@CharlesJQuarra Botan 3.0 and higher requires C++20 and minimum GCC at least 11.2

That you're seeing precisely this error is surprising in that configure.py should have detected that the version of GCC is too old, and immediately errored out.

[CharlesJQuarra]

@randombit my bad, I ran make directly, as it worked on previous pulls, once I ran configure.py I saw the updated warnings that C++17 is no longer supported for v3