Add recipe for sqlite3mc #21354

utelle · 2023-11-24T21:15:39Z

Specify library name and version: sqlite3mc/1.8.0

I'm the author of the project SQLite3 Multiple Ciphers - sqlite3mc in short. The project adds an encryption extension to the SQLite library allowing to transparently encrypt the database files. As the name suggests the extension supports several common cipher schemes (used by other Open Source projects):

wxSQLite3: AES 128 Bit CBC - No HMAC
wxSQLite3: AES 256 Bit CBC - No HMAC
sqleet: ChaCha20 - Poly1305 HMAC
SQLCipher: AES 256 Bit CBC - SHA1/SHA256/SHA512 HMAC
System.Data.SQLite: RC4
Ascon: Ascon-128 v1.2 (Lightweight Authenticated Encryption)

The actual cipher scheme used for database encryption can be chosen at runtime.

I was asked by a user of my project, @Myroendan, whether a Conan recipe could be provided. So, here it is.

I've read the contributing guidelines.
I've used a recent Conan client version close to the currently deployed.
I've tried at least one configuration locally with the conan-center hook activated.

CLAassistant · 2023-11-24T21:15:45Z

All committers have signed the CLA.

Remove C++ compiler options, because this library is a C library

SpaceIm · 2023-11-26T08:13:01Z

recipes/sqlite3mc/all/conandata.yml

+  "1.8.0":
+    url: "https://github.com/utelle/SQLite3MultipleCiphers/archive/refs/tags/v1.8.0.tar.gz"
+    sha256: "13D9B939BEF7C7371D58A3874F83B18CF330EB2171205B3680ACDDB2215BE0E5"
+    strip_root: true


Suggested change

strip_root: true

SpaceIm · 2023-11-26T08:13:14Z

recipes/sqlite3mc/all/conanfile.py

+        self.settings.rm_safe("compiler.libcxx")
+
+    def source(self):
+        get(self, **self.conan_data["sources"][self.version])


Suggested change

get(self, **self.conan_data["sources"][self.version])

get(self, **self.conan_data["sources"][self.version], strip_root=True)

recipes/sqlite3mc/all/conanfile.py

recipes/sqlite3mc/all/test_package/CMakeLists.txt

recipes/sqlite3mc/all/test_package/conanfile.py

recipes/sqlite3mc/all/conanfile.py

AbrilRBS

Hi @utelle thanks a lot for taking the time to contribute the recipe for your library, we're always happy to get author's involvment with their recipes as that usually makes the resulting packages much better

We'll now take a look at the recipe and try to build some permutation of some of the options you've added for the library, thanks a lot for your patience :)

Co-authored-by: SpaceIm <[email protected]>

AbrilRBS · 2023-12-12T11:15:35Z

Builds look good on my side, will properly review once the CI finishes this run, thanks a lot for your patience! :)

utelle · 2024-02-17T17:53:57Z

Over 2 months have passed ... and I see no progress at all. Is there anything I can or should do from my side?

uilianries · 2024-02-19T16:27:19Z

recipes/sqlite3mc/all/conanfile.py

+            self.cpp_info.libs = ["sqlite3mc"]
+        else:
+            self.cpp_info.libs = ["sqlite3mc_static"]
+        if self.settings.os in ("Linux", "Macos"):


Suggested change

if self.settings.os in ("Linux", "Macos"):

if self.settings.os in ("Linux", "FreeBSD"):

Could you please explain, why "Macos" should be replaced by "FreeBSD"? IMHO this can't be correct, because

the libraries pthread, dl, and m are required for Macos, too, and

in line 224 self.settings.os is explicitly checked for "Macos" again, because an additional attribute is required under Macos.

I could understand, if you would want the following change:

if self.settings.os in ("Linux", "FreeBSD", "Macos"):

I see that these are indeed linked against by the project on macOS: https://github.com/utelle/SQLite3MultipleCiphers/blob/7e8ecba9a82ea1336b118a21ccb733f4ed292efa/CMakeLists.txt#L262-L266

Excuse the dumb question (I'm not that familiar with the platform): do m and dl even exist on macOS? It's the first time I'm seeing these being linked against outside of Linux/FreeBSD. As far as I know, these are only provided by glibc and similar libc implementations, which is not available on macOS.

I see that these are indeed linked against by the project on macOS: https://github.com/utelle/SQLite3MultipleCiphers/blob/7e8ecba9a82ea1336b118a21ccb733f4ed292efa/CMakeLists.txt#L262-L266

Excuse the dumb question (I'm not that familiar with the platform): do m and dl even exist on macOS? It's the first time I'm seeing these being linked against outside of Linux/FreeBSD. As far as I know, these are only provided by glibc and similar libc implementations, which is not available on macOS.

Actually, I'm not very familiar with macOS either. However, the GitHub CI action (for example this run) runs without errors on macOS, and up to now I haven't received any complaints from users of my component.

uilianries · 2024-02-19T16:29:50Z

recipes/sqlite3mc/all/conanfile.py

+        tc.variables["SQLITE_ENABLE_CSV"] = self.options.enable_csv
+        tc.variables["SQLITE_ENABLE_EXTFUNC"] = self.options.enable_extfunc
+        tc.variables["SQLITE_ENABLE_GEOPOLY"] = self.options.enable_geopoly
+        tc.variables["SQLITE_ENABLE_JSON1"] = self.options.enable_json1


I tried to find an usage for this option, but I failed to find it. I see it in the CMakeLists, but I don't see it in the code, I mean, a required definition like ifdef:

https://github.com/search?q=repo%3Autelle%2FSQLite3MultipleCiphers%20SQLITE_ENABLE_JSON1&type=code

Could you please explain a bit more how does it work, same to other options?

I tried to find an usage for this option, but I failed to find it. I see it in the CMakeLists, but I don't see it in the code, I mean, a required definition like ifdef:

https://github.com/search?q=repo%3Autelle%2FSQLite3MultipleCiphers%20SQLITE_ENABLE_JSON1&type=code

Could you please explain a bit more how does it work, same to other options?

Several SQLite extensions can be included optionally at compile time using #ifdef statements. Mostly this happens in the main C file sqlite3mc.c (like for SQLITE_ENABLE_CSV and SQLITE_ENABLE_EXTFUNC).

But there are also extension that are already included in the original (resp patched) SQLite source amalgamation C file sqlite3patched.c (like for SQLITE_ENABLE_GEOPOLY).

The symbol SQLITE_ENABLE_JSON1 is different. This symbol does indeed not exist any more. In prior SQLite versions it was available, but in recent versions the JSON extension is included per default, but it could be explicitly excluded by specifying the symbol SQLITE_OMIT_JSON.

For the original project SQLite3 Multiple Ciphers I decided to expose many common SQLite configuration options to give users a high degree of freedom to choose the options and extensions most appropriate for their project. Per default many extensions are enabled, but in some projects there exist memory restrictions, so that users want to be able to disable unneeded extensions.

AbrilRBS

Hi @utelle thanks a lot for your contribution, and sorry about the delay, we appreciate your patience a lot, some PRs get longer to get reviewed than other, sorry about that :)

AbrilRBS · 2024-02-19T16:35:02Z

recipes/sqlite3mc/all/test_package/main.c

+
+int main(int argc, char *argv[])
+{
+    sqlite3 *db;


Hi! Could we get a simplified version of this? It's not in CCI's policy to have extensive tests in the test_package, it should be as minimal as possible while testing the correct package of the library :)

Hi! Could we get a simplified version of this? It's not in CCI's policy to have extensive tests in the test_package, it should be as minimal as possible while testing the correct package of the library :)

Well, this is the most simple test I could think of. To verify that the package was build correctly it is necessary to create an encrypted database with some content and to access it with the right passphrase and a wrong passphrase afterwards.

If that test is too complex, we don't need any test at all.

I removed the configuration option SQLITE_ENABLE_JSON1, because this option is no longer used by SQLite. And I added FreeBSD to the os list (instead of replacing Macos by FreeBSD), because this seems to be the only logical modification.

Regarding the "extensive" test I have no idea what you expect. What could and/or should be tested, if not the proper build of the package?

Hi,
we could remove the error handlings from the test, use logical OR on the result during operations, and check it once in the end. That would make it 35-40 lines shorter. Would that work?

we could remove the error handlings from the test, use logical OR on the result during operations, and check it once in the end. That would make it 35-40 lines shorter. Would that work?

It is certainly possible to reduce the number of lines of code by improving the error handling code (for example, by introducing a small function that checks the return code and emits an error message if necessary), but I'm not sure if that is what @RubenRBS intends.

conan-center-bot · 2024-02-19T20:16:44Z

Conan v1 pipeline ✔️

All green in build 10 (c2fd07ed490785450ed7e1b62bc99f2d090a9a7c):

sqlite3mc/1.8.0:
All packages built successfully! (All logs)

Conan v2 pipeline ✔️

Note: Conan v2 builds are now mandatory. Please read our discussion about it.

All green in build 10 (c2fd07ed490785450ed7e1b62bc99f2d090a9a7c):

sqlite3mc/1.8.0:
All packages built successfully! (All logs)

utelle · 2024-03-21T16:01:35Z

@RubenRBS

Hi @utelle thanks a lot for your contribution, and sorry about the delay, we appreciate your patience a lot, some PRs get longer to get reviewed than other, sorry about that :)

Another month went by ... and my latest modifications and comments didn't get any helpful response. I have to admit that I gradually loose interest in this endeavor.

AbrilRBS · 2024-03-21T18:04:47Z

Sorry about this, I know that feeling ignored after taking the time to contribute a PR with all the effort it requires can be discouraging - nothing further from the truth, you contribution is truly appreciated.

Sometimes some PRs fall thru the cracks due to the sheer volume of PRs that CCI recives - we're working on systems to be able to better handle that load, so that things like this don't happen again, but that'd still take some time

I've approved to unblock this and will ask the team to send a follow-up pr if necessary to get the final touches if any are needed, but again, sorry about the delay :)

utelle · 2024-03-21T21:22:22Z

I know that feeling ignored after taking the time to contribute a PR with all the effort it requires can be discouraging

Well, I think I really showed a lot of patience while the PR was queued for processing, but once review(s) began I had expected a somewhat more continuous process. Working on a PR with long breaks inbetween requires more time and energy on both sides to pick up the loose ends where they were left off.

Sometimes some PRs fall thru the cracks due to the sheer volume of PRs that CCI recives - we're working on systems to be able to better handle that load, so that things like this don't happen again, but that'd still take some time

I fully understand that it is difficult to handle such a high volume of PRs, especially as the work is mostly done by volunteers in their sparetime. And of course it is important to carefully check the quality of new recipes to give Conan users a pleasant experience.

I've approved to unblock this and will ask the team to send a follow-up pr if necessary to get the final touches if any are needed, but again, sorry about the delay :)

Thanks.

In this context I have a question: how should updates of the recipe be handled? Actually, the current version of SQLite3 Multiple Ciphers is already 1.8.4. Which versions should be added to the Conan recipe? All intermediate versions? Or just the latest? Should old versions be dropped from the recipe? Or kept forever?

* Add recipe for sqlite3mc * Adjust recipe Remove C++ compiler options, because this library is a C library * Add CMake version requirement * Add CMake requirement to test package * Add missing header in test * Adjust according to review * Fix conan v1 related issues * Apply suggestions from code review Co-authored-by: SpaceIm <[email protected]> * Remove option SQLITE_ENABLE_JSON1, add FreeBSD to os list --------- Co-authored-by: Rubén Rincón Blanco <[email protected]> Co-authored-by: SpaceIm <[email protected]>

Add recipe for sqlite3mc

ce76295

conan-center-bot added the Failed label Nov 24, 2023