Allow linking multiple object files built from Souffle generated code

[ruricolist]

This addresses #2356 using the approach linked by @quentin in his comments there. I've taken the linked commit from his branch and adjusted it to apply to Souffle master, with some adjustments for generating multiple files.

Since the inability to link object files of Souffle programs is relevant to multiple projects using Souffle, I think it is desirable to address the problem directly in the short term.

[codecov]

Codecov Report

Merging #2371 (2c9248c) into master (b311946) will increase coverage by 0.00%.
The diff coverage is 93.57%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2371   +/-   ##
=======================================
  Coverage   77.52%   77.52%           
=======================================
  Files         466      467    +1     
  Lines       30690    30778   +88     
=======================================
+ Hits        23791    23862   +71     
- Misses       6899     6916   +17     

Impacted Files	Coverage Δ
src/synthesiser/GenDb.h	93.75% <78.26%> (-6.25%)	⬇️
src/MainDriver.cpp	70.23% <92.81%> (ø)
src/souffle.cpp	100.00% <100.00%> (ø)
src/synthesiser/GenDb.cpp	97.29% <100.00%> (+0.19%)	⬆️
src/synthesiser/Synthesiser.cpp	84.84% <100.00%> (+0.05%)	⬆️
src/include/souffle/utility/ParallelUtil.h	83.72% <0.00%> (-1.56%)	⬇️
src/reports/DebugReport.cpp	20.19% <0.00%> (-0.97%)	⬇️
src/ram/analysis/Index.cpp	89.71% <0.00%> (-0.59%)	⬇️
src/include/souffle/datastructure/BTree.h	97.91% <0.00%> (-0.42%)	⬇️
... and 3 more

[quentin]

Hi @ruricolist thanks for integrating this!
By any chance, could you add a dedicated test that demonstrates the integration of multiple souffle programs ?

[b-scholz]

Please can you resolve the conflicts? Quentin has recently restructured the driver so that Souffle works better as a library.

[ruricolist]

I've rebased and added two tests (one to show how linking is broken without namespaces, one to show that namespaces fix the problem).

What's being tested here doesn't fit any of the predefined CMake test patterns, so most of the actual test is in Python.

[quentin]

Looks good to me, thanks for this port of my original change.

[SamedBayer]

Hi, when using Souffle version 2.4, I encounter a problem that did not exist with version 2.2. The issue arises when I attempt to generate and link multiple Souffle files. It gives the same error with #2356.

The specific process of generating *.cpp files I follow is:

souffle -g A.cpp A.dl
souffle -g B.cpp B.dl
souffle -g C.cpp C.dl  

Where A.dl is included in both B.dl and C.dl. This procedure was functioning correctly in version 2.2, but it leads to errors in the most recent version.

I am looking for assistance on how to use namespaces in my Souffle programs to resolve this issue. Specifically, can you please provide an example of a solution that employs namespaces to fix this problem?

[quentin]

Hi,
Are you trying to produce a single souffle program with sources A, B, and C ?
Or are you trying to produce 2 souffle programs: (A + B) and (A + C) ?

In the first case, you need to create a top level datalog file that includes A, B and C, and pass only that top file to souffle -g.

In the second case, you need to generate code with different namespaces for B and C:

• souffle -g B.cpp --NS B B.dl will generate code under the namespace B { ... }
• souffle -g C.cpp --NS C C.dl will generate code under the namespace C { ... }
• there is no need to call souffle for A since it is already included by B and C.

[SamedBayer]

Hi again,

It was the second case and it worked with different namespaces.
Thanks.