[Metricbeat] Driver Library requirements for ODBC package in golang.

[kush-elastic]

Hi
We are trying to create IBM DB2 Metricbeat Module. but we are facing issues while building binary.

Here is the detailed technical analysis of go_ibm_db and odbc package.

Following are the steps and issues we are facing for go_ibm_db package:
Using go_ibm_db library with the cli driver before building metricbeat

• In order to use this library, it is required to export a few environment variables mentioned below before building metricbeat. The library uses these variables to locate the cli driver at the build time. Here is the link for reference:

export DB2HOME=/root/ibm/clidriver
export CGO_CFLAGS=-I$DB2HOME/include
export CGO_LDFLAGS=-L$DB2HOME/lib
export LD_LIBRARY_PATH=/root/ibm/clidriver/lib

After setting these env variables manually, we tried building the metricbeat:

• The metricbeat build was successful
• However, when we try to start/run the metric beat binary, it shows an error that it is not able to find cli libraries. After debugging, we determined that the metricbeat requires the environment variables are also set before starting the metricbeat.
• The problem with this is, the user using the metricbeat would have to ensure that the drivers are installed and the env variables are set even if he/she does not want to use the IBM DB2 metricbeat module. Since, when we start the metricbeat after bundling this library, it tries to find this cli driver irrespective of whether the user wants to use this driver or not, this would be a bad experience for the users.
• We also tried bundling the driver along with the metricbeat and setting the env paths to point that bundled driver at run-time, but the metricbeat still throws an error of not being able to find the driver on start-up.

Following are the steps and issues we are facing for odbc package:
Using odbc library with the cli driver before building metricbeat

• In order to use this library, it is required to install unixODBC before building metricbeat. The library uses odbc dll in Microsoft windows and unixODBC everywhere else. Here is the link for reference
• If we don't install unixODBC then it is showing the below error:

github.com/alexbrainman/odbc/api/api_unix.go:14:11: fatal error: 'sql.h' file not found"

After installing unixODBC manually, we tried building the metricbeat:

• The metricbeat build was successful
• However, when we try to start/run the metric beat binary, it shows an error that it is not able to find cli libraries. After debugging, we determined that the metricbeat requires the unixODBC installed before starting the metricbeat.
• The problem with this is, the user using the metricbeat would have to ensure that the unixODBC is installed even if he/she does not want to use the IBM DB2 metricbeat module. Since, when we start the metricbeat after bundling this library, it tries to find lib files provided by unixODBC irrespective of whether the user wants to use this driver or not, this would be a bad experience for the users.

How should we proceed further?

I am attaching the IBM DB2 issue here. and also SQL Module issue as they have faced somewhat similar issues and were also using the same odbc library.

Related #13106
Related #13257

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[mtojek]

ping @jsoriano @andrewkroh @sayden

I'm wondering if we can work out a convenient solution to this problem. Otherwise, we won't be able to integrate with products requiring dynamic libraries.

[ChrsMark]

This reminds me the issues we had at #8870.

[mtojek]

I'm wondering if we can somehow extend the SQL module with IBM DB support.

[jsoriano]

This reminds me the issues we had at #8870.

Yes, the situation seems to be quite similar.

I have tried to build metricbeat with the go_ibm_db driver, following the instructions given in the project, and as Kush mentions, metricbeat builds. Then to run it you need to pass the same environmental variables, otherwise, metricbeat fails to start:

error while loading shared libraries: libdb2.so.1: cannot open shared object file: No such file or directory

I think that to avoid this problem we would need to fork the library, and change how it is initialized. It is BSD-3, so in principle we could do it, but it'd be a bit inconvenient to need to maintain it long term. But this could be something to evaluate.

With the proper environment variables, then the SQL module seems to work. With a config like the following one I get connectivity errors (expected because I don't have db2 running locally :D):

metricbeat.modules:
  - module: sql
    metricsets: [query]
    hosts: ['HOSTNAME=localhost;DATABASE=db;PORT=1234;UID=username;PWD=password']
    driver: go_ibm_db
    sql_query: 'select ...

Error fetching data for metricset sql.query: error opening connection: testing connection: SQLDriverConnect: {08001} [IBM][CLI Driver] SQL30081N  A communication error has been detected....

So to include it in metricbeat I think we should, at least, do:

• Use build tags so not all developers need to have it installed with the proper environment variables.
• Use a forked version of the library with not so "aggressive" initialization (or try to apply these changes in upstream).
• Modify CI jobs to include this library in testing and builds.

Other ideas explored in the document shared by Christos could apply also here.

It could be specially interesting to try again the plugin approach, this is something that could be nice for any other tricky connector, but we would have to think on how to distribute plugins, what opens its own cans of worms.

---

A possibly more future-proof approach is to try to do it in a separate binary that is started by Elastic Agent. Then it could be implemented using any library that is convenient to integrate with DB2. An integration would be added that uses this binary.

I wouldn't know how to start with this, but possibly a good starting point is to check how to add an spec (as these ones) and what is needed for elastic-agent to discover these binaries on run time.

Pinging also @kvch and @blakerouse in case they have other ideas.

[andrewkroh]

In order to create a dependency on a shared object that is not distributed with Beats (it's an optional dependency) we need to use dlopen to dynamically load it at runtime.

If Beats is linked against the shared object (i.e. it shows up when you do ldd <beatname>) then everyone that uses the Beat must have the library present on their system. That's only reasonable if we provide the shared object (or a stub) with our packages.

The journald input in Filebeat (uses libsystemd) and the system/package dataset in Auditbeat (uses librpm) are examples of this.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[mattia-eleuteri]

very relevent in our case 👍

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!