chore(observability): Add OpenTelemetry instrumentation with Honeycom…

…b exporter (#142)

* add LlamaIndex callback handler for OTel
* chore: force logger config in `setup.py` to override the logger handler added by OTel earlier in the lifecycle.
* docs: update dev guide

.gitignore

@@ -188,3 +188,6 @@ docker.env
 # ignore docs dynamically generated at build time.
 /docs/api
 /docs/tests
+
+# ignore Infisical config file
+.infisical.json

docs/developer-guide/getting-started.md

@@ -50,103 +50,115 @@ This Project depends on the following projects.
 
 Some useful examples of how this project can be used:
 
-- Install requirements
+### Install requirements
 
   ```sh
   poe install-dev
   ```
 
-- Run tests
+### Run tests
 
-  - Unit tests
+- Unit tests
 
-  ```sh
-  poe test
-  ``` 
+```sh
+poe test
+``` 
 
-  - Integration tests
-
-    - Prepare env vars by supplying a pytest.env file required for the integration tests
+- Integration tests
+
+ - Prepare env vars by supplying a pytest.env file required for the integration tests
+
+   ```sh
+   cp misc/pytest.env.template pytest.env
+   ## Make edits on pytest.env with the correct values for the test environment then run the tests
+   ```
+
+```sh
+poe test-integration
+```
+
+### Run the project
+
+- Prepare env vars by supplying a Streamlit secrets file
 
-      ```sh
-      cp misc/pytest.env.template pytest.env
-      ## Make edits on pytest.env with the correct values for the test environment then run the tests
-      ```
-
   ```sh
-  poe test-integration
+  cp misc/secrets.toml.template .streamlit/secrets.toml
+  ## Make edits on .streamlit/secrets.toml
+  ## Customise other files in .streamlit/ directory to influence Streamlit behaviour
   ```
 
-- Run the project
+- TIP: Make sure `.streamlit` directory is in the `.gitignore` file so it isn't checked in with secrets.
 
-  - Prepare env vars by supplying a Streamlit secrets file
+- Run the application and it will be available at http://localhost:8501
 
-    ```sh
-    cp misc/secrets.toml.template .streamlit/secrets.toml
-    ## Make edits on .streamlit/secrets.toml
-    ## Customise other files in .streamlit/ directory to influence Streamlit behaviour
-    ```
-
-  - TIP: Make sure `.streamlit` directory is in the `.gitignore` file so it isn't checked in with secrets.
+  ```sh
+  poe run
+  ```
 
-  - Run the application
+- To change the port number
 
-    ```sh
-    poe run
-    ```
+  ```sh
+  poe run --port PORT
+  ```
 
-  and it will be available at http://localhost:8501
+### Run with OpenTelemetry (OTel) Instrumentation
 
-  - To change the port number
+This runs the `poe run` task above with the `opentelemetry-instrument` command which auto instruments several libraries with tracing and sends traces to the configured exporter. Honeycomb env vars see the `/misc/secrets.toml.template`
 
-    ```sh
-    poe run --port PORT
-    ```
+- `poe run-otel`
+
 
-- Run doc site locally
+### Run doc site locally
 
   ```sh
   poe doc
   ```
 
-- Generate doc site
+### Generate doc site
 
   ```sh
   poe doc-html
   ```
 
-- Build a docker image for tests
+### Build a docker image for tests
+
+- `poe docker-build --target test --build-tag 3.10-alpine --test true`
+- `poe docker-run --target test`
+
+### Build a container image to run the root files only without running any test
+
+- Build image
 
   ```sh
-  poe docker-build --target test --build-tag 3.10-alpine --test true
-  poe docker-run --target test
+  poe docker-build
   ```
 
-- Build a docker image to run the root files only without running any test
+- Set env vars
 
-  - Build image
+  ```sh
+  cp misc/docker.env.template docker.env
+  ## Make edits on docker.env
+  ```
 
-    ```sh
-    poe docker-build
-    ```
+- TIP: Make sure `docker.env` is in the `.gitignore` file so it isn't checked in with secrets. It's also safer if you only have the environment variable name for secrets and set the value in your shell with `export`.
 
-  - Set env vars
+### Build a container image with cached dependencies
 
-    ```sh
-    cp misc/docker.env.template docker.env
-    ## Make edits on docker.env
-    ```
+This is useful when iterating on the dockerfile itself
 
-  - TIP: Make sure `docker.env` is in the `.gitignore` file so it isn't checked in with secrets. It's also safer if you only have the environment variable name for secrets and set the value in your shell with `export`.
+  ```sh
+  poe docker-build-pipcached
+  ```
 
-  - Run container
+### Run container
 
-    ```sh
-    poe docker-run
-    ```
+  ```sh
+  poe docker-run
+  ```
 
-- Release
+### Release
 
+  Bump the version number in `pyproject.toml` (we follow [SemVer](https://semver.org/))
   Switch to `main` branch locally and run
 
   ```sh

misc/docker.env.template

@@ -14,3 +14,5 @@ DOCQ_SMTP_FROM="Docq.AI Support <[email protected]>" # A custom sender email
 # SERVER SETTINGS
 DOCQ_SERVER_ADDRESS = "http://localhost:8501" # Web address for the docq server, used for generating verification urls.
 
+OTEL_SERVICE_NAME = "docq-" #for local dev "docq-dev-<yourname>". Prod "docq-prod"
+HONEYCOMB_API_KEY = # or other Otel tracing backend. 

misc/secrets.toml.template

@@ -11,3 +11,5 @@ DOCQ_SMTP_FROM = "Docq.AI Support <[email protected]>" # A custom sender email
 
 # SERVER SETTINGS
 DOCQ_SERVER_ADDRESS = "http://localhost:8501" # Web address for the docq server, used for generating verification urls.
+OTEL_SERVICE_NAME = "docq-" #for local dev "docq-dev-<yourname>". Prod "docq-prod"
+HONEYCOMB_API_KEY = # or other Otel tracing backend.