Won't compile on FreeBSD 14.0 with the default GNUmakefile #370

bretton · 2024-07-22T20:01:31Z

I can't get this to build on FreeBSD 14.0 without some changes as covered below. They're rough and probably not the right way to do it. Can you look at making it more freebsd friendly please? I don't know how well it runs yet, however improving compile process is a start. I have podman containers working on the OS.

pkg install -y git go
git clone https://github.com/hashicorp/nomad-driver-podman
cd nomad-driver-podman
cp GNUmakefile Makefile
make dev

ERRORS

make: "/root/nomad-driver-podman/Makefile" line 4: Unknown modifier " -f1"
make: "/root/nomad-driver-podman/Makefile" line 8: Invalid line type
make: "/root/nomad-driver-podman/Makefile" line 9: Unknown modifier " -f1"
make: "/root/nomad-driver-podman/Makefile" line 10: Invalid line type
make: "/root/nomad-driver-podman/Makefile" line 81: Invalid line type
make: "/root/nomad-driver-podman/Makefile" line 83: Invalid line type
make: "/root/nomad-driver-podman/Makefile" line 85: Invalid line type
make: Fatal errors encountered -- cannot continue
make: stopped in /root/nomad-driver-podman

Versions

14.0-RELEASE-p8
go version go1.21.12 freebsd/amd64

SOLUTION:

edit Makefile and make the following edits

bash -> sh
remove cut statement
comment out ifndef/ifneq statements

SHELL = sh

# Handle multi-path environments
GOPATH := $(shell go env GOPATH)

# Respect $GOBIN if set in environment or via $GOENV file.
BIN := $(shell go env GOBIN)
#ifndef BIN
#BIN := $(GOPATH)/bin
#endif
default: help

HELP_FORMAT="    \033[36m%-25s\033[0m %s\n"
.PHONY: help
help: ## Display this usage information
	@echo "Valid targets:"
	@grep -E '^[^ ]+:.*?## .*$$' $(MAKEFILE_LIST) | \
		sort | \
		awk 'BEGIN {FS = ":.*?## "}; \
			{printf $(HELP_FORMAT), $$1, $$2}'
	@echo ""

.PHONY: changelogfmt
changelogfmt: ## Format changelog GitHub links
	@echo "--> Making [GH-xxxx] references clickable..."
	@sed -E 's|([^\[])\[GH-([0-9]+)\]|\1[[GH-\2](https://github.com/hashicorp/nomad-driver-podman/issues/\2)]|g' CHANGELOG.md > changelog.tmp && mv changelog.tmp CHANGELOG.md

.PHONY: check
check: hclfmt ## Lint the source code
	@echo "==> Linting source code ..."
	@$(BIN)/golangci-lint run
	@echo "==> vetting hc-log statements"
	@$(BIN)/hclogvet $(CURDIR)

.PHONY: hclfmt
hclfmt: ## Format HCL files with hclfmt
	@echo "--> Formatting HCL"
	@find . -name '.git' -prune \
	        -o \( -name '*.nomad' -o -name '*.hcl' -o -name '*.tf' \) \
	      -print0 | xargs -0 hclfmt -w
	@if (git status -s | grep -q -e '\.hcl$$' -e '\.nomad$$' -e '\.tf$$'); then echo The following HCL files are out of sync; git status -s | grep -e '\.hcl$$' -e '\.nomad$$' -e '\.tf$$'; exit 1; fi

.PHONY: deps
deps: ## Install build dependencies
	@echo "==> Installing build dependencies ..."
	go install github.com/golangci/golangci-lint/cmd/[email protected]
	go install github.com/hashicorp/go-hclog/[email protected]
	go install gotest.tools/[email protected]
	go install github.com/hashicorp/hcl/v2/cmd/hclfmt@d0c4fa8b0bbc2e4eeccd1ed2a32c2089ed8c5cf1

.PHONY: clean
clean: ## Cleanup previous build
	@echo "==> Cleanup previous build"
	rm -f ./build/nomad-driver-podman

.PHONY: dev
dev: clean build/nomad-driver-podman ## Build the nomad-driver-podman plugin

build/nomad-driver-podman:
	@echo "==> Building driver plugin ..."
	mkdir -p build
	CGO_ENABLED=0 \
	go build -o build/nomad-driver-podman .

.PHONY: test
test: ## Run unit tests
	@echo "==> Running unit tests ..."
	go test -v -race ./...

.PHONY: test-ci
test-ci: ## Run unit tests in CI
	@echo "==> Running unit tests in CI ..."
	@$(BIN)/gotestsum --format=testname --rerun-fails=0 --packages=". ./api" -- \
		-cover \
		-timeout=10m \
		-count=1 \
		. ./api

.PHONY: version
version:
#ifneq (,$(wildcard version/version_ent.go))
#	@$(CURDIR)/scripts/version.sh version/version.go version/version_ent.go
#else
	@$(CURDIR)/scripts/version.sh version/version.go version/version.go
#endif

# CRT release compilation
dist/%/nomad-driver-podman: GO_OUT ?= $@
dist/%/nomad-driver-podman:
	@echo "==> RELEASE BUILD of $@ ..."
	CGO_ENABLED=0 \
	GOOS=linux GOARCH=$(lastword $(subst _, ,$*)) \
	go build -trimpath -o $(GO_OUT)

# CRT release packaging (zip only)
.PRECIOUS: dist/%/nomad-driver-podman
dist/%.zip: dist/%/nomad-driver-podman
	@echo "==> RELEASE PACKAGING of $@ ..."
	@cp LICENSE $(dir $<)LICENSE.txt
	zip -j $@ $(dir $<)*

then run

make dev

when done

cp build/nomad-driver-podman /usr/local/libexec/nomad/plugins/

The text was updated successfully, but these errors were encountered:

bretton · 2024-08-02T18:51:42Z

This compiles with the altered Makefile and works mostly well with nomad client installed from the latest pkg sources. I can run freebsd-native podman containers, via nomad jobs, on freebsd, in seconds.

tgross · 2024-08-02T19:05:30Z

@bretton we might be willing to accept a PR but none of the folks working on the team have the bandwidth to test FreeBSD as a platform (much the same as with Nomad core), so it'd really be up to you to contribute that.

bretton · 2024-08-03T11:11:08Z

PEBKAC error, the issue can be solved by ensuring bash and gmake are installed from ports, and using gmake dev instead of make dev.

# pkg install -y bash git go gmake
# git clone https://github.com/hashicorp/nomad-driver-podman
# cd nomad-driver-podman
# gmake dev
==> Cleanup previous build
rm -f ./build/nomad-driver-podman
==> Building driver plugin ...
mkdir -p build
CGO_ENABLED=0 \
go build -o build/nomad-driver-podman .

when done copy the binary

# mkdir -p /usr/local/libexec/nomad/plugins/
# cp build/nomad-driver-podman /usr/local/libexec/nomad/plugins/

OneOfTheJohns · 2024-10-14T14:07:59Z

Hi @bretton

Trying to setup nomad client with nomad-driver-podman on Freebsd host, with the help of your previous comment was able to start the nomad client at least, but still facing such error -
[ERROR] client.driver_mgr.nomad-driver-podman: Could not get podman version: driver=podman @module=podman error="Get \"http://u/libpod/_ping\": dial unix ///run/podman/podman.sock: connect: no such file or directory"
Is there any written steps on how you achieved it working?

Thanks.

bretton · 2024-10-14T14:48:12Z

Hi @OneOfTheJohns

Make sure the podman service is started

pkg install podman-suite
service podman_service enable
service podman_service start

I have an example ansible playbook for setting up a test environment to demonstrate feasibility of podman containers on FreeBSD with nomad.
https://codeberg.org/Honeyguide/micropod-sampler

in particular this bit for podman-nomad-driver, however the podman service is started in a different role.
https://codeberg.org/Honeyguide/micropod-sampler/src/branch/main/roles/install_nomad_driver_podman/tasks/main.yml

OneOfTheJohns · 2024-10-14T15:27:12Z

Thanks @bretton for such swift answer and for sharing code.

I made sure that podman service is running, but it still throwing the same error.
Will try out your ansible playbook, maybe will find out where i made a mistake.
Thought now im thinking it could be a version of (podman? nomad? nomad-driver-podman?) issue.

Thank you.

bretton · 2024-10-14T15:34:10Z

@OneOfTheJohns check nomad client is running as root user?
https://discuss.hashicorp.com/t/nomad-with-podman-driver-setup-issues/69609/6

in my example playbook I have the nomad client service configured with

      sysrc nomad_user="root"
      sysrc nomad_group="wheel"
      sysrc nomad_env="PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin"
      sysrc nomad_args="-config=/usr/local/etc/nomad/client.hcl"
      sysrc nomad_debug="YES"

OneOfTheJohns · 2024-10-15T10:40:51Z

@bretton thanks once again for another tip, but still same error, as i was already running nomad client using root user. Will try to explain all steps i have taken so far in troubleshooting:

To install podman, i followed https://podman.io/docs/installation#installing-on-freebsd-140

Checked if podman service is running

root@:~ # service podman_service onestatus
podman_service is running as pid 1476.

Checked if permissions of the client directory are set (accodring to https://discuss.hashicorp.com/t/nomad-with-podman-driver-setup-issues/69609/6) they should be 0700

root@:~ # ls -la /usr/local/libexec/nomad/
total 23
drwx------  5 root wheel   6 Oct 15 08:55 .
drwx------  9 root wheel  18 Oct 15 08:55 ..
drwx------  2 root wheel   2 Oct 15 08:55 alloc
-rwx------  1 root wheel 394 Oct 15 08:55 checkpoint-signature
drwx------  2 root wheel   5 Oct 15 08:55 client
drwx------  2 root wheel   3 Oct 14 15:22 plugins

The contents of the client.hcl

root@:~ # cat /usr/local/etc/nomad/client.hcl

client {
    enabled = true
    servers = ["10.2.6.135"]
}
log_level = "DEBUG"
data_dir = "/usr/local/libexec/nomad"

Im starting the nomad client using root user

su -m root -c "nohup nomad agent -client -config=/usr/local/etc/nomad/client.hcl -servers=10.2.6.135 &"

I tried to search for the "run/podman/podman.sock" anywhere on the filesystem, but it seems that it was not created?

root@:~ # find / -name "*podman.socket*"
root@:~ #

So by now im quite confused, why would the podman socket not be open, if the service is running (if of course the service im running is not broken?). Still havent tested your ansible playbook though (only looked at it).

All of the logs im getting :

==> WARNING: mTLS is not configured - Nomad is not secure without mTLS!
==> Loaded configuration from /usr/local/etc/nomad/client.hcl
==> Starting Nomad agent...
==> Nomad agent configuration:

       Advertise Addrs: HTTP: 10.2.6.137:4646
            Bind Addrs: HTTP: [0.0.0.0:4646]
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: false
               Version: 1.8.1
               
==> Nomad agent started! Log data will stream in below:

    2024-10-15T09:50:06.581+0300 [WARN]  agent.plugin_loader: plugin not referenced in the agent configuration file, future versions of Nomad will not load this plugin until the agent configuration is updated: plugin_dir=/usr/local/libexec/nomad/plugins plugin=nomad-driver-podman
    2024-10-15T09:50:06.581+0300 [DEBUG] agent.plugin_loader: starting plugin: plugin_dir=/usr/local/libexec/nomad/plugins path=/usr/local/libexec/nomad/plugins/nomad-driver-podman args=["/usr/local/libexec/nomad/plugins/nomad-driver-podman"]
    2024-10-15T09:50:06.584+0300 [DEBUG] agent.plugin_loader: plugin started: plugin_dir=/usr/local/libexec/nomad/plugins path=/usr/local/libexec/nomad/plugins/nomad-driver-podman pid=1649
    2024-10-15T09:50:06.584+0300 [DEBUG] agent.plugin_loader: waiting for RPC address: plugin_dir=/usr/local/libexec/nomad/plugins plugin=/usr/local/libexec/nomad/plugins/nomad-driver-podman
    2024-10-15T09:50:06.602+0300 [DEBUG] agent.plugin_loader: using plugin: plugin_dir=/usr/local/libexec/nomad/plugins version=2
    2024-10-15T09:50:06.602+0300 [DEBUG] agent.plugin_loader.nomad-driver-podman: plugin address: plugin_dir=/usr/local/libexec/nomad/plugins network=unix address=/tmp/plugin3538403212 timestamp="2024-10-15T09:50:06.602+0300"
    2024-10-15T09:50:06.606+0300 [DEBUG] agent.plugin_loader.stdio: received EOF, stopping recv loop: plugin_dir=/usr/local/libexec/nomad/plugins err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2024-10-15T09:50:06.607+0300 [INFO]  agent.plugin_loader: plugin process exited: plugin_dir=/usr/local/libexec/nomad/plugins plugin=/usr/local/libexec/nomad/plugins/nomad-driver-podman id=1649
    2024-10-15T09:50:06.609+0300 [DEBUG] agent.plugin_loader: plugin exited: plugin_dir=/usr/local/libexec/nomad/plugins
    2024-10-15T09:50:06.611+0300 [DEBUG] agent.plugin_loader.docker: using client connection initialized from environment: plugin_dir=/usr/local/libexec/nomad/plugins
    2024-10-15T09:50:06.611+0300 [DEBUG] agent.plugin_loader: starting plugin: plugin_dir=/usr/local/libexec/nomad/plugins path=/usr/local/libexec/nomad/plugins/nomad-driver-podman args=["/usr/local/libexec/nomad/plugins/nomad-driver-podman"]
    2024-10-15T09:50:06.613+0300 [DEBUG] agent.plugin_loader: plugin started: plugin_dir=/usr/local/libexec/nomad/plugins path=/usr/local/libexec/nomad/plugins/nomad-driver-podman pid=1650
    2024-10-15T09:50:06.613+0300 [DEBUG] agent.plugin_loader: waiting for RPC address: plugin_dir=/usr/local/libexec/nomad/plugins plugin=/usr/local/libexec/nomad/plugins/nomad-driver-podman
    2024-10-15T09:50:06.631+0300 [DEBUG] agent.plugin_loader.nomad-driver-podman: plugin address: plugin_dir=/usr/local/libexec/nomad/plugins address=/tmp/plugin1891338384 network=unix timestamp="2024-10-15T09:50:06.631+0300"
    2024-10-15T09:50:06.631+0300 [DEBUG] agent.plugin_loader: using plugin: plugin_dir=/usr/local/libexec/nomad/plugins version=2
    2024-10-15T09:50:06.633+0300 [DEBUG] agent.plugin_loader.nomad-driver-podman: http baseurl: plugin_dir=/usr/local/libexec/nomad/plugins @module=podman url=unix:///run/podman/podman.sock timestamp="2024-10-15T09:50:06.633+0300"
    2024-10-15T09:50:06.633+0300 [DEBUG] agent.plugin_loader.nomad-driver-podman: http baseurl: plugin_dir=/usr/local/libexec/nomad/plugins @module=podman url=unix:///run/podman/podman.sock timestamp="2024-10-15T09:50:06.633+0300"
    2024-10-15T09:50:06.634+0300 [DEBUG] agent.plugin_loader.stdio: received EOF, stopping recv loop: plugin_dir=/usr/local/libexec/nomad/plugins err="rpc error: code = Unavailable desc = error reading from server: EOF"
    2024-10-15T09:50:06.634+0300 [INFO]  agent.plugin_loader: plugin process exited: plugin_dir=/usr/local/libexec/nomad/plugins plugin=/usr/local/libexec/nomad/plugins/nomad-driver-podman id=1650
    2024-10-15T09:50:06.634+0300 [DEBUG] agent.plugin_loader: plugin exited: plugin_dir=/usr/local/libexec/nomad/plugins
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=podman type=driver plugin_version=0.6.2
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=raw_exec type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=exec type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=qemu type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=java type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=docker type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  agent: detected plugin: name=mock_driver type=driver plugin_version=0.1.0
    2024-10-15T09:50:06.635+0300 [INFO]  client: using state directory: state_dir=/usr/local/libexec/nomad/client
    2024-10-15T09:50:06.635+0300 [INFO]  client: using alloc directory: alloc_dir=/usr/local/libexec/nomad/alloc
    2024-10-15T09:50:06.635+0300 [INFO]  client: using dynamic ports: min=20000 max=32000 reserved=""
    2024-10-15T09:50:06.649+0300 [DEBUG] client.fingerprint_mgr: built-in fingerprints: fingerprinters=["arch", "cni", "consul", "cpu", "host", "landlock", "memory", "network", "nomad", "plugins_cni", "signal", "storage", "vault", "env_aws", "env_gce", "env_azure", "env_digitalocean"]
    2024-10-15T09:50:06.649+0300 [DEBUG] client.fingerprint_mgr: CNI config dir is not set or does not exist, skipping: cni_config_dir=/opt/cni/config
    2024-10-15T09:50:06.649+0300 [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=consul initial_period=15s
    2024-10-15T09:50:06.650+0300 [DEBUG] client.fingerprint_mgr.cpu: detected CPU model: name="Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz"
    2024-10-15T09:50:06.650+0300 [DEBUG] client.fingerprint_mgr.cpu: detected CPU frequency: mhz=2400
    2024-10-15T09:50:06.650+0300 [DEBUG] client.fingerprint_mgr.cpu: detected CPU core count: cores=16
    2024-10-15T09:50:06.653+0300 [WARN]  client.fingerprint_mgr.landlock: failed to fingerprint kernel landlock feature: error="landlock not supported on this platform"
    2024-10-15T09:50:06.656+0300 [DEBUG] client.fingerprint_mgr.network: link speed could not be detected and no speed specified by user, falling back to default speed: interface=em0 mbits=1000
    2024-10-15T09:50:06.657+0300 [DEBUG] client.fingerprint_mgr.network: detected interface IP: interface=em0 IP=10.2.6.137
    2024-10-15T09:50:06.657+0300 [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: interface=em0 mbits=1000
    2024-10-15T09:50:06.662+0300 [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: interface=em1 mbits=1000
    2024-10-15T09:50:06.662+0300 [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: interface=lo0 mbits=1000
    2024-10-15T09:50:06.671+0300 [WARN]  client.fingerprint_mgr.cni_plugins: failed to read CNI plugins directory: cni_path=/opt/cni/bin error="open /opt/cni/bin: no such file or directory"
    2024-10-15T09:50:06.674+0300 [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=vault initial_period=15s
    2024-10-15T09:50:12.681+0300 [DEBUG] client.fingerprint_mgr.env_gce: could not read value for attribute: attribute=machine-type error="Get \"http://169.254.169.254/computeMetadata/v1/instance/machine-type\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
    2024-10-15T09:50:12.681+0300 [DEBUG] client.fingerprint_mgr.env_gce: error querying GCE Metadata URL, skipping
    2024-10-15T09:50:14.682+0300 [DEBUG] client.fingerprint_mgr.env_azure: could not read value for attribute: attribute=compute/azEnvironment error="Get \"http://169.254.169.254/metadata/instance/compute/azEnvironment?api-version=2019-06-04&format=text\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
    2024-10-15T09:50:16.682+0300 [DEBUG] client.fingerprint_mgr.env_digitalocean: failed to request metadata: attribute=region error="Get \"http://169.254.169.254/metadata/v1/region\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
    2024-10-15T09:50:16.682+0300 [DEBUG] client.fingerprint_mgr: detected fingerprints: node_attrs=["arch", "cpu", "host", "network", "nomad", "signal", "storage"]
    2024-10-15T09:50:16.682+0300 [INFO]  client.plugin: starting plugin manager: plugin-type=csi
    2024-10-15T09:50:16.682+0300 [INFO]  client.plugin: starting plugin manager: plugin-type=driver
    2024-10-15T09:50:16.682+0300 [INFO]  client.plugin: starting plugin manager: plugin-type=device
    2024-10-15T09:50:16.682+0300 [DEBUG] client.device_mgr: exiting since there are no device plugins
    2024-10-15T09:50:16.682+0300 [DEBUG] client.plugin: waiting on plugin manager initial fingerprint: plugin-type=device
    2024-10-15T09:50:16.682+0300 [DEBUG] client.driver_mgr.docker: using client connection initialized from environment: driver=docker
    2024-10-15T09:50:16.682+0300 [DEBUG] client.plugin: finished plugin manager initial fingerprint: plugin-type=device
    2024-10-15T09:50:16.682+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=raw_exec health=undetected description=disabled
    2024-10-15T09:50:16.682+0300 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=podman error="Reattachment process not found"
    2024-10-15T09:50:16.682+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=qemu health=undetected description=""
    2024-10-15T09:50:16.682+0300 [DEBUG] client.plugin: waiting on plugin manager initial fingerprint: plugin-type=driver
    2024-10-15T09:50:16.682+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=mock_driver health=healthy description=Healthy
    2024-10-15T09:50:16.682+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=exec health=undetected description="exec driver unsupported on client OS"
    2024-10-15T09:50:16.683+0300 [DEBUG] client.driver_mgr: starting plugin: driver=podman path=/usr/local/libexec/nomad/plugins/nomad-driver-podman args=["/usr/local/libexec/nomad/plugins/nomad-driver-podman"]
    2024-10-15T09:50:16.683+0300 [DEBUG] client.driver_mgr.docker: could not connect to docker daemon: driver=docker endpoint=unix:///var/run/docker.sock error="Get \"http://unix.sock/version\": dial unix /var/run/docker.sock: connect: no such file or directory"
    2024-10-15T09:50:16.683+0300 [ERROR] client.driver_mgr.docker: failed to list pause containers for recovery: driver=docker error="Get \"http://unix.sock/containers/json?filters=%7B%22label%22%3A%5B%22com.hashicorp.nomad.alloc_id%22%5D%7D\": dial unix /var/run/docker.sock: connect: no such file or directory"
    2024-10-15T09:50:16.683+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=docker health=undetected description="Failed to connect to docker daemon"
    2024-10-15T09:50:16.685+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=java health=undetected description=""
    2024-10-15T09:50:16.685+0300 [DEBUG] client.driver_mgr: plugin started: driver=podman path=/usr/local/libexec/nomad/plugins/nomad-driver-podman pid=1668
    2024-10-15T09:50:16.685+0300 [DEBUG] client.driver_mgr: waiting for RPC address: driver=podman plugin=/usr/local/libexec/nomad/plugins/nomad-driver-podman
    2024-10-15T09:50:16.686+0300 [DEBUG] client.server_mgr: new server list: new_servers=[10.2.6.135:4647, 10.2.6.135:4647] old_servers=[]
    2024-10-15T09:50:16.703+0300 [DEBUG] client.driver_mgr.nomad-driver-podman: plugin address: driver=podman address=/tmp/plugin3145325583 network=unix timestamp="2024-10-15T09:50:16.703+0300"
    2024-10-15T09:50:16.703+0300 [DEBUG] client.driver_mgr: using plugin: driver=podman version=2
    2024-10-15T09:50:16.705+0300 [DEBUG] client.driver_mgr.nomad-driver-podman: http baseurl: driver=podman @module=podman url=unix:///run/podman/podman.sock timestamp="2024-10-15T09:50:16.705+0300"
    2024-10-15T09:50:16.717+0300 [ERROR] client.driver_mgr.nomad-driver-podman: Could not get podman version: driver=podman @module=podman error="Get \"http://u/libpod/_ping\": dial unix ///run/podman/podman.sock: connect: no such file or directory" timestamp="2024-10-15T09:50:16.717+0300"
    2024-10-15T09:50:16.718+0300 [DEBUG] client.driver_mgr: initial driver fingerprint: driver=podman health=undetected description="Cannot connect to any Podman socket."
    2024-10-15T09:50:16.718+0300 [DEBUG] client.driver_mgr: detected drivers: drivers="map[healthy:[mock_driver] undetected:[raw_exec qemu exec docker java podman]]"
    2024-10-15T09:50:16.718+0300 [DEBUG] client.plugin: finished plugin manager initial fingerprint: plugin-type=driver
    2024-10-15T09:50:16.718+0300 [INFO]  client: started client: node_id=e090c84b-06f0-3568-598a-1abb3983df98
    2024-10-15T09:50:16.719+0300 [DEBUG] client: updated allocations: index=51 total=0 pulled=0 filtered=0
    2024-10-15T09:50:16.719+0300 [DEBUG] client: allocation updates: added=0 removed=0 updated=0 ignored=0
    2024-10-15T09:50:16.719+0300 [DEBUG] client: allocation updates applied: added=0 removed=0 updated=0 ignored=0 errors=0
    2024-10-15T09:50:16.720+0300 [DEBUG] http: UI is enabled
    2024-10-15T09:50:16.721+0300 [DEBUG] http: UI is enabled
    2024-10-15T09:50:16.726+0300 [DEBUG] client.server_mgr: new server list: new_servers=[10.2.6.135:4647] old_servers=[10.2.6.135:4647, 10.2.6.135:4647]
    2024-10-15T09:50:16.726+0300 [INFO]  client: node registration complete
    2024-10-15T09:50:19.919+0300 [DEBUG] client: state updated: node_status=ready
    2024-10-15T09:50:23.960+0300 [DEBUG] client: state changed, updating node and re-registering
    2024-10-15T09:50:23.967+0300 [INFO]  client: node registration complete
==> Newer Nomad version available: 1.9.0 (currently running: 1.8.1)
    2024-10-15T09:50:46.845+0300 [ERROR] client.driver_mgr.nomad-driver-podman: Could not get podman version: driver=podman @module=podman error="Get \"http://u/libpod/_ping\": dial unix ///run/podman/podman.sock: connect: no such file or directory" timestamp="2024-10-15T09:50:46.845+0300"

bretton · 2024-10-15T14:48:58Z

@OneOfTheJohns my nomad is setup to use /var/tmp/nomad with freebsd pkg install

# ls -al /var/tmp/nomad
total 15
drwx------  4 root wheel   5 Jul 30 22:41 .
drwxrwxrwt  6 root wheel   7 Aug  4 17:15 ..
drwx--x--x  3 root wheel   3 Aug  4 17:15 alloc
-rw-r--r--  1 root wheel 394 Jul 30 22:41 checkpoint-signature
drwx------  2 root wheel   5 Jul 30 22:14 client

and /usr/local/libexec/nomad/ only has a plugins directory with contents from the nomad-driver-podman setup

-rwxr-xr-x  1 root wheel 31680892 Aug  4 17:13 nomad-driver-podman

my /usr/local/etc/nomad/client.hcl

bind_addr = "192.168.88.20"
datacenter = "micropod"
data_dir = "/var/tmp/nomad"
advertise {
  # This should be the IP of THIS MACHINE and must be routable by every node
  # in your cluster
  http = "192.168.88.20"
  rpc = "192.168.88.20"
  serf = "192.168.88.20"
}
client {
  enabled = true
  servers = [ "10.88.0.12" ]
}
plugin "raw_exec" {
  config {
    enabled = true
  }
}
plugin "nomad-driver-podman" {
  config {
    socket_path = "unix://var/run/podman/podman.sock"
    # Customize other Podman driver plugin options here if needed
  }
}
plugin_dir = "/usr/local/libexec/nomad/plugins"
consul {
  address = "10.88.0.11:8500"
  client_service_name = "nomad.client.micropod.consul"
  auto_advertise = true
  client_auto_join = true
}
tls {
  http = false
  rpc = false
  verify_server_hostname = false
  verify_https_client = false
}
telemetry {
  collection_interval = "15s"
  publish_allocation_metrics = true
  publish_node_metrics = true
  prometheus_metrics = true
  disable_hostname = true
}
enable_syslog=true
log_level="WARN"
syslog_facility="LOCAL1"

OneOfTheJohns · 2024-10-16T08:45:33Z

Thanks @bretton for your time, i finally fixed my issue.
The problem was indeed in socket not being created for some reason.
But i was able to create it by running :

su -m root -c "nohup podman system service --time=0 unix:///tmp/podman.sock &"

bretton changed the title ~~Won't compile on FreeBSD 14.0 with thr default GNUmakefile~~ Won't compile on FreeBSD 14.0 with the default GNUmakefile Jul 22, 2024

bretton closed this as completed Aug 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Won't compile on FreeBSD 14.0 with the default GNUmakefile #370

Won't compile on FreeBSD 14.0 with the default GNUmakefile #370

bretton commented Jul 22, 2024

bretton commented Aug 2, 2024

tgross commented Aug 2, 2024

bretton commented Aug 3, 2024

OneOfTheJohns commented Oct 14, 2024

bretton commented Oct 14, 2024

OneOfTheJohns commented Oct 14, 2024

bretton commented Oct 14, 2024

OneOfTheJohns commented Oct 15, 2024

bretton commented Oct 15, 2024

OneOfTheJohns commented Oct 16, 2024

Won't compile on FreeBSD 14.0 with the default GNUmakefile #370

Won't compile on FreeBSD 14.0 with the default GNUmakefile #370

Comments

bretton commented Jul 22, 2024

ERRORS

Versions

SOLUTION:

bretton commented Aug 2, 2024

tgross commented Aug 2, 2024

bretton commented Aug 3, 2024

OneOfTheJohns commented Oct 14, 2024

bretton commented Oct 14, 2024

OneOfTheJohns commented Oct 14, 2024

bretton commented Oct 14, 2024

OneOfTheJohns commented Oct 15, 2024

To install podman, i followed https://podman.io/docs/installation#installing-on-freebsd-140

Checked if podman service is running

Checked if permissions of the client directory are set (accodring to https://discuss.hashicorp.com/t/nomad-with-podman-driver-setup-issues/69609/6) they should be 0700

The contents of the client.hcl

Im starting the nomad client using root user

I tried to search for the "run/podman/podman.sock" anywhere on the filesystem, but it seems that it was not created?

So by now im quite confused, why would the podman socket not be open, if the service is running (if of course the service im running is not broken?). Still havent tested your ansible playbook though (only looked at it).

All of the logs im getting :

bretton commented Oct 15, 2024

OneOfTheJohns commented Oct 16, 2024