Fix startup errors on STIG compliant systems due to noexec filesystems

[QU3B1M]

Description

To avoid errors when starting wazuh-indexer on STIG compliant systems, where the /var/log directory is noexec, we have moved the temporary directory to /var/lib, which (almost every case) is not set to noexec. Additionally, the .restart file, which indicates when the system should be restarted after an upgrade, has been relocated to the new ../tmp directory (previously located in /tmp).

The creation of the ../tmp directory has been moved from the postinst step to the preinst step for consistency.

Related Issues

Resolves #501

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[QU3B1M]

Reproduced the error installing wazuh-indexer package without the fix on a STIG compliant system (/var/log, /var/tmp, /tmp being noexec)

Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: A terminally deprecated method in java.lang.System has been called
Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/s>
Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: System::setSecurityManager will be removed in a future release
Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: Nov 11, 2024 12:58:44 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
Nov 11 12:58:44 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: COMPAT locale provider will be removed in a future release
Nov 11 12:58:45 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: A terminally deprecated method in java.lang.System has been called
Nov 11 12:58:45 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/sha>
Nov 11 12:58:45 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Nov 11 12:58:45 ubuntu2204.localdomain systemd-entrypoint[4549]: WARNING: System::setSecurityManager will be removed in a future release
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: ERROR: [1] bootstrap checks failed
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: [1]: system call filters failed to install; check the logs and fix your configuration or disable system>
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: ERROR: OpenSearch did not exit normally - check the logs at /var/log/wazuh-indexer/wazuh-cluster.log
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: fatal error in thread [Thread-3], exiting
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.systemd.Libsystemd.lambda$static$0(Libsystemd.java:48)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.systemd.Libsystemd.<clinit>(Libsystemd.java:47)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.systemd.SystemdPlugin.sd_notify(SystemdPlugin.java:126)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.systemd.SystemdPlugin.close(SystemdPlugin.java:152)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:89)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:131)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:114)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.node.Node.close(Node.java:1791)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:89)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:131)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.common.util.io.IOUtils.close(IOUtils.java:81)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]:         at org.opensearch.bootstrap.Bootstrap$4.run(Bootstrap.java:206)
Nov 11 12:58:50 ubuntu2204.localdomain systemd-entrypoint[4549]: Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /var/log/wa>

Validate the package with the fix can be correctly installed and started on the same system

1. Install wazuh-indexer package with the fix

   bash 01_download_and_install_package.sh -id 11777760990 -n wazuh-indexer_4.10.2-0_amd64_c996476.deb
   Fetching artifacts list...
   Checking wazuh-indexer_4.10.2-0_amd64_c996476.deb package is generated for workflow run 11777760990
   Wazuh indexer artifact detected. Artifact ID: 2170967024
   Downloading wazuh-indexer package from GitHub artifactory...
   (It could take a couple of minutes)
   Package downloaded successfully
   Decompressing wazuh-indexer package...
   Archive:  ./package.zip
     inflating: wazuh-indexer_4.10.2-0_amd64_c996476.deb  
   Package decompressed
   Installing wazuh-indexer package...
   Package installed successfully.

2. Start wazuh-indexer service

   systemctl start wazuh-indexer
   systemctl status wazuh-indexer
   
   ● wazuh-indexer.service - wazuh-indexer
        Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
        Active: active (running) since Mon 2024-11-11 13:12:23 UTC; 10s ago
          Docs: https://documentation.wazuh.com
      Main PID: 4857 (java)
         Tasks: 62 (limit: 4558)
        Memory: 1.3G
           CPU: 19.094s
        CGroup: /system.slice/wazuh-indexer.service
                └─4857 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+A>
   
   Nov 11 13:12:16 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/s>
   Nov 11 13:12:16 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
   Nov 11 13:12:16 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: System::setSecurityManager will be removed in a future release
   Nov 11 13:12:16 ubuntu2204.localdomain systemd-entrypoint[4857]: Nov 11, 2024 1:12:16 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
   Nov 11 13:12:16 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: COMPAT locale provider will be removed in a future release
   Nov 11 13:12:17 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: A terminally deprecated method in java.lang.System has been called
   Nov 11 13:12:17 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/sha>
   Nov 11 13:12:17 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
   Nov 11 13:12:17 ubuntu2204.localdomain systemd-entrypoint[4857]: WARNING: System::setSecurityManager will be removed in a future release
   Nov 11 13:12:23 ubuntu2204.localdomain systemd[1]: Started wazuh-indexer.
   

[QU3B1M]

Check service maintains its previous status on a upgrade using this PR's package

1. Check actual status

   systemctl status wazuh-indexer
   ● wazuh-indexer.service - wazuh-indexer
        Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
        Active: active (running) since Mon 2024-11-11 13:12:23 UTC; 10s ago
          Docs: https://documentation.wazuh.com
      Main PID: 4857 (java)
         Tasks: 62 (limit: 4558)
        Memory: 1.3G
           CPU: 19.094s
        CGroup: /system.slice/wazuh-indexer.service
                └─4857 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+A>

2. Upgrade wazuh-indexer using the package with the fix

   dpkg -i wazuh-indexer_4.10.2-0_amd64_c996476.deb 
   (Reading database ... 77523 files and directories currently installed.)
   Preparing to unpack wazuh-indexer_4.10.2-0_amd64_c996476.deb ...
   Running Wazuh Indexer Pre-Installation Script
   Stop existing wazuh-indexer.service
   Unpacking wazuh-indexer (4.10.2-0) over (4.10.2-0) ...
   Setting up wazuh-indexer (4.10.2-0) ...
   Running Wazuh Indexer Post-Installation Script
   Restarting wazuh-indexer service...

3. Check current status

   systemctl status wazuh-indexer
   ● wazuh-indexer.service - wazuh-indexer
        Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
        Active: active (running) since Mon 2024-11-11 13:18:26 UTC; 1min 49s ago
          Docs: https://documentation.wazuh.com
      Main PID: 5168 (java)
         Tasks: 69 (limit: 4558)
        Memory: 1.3G
           CPU: 23.221s
        CGroup: /system.slice/wazuh-indexer.service
                └─5168 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+A>

[f-galland]

Packages from this branch get installed and run properly on an almalinux 9 installation with /tmp, /var/log and /var/tmp mounted with the noexec flag:

[root@node-1 scripts]# cat /etc/os-release 
NAME="AlmaLinux"
VERSION="9.3 (Shamrock Pampas Cat)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.3"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.3 (Shamrock Pampas Cat)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.3"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.3"


[root@node-1 scripts]# mount | grep noexec
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
none on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
none on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
/dev/mapper/almalinux_alma9-root on /tmp type xfs (rw,noexec,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/almalinux_alma9-root on /var/log type xfs (rw,noexec,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/almalinux_alma9-root on /var/tmp type xfs (rw,noexec,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
none on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)

[root@node-1 scripts]# systemctl status wazuh-indexer --no-pager -l
● wazuh-indexer.service - wazuh-indexer
     Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; disabled; preset: disabled)
     Active: active (running) since Mon 2024-11-25 15:34:30 UTC; 2min 39s ago
       Docs: https://documentation.wazuh.com
   Main PID: 5641 (java)
      Tasks: 81 (limit: 24731)
     Memory: 1.3G
        CPU: 28.488s
     CGroup: /system.slice/wazuh-indexer.service
             └─5641 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.security.manager=allow -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/var/lib/wazuh-indexer/tmp -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/wazuh-indexer -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log "-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m" -Djava.security.manager=allow -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///etc/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED -XX:MaxDirectMemorySize=536870912 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp "/usr/share/wazuh-indexer/lib/*" org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Nov 25 15:34:23 node-1 systemd-entrypoint[5641]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.16.0.jar)
Nov 25 15:34:23 node-1 systemd-entrypoint[5641]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Nov 25 15:34:23 node-1 systemd-entrypoint[5641]: WARNING: System::setSecurityManager will be removed in a future release
Nov 25 15:34:23 node-1 systemd-entrypoint[5641]: Nov 25, 2024 3:34:23 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
Nov 25 15:34:23 node-1 systemd-entrypoint[5641]: WARNING: COMPAT locale provider will be removed in a future release
Nov 25 15:34:24 node-1 systemd-entrypoint[5641]: WARNING: A terminally deprecated method in java.lang.System has been called
Nov 25 15:34:24 node-1 systemd-entrypoint[5641]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.16.0.jar)
Nov 25 15:34:24 node-1 systemd-entrypoint[5641]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Nov 25 15:34:24 node-1 systemd-entrypoint[5641]: WARNING: System::setSecurityManager will be removed in a future release
Nov 25 15:34:30 node-1 systemd[1]: Started wazuh-indexer.


[root@node-1 scripts]# curl -sku admin:admin https://localhost:9200
{
  "name" : "node-1",
  "cluster_name" : "wazuh-cluster",
  "cluster_uuid" : "LPWidYpFS4uM5fNvKZ1X0w",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "rpm",
    "build_hash" : "c99647645cd8e8871653c65d6c451a32711ded50",
    "build_date" : "2024-11-11T11:37:51.521499Z",
    "build_snapshot" : false,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

[f-galland]

LGTM!