DAOS-14410 common: update PMDK to version 2.0.0

ci/provisioning/post_provision_config_common_functions.sh

@@ -266,11 +266,11 @@ post_provision_config_nodes() {
 
     if $CONFIG_POWER_ONLY; then
         rm -f "$REPOS_DIR"/*.hpdd.intel.com_job_daos-stack_job_*_job_*.repo
-        time dnf -y erase fio fuse ior-hpc mpich-autoload               \
+        time dnf -y erase fio fuse ior-hpc mpich-autoload          \
                      ompi argobots cart daos daos-client dpdk      \
                      fuse-libs libisa-l libpmemobj mercury mpich   \
                      pmix protobuf-c spdk libfabric libpmem        \
-                     libpmemblk munge-libs munge slurm             \
+                     munge-libs munge slurm                        \
                      slurm-example-configs slurmctld slurm-slurmmd
     fi
 

ci/provisioning/post_provision_config_nodes_UBUNTU_20_04.sh

@@ -8,7 +8,7 @@ post_provision_config_nodes() {
     #                 ompi argobots cart daos daos-client dpdk      \
     #                 fuse-libs libisa-l libpmemobj mercury mpich   \
     #                 pmix protobuf-c spdk libfabric libpmem        \
-    #                 libpmemblk munge-libs munge slurm             \
+    #                 munge-libs munge slurm                        \
     #                 slurm-example-configs slurmctld slurm-slurmmd
     #fi
     codename=$(lsb_release -s -c)

debian/changelog

@@ -1,3 +1,16 @@
+daos (2.5.100-10) unstable; urgency=medium
+  [ Tomasz Gromadzki ]
+  * Update PMDK to 2.0.0
+    * Remove libpmemblk from dependencies.
+    * Start using BUILD_EXAMPLES=n and BUILD_BENCHMARKS=n instead of patches.
+    * Stop using BUILD_RPMEM=n (removed) and NDCTL_DISABLE=y (invalid).
+    * Point https://github.com/pmem/pmdk as the main PMDK reference source.
+    NOTE: PMDK upgrade to 2.0.0 does not affect any API call used by DAOS.
+          libpmemobj (and libpmem) API stays unchanged.
+
+ -- Tomasz Gromadzki <[email protected]>  Mon, 02 Oct 2023 12:52:00 -0400
+
+
 daos (2.5.100-9) unstable; urgency=medium
 
   [ Brian J. Murrell ]

debian/control

@@ -18,7 +18,7 @@ Build-Depends: debhelper (>= 10),
                python3-distro,
                libabt-dev,
                libucx-dev,
-               libpmemobj-dev (>= 1.12.1~rc1),
+               libpmemobj-dev (>= 2.0.0),
                libfuse3-dev,
                libprotobuf-c-dev,
                libjson-c-dev,

docs/admin/hardware.md

@@ -21,7 +21,7 @@ servers.
 DAOS requires a 64-bit processor architecture and is primarily developed
 on Intel x86\_64 architecture. The DAOS software and the libraries it
 depends on (e.g., [ISA-L](https://github.com/intel/isa-l),
-[SPDK](https://pmem.io/pmdk/), [PMDK](https://spdk.io/), and
+[SPDK](https://spdk.io/), [PMDK](https://github.com/pmem/pmdk/), and
 [DPDK](https://www.dpdk.org/) can take
 advantage of Intel Intel Streaming SIMD (SSE) and Intel Advanced Vector (AVX) extensions.
 

docs/admin/troubleshooting.md

@@ -907,7 +907,7 @@ Please refer the [ndctl list](https://docs.pmem.io/ndctl-user-guide/ndctl-man-pa
 
 The pmempool is a management tool for Persistent Memory pool files created by PMDK libraries.
 DAOS uses the PMDK library to manage persistence inside ext4 files.
-[pmempool](https://pmem.io/pmdk/manpages/linux/v1.9/pmempool/pmempool-check.1/) can check consistency of a given pool file.
+[pmempool](https://github.com/pmem/pmdk/blob/stable-2.0/doc/pmempool/pmempool-check.1.md) can check consistency of a given pool file.
 It can be run with -r (repair) option which can fix some of the issues with pool file. DAOS will have more number of such pool file (vos-*), based
 on number of targets mention per daos engine. User may need to check each vos pool file for corruption on faulty pool.
 

docs/overview/terminology.md

@@ -32,7 +32,7 @@
 |[OFI](https://ofiwg.github.io/libfabric/)|Open Fabrics Interfaces|
 |OS|Operating System|
 |PM|Persistent Memory|
-|[PMDK](https://pmem.io/pmdk/)|Persistent Memory Devevelopment Kit|
+|[PMDK](https://github.com/pmem/pmdk)|Persistent Memory Devevelopment Kit|
 |[RAFT](https://raft.github.io/)|Raft is a consensus algorithm used to distribute state transitions among DAOS server nodes.|
 |RAS|Reliability, Availability & Serviceability|
 |RDB|Replicated Database, containing pool metadata and maintained across DAOS servers using the Raft algorithm.|

site_scons/components/__init__.py

@@ -265,9 +265,9 @@ def define_components(reqs):
                 retriever=GitRepoRetriever('https://github.com/pmem/pmdk.git'),
                 commands=[['make',
                            'all',
-                           'BUILD_RPMEM=n',
                            'NDCTL_ENABLE=n',
-                           'NDCTL_DISABLE=y',
+                           'BUILD_EXAMPLES=n',
+                           'BUILD_BENCHMARKS=n',
                            'DOC=n',
                            'EXTRA_CFLAGS="-Wno-error"',
                            'install',

src/vos/README.md

@@ -58,7 +58,7 @@ Please refer to the <a href="../bio/README.md">Blob I/O</a> (BIO) module for mor
 Special care is taken when developing and modifying the VOS layer because any software bug could corrupt data structures in persistent memory.
 The VOS, therefore, checksums its persistent data structures despite the presence of hardware ECC.
 
-The VOS provides a lightweight I/O stack fully in user space, leveraging the <a href="pmem.io">PMDK</a> open-source libraries developed to support this programming model.
+The VOS provides a lightweight I/O stack fully in user space, leveraging the <a href="https://github.com/pmem/pmdk">PMDK</a> open-source libraries developed to support this programming model.
 
 <a id="64"></a>
 
@@ -73,7 +73,7 @@ It is worth noting that such transactions are different from the DAOS transactio
 Persistent memory transactions must guarantee consistency of VOS internal data structures when processing incoming requests, regardless of their epoch number.
 Transactions over persistent memory can be implemented in many different ways, e.g., undo logs, redo logs, a combination of both, or copy-on-write.
 
-<a href="https://pmem.io">PMDK</a> is an open source collection of libraries for using persistent memory, optimized specifically for NVRAM.
+<a href="https://github.com/pmem/pmdk">PMDK</a> is an open source collection of libraries for using persistent memory, optimized specifically for NVRAM.
 Among these is the libpmemobj library, which implements relocatable persistent heaps called persistent memory pools.
 This includes memory allocation, transactions, and general facilities for persistent memory programming.
 The transactions are local to one thread (not multi-threaded) and rely on undo logs.

src/vos/vos_dtx.c

@@ -10,7 +10,6 @@
  */
 #define D_LOGFAC	DD_FAC(vos)
 
-#include <libpmem.h>
 #include <daos_srv/vos.h>
 #include <daos/common.h>
 #include "vos_layout.h"

utils/build.config

@@ -3,7 +3,7 @@ component=daos
 
 [commit_versions]
 ARGOBOTS = v1.1
-PMDK = 1.12.1
+PMDK = 2.0.0
 ISAL = v2.30.0
 ISAL_CRYPTO = v2.23.0
 SPDK = v22.01.2

utils/rpms/daos.spec

@@ -15,7 +15,7 @@
 
 Name:          daos
 Version:       2.5.100
-Release:       9%{?relval}%{?dist}
+Release:       10%{?relval}%{?dist}
 Summary:       DAOS Storage Engine
 
 License:       BSD-2-Clause-Patent
@@ -49,7 +49,7 @@ BuildRequires: libabt-devel >= 1.0rc1
 BuildRequires: libjson-c-devel
 BuildRequires: boost-devel
 %endif
-BuildRequires: libpmemobj-devel >= 1.12.1~rc1
+BuildRequires: libpmemobj-devel >= 2.0.0
 %if (0%{?rhel} >= 8)
 BuildRequires: fuse3-devel >= 3
 %else
@@ -145,11 +145,11 @@ Requires: ndctl
 # needed to set PMem configuration goals in BIOS through control-plane
 %if (0%{?suse_version} >= 1500)
 Requires: ipmctl >= 03.00.00.0423
-Requires: libpmemobj1 >= 1.12.1~rc1-1.suse1500
+Requires: libpmemobj1 >= 2.0.0-1.suse1500
 Requires: libfabric1 >= %{libfabric_version}
 %else
 Requires: ipmctl >= 03.00.00.0468
-Requires: libpmemobj >= 1.12.1~rc1-1%{?dist}
+Requires: libpmemobj >= 2.0.0-1%{?dist}
 %endif
 Requires: libfabric >= %{libfabric_version}
 Requires: mercury >= %{mercury_version}
@@ -585,6 +585,15 @@ getent passwd daos_agent >/dev/null || useradd -s /sbin/nologin -r -g daos_agent
 # No files in a shim package
 
 %changelog
+* Mon Oct 02 2023 Tomasz Gromadzki <[email protected]> 2.5.100-10
+- Update to PMDK 2.0.0
+  * Remove libpmemblk from dependencies.
+  * Start using BUILD_EXAMPLES=n and BUILD_BENCHMARKS=n instead of patches.
+  * Stop using BUILD_RPMEM=n (removed) and NDCTL_DISABLE=y (invalid).
+  * Point https://github.com/pmem/pmdk as the main PMDK reference source.
+  NOTE: PMDK upgrade to 2.0.0 does not affect any API call used by DAOS.
+        libpmemobj (and libpmem) API stays unchanged.
+
 * Wed Aug 23 2023 Brian J. Murrell <[email protected]> 2.5.100-9
 - Update fuse3 requirement to R: /usr/bin/fusermount3 by path
   rather than by package name, for portability and future-proofing