Separate probes for PV provisioning and mounting

[ushitora-anqou]

The current probe checks that both a new provisioning of a PV and its mounting succeed on every Node.
This guarantee is sufficient but not necessary; although mounting an already provisioned PV should succeed on every node, it is sufficient that a new provisioning succeeds on at least one Node.

To address the above issue, the new architecture has the following two types of probes:

• provision-probe, which checks that a new provision succeeds; and
• mount-probe, which checks that a PV (possibly already provisioned) can be successfully mounted on each Node.

In addition, PieProbe custom resource is introduced to group the probes that monitor a StorageClass. Each probe has an owner reference to a PieProbe and is GCed when the referenced PieProbe is deleted. See also issue #50.

This PR introduces the following new metrics:

• pie_io_write_latency_on_mount_probe_seconds
  • Experimental metrics. IO latency of write, benchmarked on mount-probe Pods.
  • TYPE: gauge
• pie_io_read_latency_on_mount_probe_seconds
  • Experimental metrics. IO latency of read, benchmarked on mount-probe Pods.
  • TYPE: gauge
• pie_mount_probe_total
  • Experimental metrics. The number of attempts of the creation of the mount-probe Pod object and the creation of the container.
  • TYPE: counter
• pie_performance_on_mount_probe_total
  • Experimental metrics. The number of attempts of performing the IO benchmarks on mount-probe Pods.
  • TYPE: counter
• pie_provision_probe_total
  • Experimental metrics. The number of attempts of the creation of the provision-probe Pod object and the creation of the container.
  • TYPE: counter

Note that old metrics (e.g., pie_io_write_latency_seconds and pie_create_probe_total) should behave in the same way as before merging this patch.

[ushitora-anqou]

I updated my PR. The changes are: https://github.com/topolvm/pie/compare/a9b5c369798460d423528113786504830e4f809f..c019c385d3bcb19bc66f0eb728adafd5d1d8d86c

• I rebased this branch onto the current main branch to resolve the conflicts.
• I dropped the commit of adding the config/ directory to .gitignore.

[pluser]

LGTM!