perf_event_open fails on rpi4/32-bit

[kraj]

strace has serveral failures which all lead to one problem that is __NR_perf_event_open syscall fails reliably on rpi4/32-bit kernel. Here is small example which should work fine if PMU was enabled and functioning correctly

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/ioctl.h>
#include <linux/perf_event.h>
#include <asm/unistd.h>

static long
perf_event_open(struct perf_event_attr *hw_event, pid_t pid,
int cpu, int group_fd, unsigned long flags)
{
int ret;

ret = syscall(__NR_perf_event_open, hw_event, pid, cpu,
group_fd, flags);
return ret;
}

int
main(int argc, char **argv)
{
struct perf_event_attr pe;
long long count;
int fd;

memset(&pe, 0, sizeof(struct perf_event_attr));
pe.type = PERF_TYPE_HARDWARE;
pe.size = sizeof(struct perf_event_attr);
pe.config = PERF_COUNT_HW_INSTRUCTIONS;
pe.disabled = 1;
pe.exclude_kernel = 1;
pe.exclude_hv = 1;

fd = perf_event_open(&pe, 0, -1, -1, 0);
if (fd == -1) {
fprintf(stderr, "Error opening leader %llx\n", pe.config);
exit(EXIT_FAILURE);
}

ioctl(fd, PERF_EVENT_IOC_RESET, 0);
ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);

printf("Measuring instruction count for this printf\n");

ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
read(fd, &count, sizeof(long long));

printf("Used %lld instructions\n", count);

close(fd);
}

This however returns Error opening leader 1

a working system returns

Measuring instruction count for this printf
Used 76584 instructions

There is no trace of PMU driver in dmesg either, which lets me think that is PMU not
enabled on this kernel somehow. I am using u-boot for bootloader and latest kernel
Linux version 4.19.108-v7l

CONFIG_HW_PERF_EVENTS and CONFIG_PERF_EVENTS are set to 'y' in .config

[pelwell]

Interesting. Pi 3 works, as does Pi 4 on 64-bit (just add arm_64bit=1 to config.txt).

[kraj]

yes with musl based rootfs I am seeing

[    0.338552] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available

and the example seems to work ok too. Last time I tested with glibc based rootfs and that's where I had the problem moreover above dmesg was also not there which is puzzling i will build a glibc based rootfs again today and retry

[pelwell]

I've been writing documentation, but I'm taking a closer look at this now.

[pelwell]

The problem stems from the fact that ARCH=arm has no driver for the A72 PMU. However, both the A15 and A7 drivers seem to work to my limited testing (running your test app). Can you save me some searching and say which you think should be the fallback?

[pelwell]

If you can, try building your own updated DTB after applying this patch:

diff --git a/arch/arm/boot/dts/bcm2838.dtsi b/arch/arm/boot/dts/bcm2838.dtsi
index 5f837e8..0f54f05 100644
--- a/arch/arm/boot/dts/bcm2838.dtsi
+++ b/arch/arm/boot/dts/bcm2838.dtsi
@@ -224,7 +224,7 @@
        };
 
        arm-pmu {
-               compatible = "arm,cortex-a72-pmu";
+               compatible = "arm,cortex-a72-pmu", "arm,cortex-a15-pmu";
                interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>,
                        <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>,
                        <GIC_SPI 18 IRQ_TYPE_LEVEL_HIGH>,

Also try with "arm,cortex-a7-pmu" instead.

[pelwell]

I've gone with the cortex-a15-pmu version, on the grounds that it works better than nothing. I'm willing to change this again if somebody comes up with a better suggestion.

[kraj]

@pelwell I am building an image to test this patch out on pi4/32bit today. Will let you know the results on my side

[lategoodbye]

I've gone with the cortex-a15-pmu version, on the grounds that it works better than nothing. I'm willing to change this again if somebody comes up with a better suggestion.

This issue has been discussed during upstreaming the DTS for the BCM2711. The proper solution would be to add arm,cortex-a72-pmu support for arm32, but the motivation of the responsible kernel developer wasn't that high :-(

[kraj]

I've gone with the cortex-a15-pmu version, on the grounds that it works better than nothing. I'm willing to change this again if somebody comes up with a better suggestion.

this patch works on my end. So perhaps we can apply it and keep it until its fixed right way upstream

[pelwell]

It's already applied, and will be ported forward to the newer kernels.

[kraj]

fantastic fixed with e99b135