Investigate false positive for Spectre/Meltdown on AWS

[vpetersson]

During a customer demo, we got a false positive for Spectre/Meltdown an AWS. The system was running Ubuntu 18.04 with the latest patches installed (i.e. no kernel upgrades available).

Possibly related to WoTTsecurity/agent#252.

[a-martynovich]

@vpetersson Was Ubuntu running in AWS container? Do you still have this container?

[vpetersson]

I don't have access to the VM as it was a customers. But yes, it was just a vanilla VM on AWS, so should be easy to reproduce.

[a-martynovich]

I don’t have AWS plan so if you could give me a VM to play around I could reproduce it

[vpetersson]

@a-martynovich Yes I will provide you with a VM when we need it, but it's not in the current sprint.

[a-martynovich]

@vpetersson it is in the current sprint, so I need the VM now

[a-martynovich]

One more thing: just like with Spectre (mis)detection inside VirtualBox, we have no way to know if the host system is actually protected or not. If the hypervisor makes the kernel believe it can't enable all of its protections, then the whole system may be vulnerable in case if the host actually has some protections disabled. Which, as I said, we can't detect.

[a-martynovich]

@vpetersson AWS system reports it's vulnerable to MDS and spec_store_bypass:

[ec2-user@ip-10-0-0-73 ~]$ grep '' /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:KVM: Vulnerable
/sys/devices/system/cpu/vulnerabilities/l1tf:Mitigation: PTE Inversion
/sys/devices/system/cpu/vulnerabilities/mds:Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: usercopy/swapgs barriers and __user pointer sanitization
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, STIBP: disabled, RSB filling
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected
[ec2-user@ip-10-0-0-73 ~]$ 

Also there is this 2020 presentation which proves my point

Here's the video recording.

[vpetersson]

I see your point. However, the things with RAs is that if you cannot take an action on them, they are useless. Hence, if we display "Vulnerable to X" but there is nothing the user can do to resolve it, it is merely an annoyance to the users.

[a-martynovich]

@vpetersson Then we shouldn't scan for those vulnerabilities on AWS?

[vpetersson]

If there is no mitigation strategy available, we should hide the RA.

[a-martynovich]

The mitigations mentioned in the report I’ve given are:

• disable SMT by adding nosmt flag to kernel cmdline
• disable hyperthreading (this is probably possible by choosing another instance type)

[vpetersson]

Cool. I didn't have time to read the full report. Disabling SMT will have a massive impact on the performance. Thus I don't think it's not really a (realistic) option. Disabling HT by switching to another instance type is not realistically.

Rule of thumb for RAs: They should not massively impact performance and they should be possible to apply using a single wott-agent patch command.

[vpetersson]

@vpetersson Then we shouldn't scan for those vulnerabilities on AWS?

Yeah let's hide it from AWS for now.

[a-martynovich]

Depends on WoTTsecurity/agent#260