Support for Touch Host Controller (Tigerlake+) #8
Comments
This is pretty much correct. With Tiger Lake, Intel deprecated the old MEI based IPTS and instead moved its functions to a standalone PCI device, the Touch Host Controller (THC). Some Lenovo device is out there using it too. Since the THC is not MEI based, I doubt this driver can be reused at all. It might be possible to reuse the iptsd daemon, if the data that the hardware outputs is the same or similar. But there is no way to be sure without having a working kernel driver. What I find interesting is that the IPTS driver probes on your device and creates its userspace interface. That either indicates that some life is left in the old interface, or that Intel adds the interface to all of their hardware (which would be annoying for upstreaming this driver). Could you post your full dmesg? |
|
I've had a quick look over the driver package for that device. There's a There's a touchpad driver with firmware (but I think that's really for the touchpad and not the touchscreen, seems to be a PixArt PCT1836 device and it does look like it's a touchpad controller similar to this one). There's also a So I'm wondering how that works exactly. There doesn't seem to be any obvious firmware around (unless that's baked into the hardware and just needs to be updated when it changes or baked into some driver). And unless the HID Event Filter thing is doing touch processing (which I kinda doubt due to the ACPI references), there doesn't seem to be any equivalent of a |
|
Also having this as a PCI device without any way of controlling touch processing (be it GuC/compute kernels or user-space processing) doesn't make much sense to me. If it'd be just another touch controller that needs some lockded-down firmware for processing, why not use I2C, SPI, USB or whatever. |
|
Well, the IPTS protocol can also transport HID reports instead of raw sensor data. I wouldn't rule it out that an OEM could do the processing in the touch controller firmware and only use IPTS for passing the data around. |
|
Yeah, but Intel themselves doing that? I'd think the NUC devices would be a showcase of sorts for them to demonstrate how things work. Maybe chipset documentation could give us a hint. |
|
There's not much more than "Touch Host Controller provides the standard SPI interface to connect to external touch ICs.". So maybe it really doesn't do processing. |
|
Oh hey, apparently there's a patent application for it: US20190102032A1.pdf. Doesn't really clear things up for me though. |
I rechecked and realized that the /dev/ipts/X devices are only there when I manually modprobe the ipts driver. The iptsd was started after that. When I start it before, I just get a "No such file or directory error". dmesg after boot: dmesg after modprobe ipts I use the linux-surface kernel build from here https://github.com/linux-surface/linux-surface/wiki/Installation-and-Setup#arch :
and have additionally installed the dkms ipts driver via the AUR. |
Ah, I didnt think about manually loading the driver. That confirms that the MEI interface is gone then. It doesnt really help you, but thanks nonetheless. |
StollD
changed the title
|
Thanks from my side for the quick response ^^ Now at least I know that I don't need to test further with this driver. |
|
The new Surface Laptop Studio quite likely also uses this interface. See archive in linux-surface/acpidumps#20 (comment) for |
|
I am getting a SP7+ for BF and I was looking into installing EndeavourOS on it, then I saw this issue. It seems the THC is going to end up in all next Surface line products. Almost all the new lineup (except the SLGO and GO3) is having it. If intel is pushing this in future hardware, does this mean that we are going to see more and more products missing touchscreen on linux ? Is Intel touchscreen related hardware widespread amongst the industry or is it confined to specific devices such as the Surface line ? The NUC laptop lines is often used to be integrated by OEM companies like Schenker (XMG, tuxedo, etc..) . Some of them are quite open-minded about linux support so I am wondering if they are talking with Intel about THC support in Linux (especially releasing a driver or technical documentation to build it) For instance the XMG Fusion 15 (https://download.schenker-tech.de/package/xmg-fusion-15-xfu15l19/) is based on the previous NUC laptop version and they ended up officially supporting it on Linux (despite the fact Intel never released a driver pack for it https://www.intel.com/content/www/us/en/products/sku/203496/intel-nuc-9-extreme-laptop-kit-lapqc71c/downloads.html) |
The Intel ones, yeah.
At the moment I don't think there are many devices apart from the Surface lineup (and maybe the Intel demo ones you mentioned) that use it. As for the future, I guess only Intel and/or device manufacturers can really answer that. In contrast to the old MEI interface that this seems to replace, the THC doesn't seem to need any processing kernels and AFAICT could be used with any ordinary SPI touchscreen controller (which seems to be the standard). As I understand it, it's just a SPI interface optimized for lower power consumption and touch usage. So the cost of adoption is a lot lower and adoption may even come with benefits, which may make this interesting to device vendors. The only reason to not choose this over other interfaces would probably be the Linux support, so if I understand this correctly there's a good chance that other manufacturers will adopt this. Some (mostly Lenovo and Dell) might not due to missing Linux support, which might incentivize Intel to actually release a driver for that in some way or another. I guess if Intel is serious about pushing this technology they will eventually provide a driver (to get manufacturers to use that interface), but combine touch and Linux and you have a very small user-base that is affected, so for them it's probably not a high priority. Regarding driver packages and Linux: That's really not how drivers work in Linux. Everything is supposed to be in the kernel, without the need for external modules (that have an unstable API which may break at any time). So if they are serious about Linux support they will make an effort of upstreaming a driver. However, keep in mind that Intel's Linux support is generally datacenter first, consumer second (or not at all). |
|
You''re right about the userbase it can be quite thin. However as other intel hardware such as Wifi/BT cards, iGPU are supported (especially because they are widespread), I keep hope... You are right about the driver package, I was mentioning the windows one because I thought it could help as a reference for future implementation/reverse engineering. The Surface BIOS handles the touchscreen input (I guess a quite basic one) so I think that driving this hardware to got at least basic touchscreen support can be attainable without huge processing in the driver/daemon. I am not a specialist about those touchscreen devices but from what I read about your analysis, do you mean the THC outputs commands that are more low-level than the old MEI ones ? |
The MEI interface was designed around a touchscreen that provides raw data which is then processed via GuC on the iGPU. So that's a lot more complex and vendors need to write/provide a firmware package that does that processing. The THC just seems to be another communication channel (at least according to their marketing slides), so vendors should be able to just plug in any existing OEM touch controller with an SPI interface. |
|
I posted a question on intel forums and got a response: TLDR: not gonna happen any time soon.... |
|
Interesting thing I noticed. |
|
@aacebedo In the "old" drivers there are essentially two modes of operation: Single-touch/simple and multi-touch. Single-touch provides just single-touch information without any pen support. Multi-touch provides a touch heatmap/proprietary data that needs to be processed either in kernel or in user-space to make sense of things and extract touch/pen information. I'm fairly certain that the new driver is just a communication channel to the touch controller, so I expect it to work in the same way. The Surface UEFI provides a driver for the single-touch mode. That's essentially active during the pre-boot environment and can be used by the UEFI menu and any bootloader, like refind. It's basically the reason why the Surface devices support touch in UEFI. Unfortunately that driver gets unloaded when we switch to the OS/kernel mode, i.e transition out of the UEFI pre-boot environment. |
|
@qzed Any idea why it unloads the driver and how to make it.. not? |
|
Things don't work that way. Think of UEFI as another (although limited) operating system next to Windows and Linux. Just like Windows drivers don't work with Linux, the UEFI drivers don't work with Linux. You could hack out the transition from pre-boot to OS mode in the kernel, but that would only ensure that the UEFI drivers stay loaded. You still can't use them (the only thing that helps you with is that now some of the hardware may be set up and configured for use, e.g. clocks and voltages). But then you have potentially two sets of drivers accessing the same hardware which in the best case leads to just instabilities. |
|
I've done a bit of reverse engineering, and currently have a semi working polling-based single-touch THC driver (tested on a Surface Pro 7+). I'm working on converting it to using interrupts, getting the multi touch data out, and figuring out some other details. Will probably put something online somewhere during the next few weeks, depending on how much free time I have. |
|
If it's spitting out HID data (which I assume) then getting multi-touch and pen data will probably only require you to send a HID feature report to the device. That's at least how the other (non GuC-processing) devices worked. So in essence, if it works with single-touch, it should also work multi-touch, no driver changes required. |
|
Interestingly the THC actually has two identical DMA receive interfaces. The second one sends HID data, so I thought maybe the first would allow access to the raw data somehow, especially since there's a function that seems to hook it up to the GuC. I spent some time poking at it, but I haven't been able to get that first interface to give me any data. And the GuC function doesn't actually seem to get invoked by the driver or any other driver that I've looked at. After I gave up on that, I saw you guys figured out the HID feature stuff in another issue, and yup, that actually works (but it still sends the multi touch data via the second DMA interface, leaving the first one unused). What I still need to do is actually get the data into userspace somehow. I think your driver uses a chardev, right? I was thinking it may be possible to make a zero copy interface via mmap-able sysfs files. |
|
AFAICR it has (according to the marketing slides I've seen) support for two separate touchscreens (this doc seems to support that). So makes sense that it has two DMA interfaces. Yeah, we're currently using a custom chardev, but in the long term I think it's probably better if we go via a hid(raw) device. Data is already sent via HID, so why not use something more or less standard? That would also make the single-level stuff work without IPTSd/user-space processing, so you'd get an automatic fall-back mode if you don't have IPTSd active/installed. Mmap could probably get latency down slightly further, but I'm not sure if it's really worth the additional hassle. |
|
Nice work! I think an issue with the mmap approach was that the MEI driver is removed every time you go into suspend, but the chardev has to stay active, because iptsd is sometimes faster than the kernel and tries to read from it before it is back online. Copying the data seemed more robust and it works just fine. Regarding the two interfaces, maybe one is strictly for commands and the other one is strictly for data? Kinda like the old IPTS has feedback and data buffers. |
|
So here's the driver: https://github.com/quo/ithc-linux And a slightly modified iptsd to go with it: https://github.com/quo/iptsd You're probably right that a zero-copy interface isn't worth it. Mapping the DMA ringbuffer into userspace isn't that difficult, but I can't really see a clean and race-free way to notify userspace of new data. So for now I just used a chardev which spits out the DMA RX data with a little header. When the device is opened/closed, the driver automatically enables/disables multitouch mode, so single touch automatically works when iptsd isn't running. I added an optional HID mode. This sends the data through the HID subsystem. It works for single touch. It creates a hidraw device that iptsd could theoretically use for multitouch, but I haven't tested that. On the one hand HID seems like the right way to do this, since the device sends HID reports. But only the single-touch data is really HID... The multitouch data doesn't really benefit from being marshalled through the HID subsystem, it just seems a little pointless (and inefficient). As for the two DMA RX interfaces, I like the theory that it could just be one for each SPI touchscreen chip. There's only a single SPI command interface and a single DMA TX interface though (but I guess one of the unknown bits could be a chip select). And the Intel driver seems to set the two interfaces up slightly differently. So I don't know. |
|
Awesome! Thanks for sharing!
Isn't the multitouch data already HID wrapped? The benefit of going full HID is that it'd be a simpler driver. Otherwise you'd have to parse HID records in the driver (or forward them via a custom interface, at which point you're more or less re-implementing hidraw), which I don't think is really necessary (we already have to process that data in user-space). That also has the benefit that the driver is more device independent. I think you can't expect non-MS devices to use the same non-HID data (on the contrary, if this driver is adapted, I think quite a lot of devices will likely use "standard" HID data). So adding custom handling code (including stuff like enabling multi-touch inside the driver) for that is more or less driver bloat for other devices. (Again, only if it's already HID-wrapped.) Sure, there is some overhead of running this through the HID subsystem. But I think there are more benefits to going the HID route than trying something custom. HID is already well tested and accepted and the kernel code is (hopefully) somewhat optimized. |
Not really. I mean, sure, they've added a report ID byte at the front, but that's it. The report descriptor just says that report ID 11 is a 3500 byte blob of vendor specific data. Not very helpful. If this were really HID, the report descriptor would describe the structure of the data in the report, so that you could at least try to handle it generically. I'm not necessarily opposed to using the HID subsystem, that's why I added the option. But Intel/MS's usage of HID just feels like a hack. |
|
Yeah, I mean that's HID. Reports really are just a report ID + report specific data (and there's a ton of custom reports out there, not necessarily input but mostly custom feature stuff). And yeah, you can see the input reports as a hack. But it's still valid HID (and it even has a spec for the "wrapper": https://usb.org/sites/default/files/hut1_3_0.pdf 16.9, although that seems to be more or less misused on MS Surface devices as the heatmap data should probably have usage I mean my main point is: The device (to me) seems like a HID communication channel. So the driver should (more or less) act like that (i.e. be a HID transport driver). Anything else (like handling MS custom heatmap data) belongs (from a design perspective) into some different driver (HID device driver) or user-space. That will likely also make it easier to upstream. |
|
Also I should mention: I don't have any issues including this as-is for now. If we go with the HID route, the IPTS driver also needs changes (which are more substantial as on the older devices it's actually not HID data). |
|
Another thing: On the AMD variants and the Surface Pro X the touch data is sent via a HID-over-SPI device. So that'd be another HID transport driver. Ideally once IPTSd understands HID, it would automatically understand that as well once we've got a driver for that. |
Interesting, I didn't notice they added a heatmap usage. Kinda funny how they barely even specced it, and then still managed to implement it differently from their own spec.
I do wonder actually how much of the processing is handled by the THC. Why call it a "Touch" controller if it's basically just an SPI-DMA bridge? It would make more sense to me if it was getting raw data over SPI, and then processing it into the single touch and multi touch HID packets we see. OTOH if the Surface Pro X has the same HID data without any sign of a THC or IPTS, that kind of disproves that idea. I'm really curious to see what data we get on non-MS hardware that uses the THC. |
That was the plan with itouch/IPTS... which essentially failed to take off. As far as I can tell (and there's really not much to base this on, e.g. this) it's just a 2x SPI controller which is marketed as "low power" and "more responsive" (whatever that means in this context, maybe that it's exclusive and not shared with some other stuff). Marketing also says something about "simultaneous pen + touch"... not sure if that just means that they have 2 interfaces. Could be that it's capable of doing some processing, but then it'd probably need some firmware. As you've said, the biggest source of info on that is probably in other devices that use it. By the way, does it look like it's designed for HID or is that also just MS stuff on top of it? If it's really just a generic SPI controller we might need to split the driver into two parts before upstreaming: One for the SPI stuff and another for piping the SPI data to an HID device... |
It has a HID command codes like GET_REPORT_DESCRIPTOR, SET_FEATURE, etc. Although the THC is possibly just passing those along to the MS hardware/firmware. But more importantly the official Intel driver treats everything as HID. So while an implementer could maybe ignore the HID stuff, and use the THC to transfer whatever non-HID data they want, they would then have to replace the Intel driver with something else to be able to use their data in Windows. |
Neat. Yeah then it makes sense to implement this as a HID transport driver. |
Fwiw, it's fairly easy to patch the MEI driver to stay active during suspend, I think that was just intel taking a shortcut there. We might eventually want that anyway to send IPTS into a special suspend mode (I looked into power saving at some point and there's some command to make the chip stop polling all the time during suspend. I never got that to work but it can get power usage during suspend down by like half a watt) |
Hi,
I know this repo is for surface devices, but the new Intel NUC M15 Laptop seems to use the same touchscreen controler. Windows recognizes it as Intel Precision Touch & Stylus or similar (currently booted on Linux but can look up the exact name if it becomes relevant). Linux
sudo lspci -vvnn:I'm on linux-surface kernel 5.10 which correctly recognizes the MEI:
However regardless of using the dkms or the builtin driver the ipstd service crashes on start:
Could it be that the Surface devices use the same controller, but connect it differently than the NUC M15? Some hints that lead me to this theory:
My Theory: While on the Surface devices the Touchscreen controler is managed by the ME (which this driver is for) und the NUC M15 the controler is directly accessed over the pci bus, which would require a completly differend driver.
I hope I'm wrong ^^.
The text was updated successfully, but these errors were encountered: