There's no LED support

[fredizzimo]

I didn't solder LEDs to the keyboard myself so I'm unable to test and implement this feature myself. It shouldn't be hard to do, using the visualization framework.

If you either want to test or implement this, please contact me.

[kejadlen]

I'd be happy to test this out. Would be interested in implementing it too, but not sure at all where to start.

[tthayer]

I'd be willing to test some things out too. Just need some guidance.

[fredizzimo]

Hi,

Sorry for not replying earlier. I have been and still am on a trip, so I haven't had time to answer and think about this.

But today I had some time in the hotel room to look more into it, It doesn't seem to be as easy as I first thought, but it shouldn't be too hard either. I think the design that makes most sense would be to expose the LEDs as gray scale display that can be controlled the same way as LCD. Each pixel would represent a led, and the color would represent the brightness.

The advantage of doing it this way, rather than something custom would be that then you could use the drawing routines of uGFX, including font rendering. Another advantage is that the same abstraction would work for keyboards with RGB leds as well.

In order to do this, a new uGFX display driver would need to be made, just like the LCD driver (in the drivers folder of the main repository). This driver would have to use the I2C functionality of the Chibios to communicate with the LCD controller according to the specs. http://www.issi.com/WW/pdf/31FL3731C.pdf.

Some tweaking of the makefiles and the chconf is needed, to enable the build of the display, and to enable multi-display support (which works slightly differently than for using just a single display) for uGFX.

A boardfile would also need to be made (also in the drivers folder), that enables the right input and output modes for the pins, to support the I2C communication

Then this would need to be integrated into the visualizer library like the LCD.

I think I can get something for you to test quite soon after I'm back. But I would of course be very happy if the instructions here are enough to get you started in actually implementing itself.

[fredizzimo]

Hi @kejadlen, @tthayer.

I have now implemented the first version of LED support. But I have no idea if it actually works. The only thing I know is that it doesn't crash, and the rest seems to work OK.

So it would be great if you could test it out.

I have submitted my changes in the led branch, so to take it into use you have to do this

git fetch origin
git checkout -b led origin/led
git submodule sync --recursive
git submodule update --init --recursive

Then make and flash both halves. When the keyboard starts you should see a looping test. It would be great if you could take a video or something, so I can see that it works properly, but it's ok if you just verify it yourselves too.

The test should go like this

1. Fade in all LEDs to full brighness in 1 second
2. Show all leds for 1 seond
3. Fade out all LEDs so that they are completely shut down in 1 second
4. Start to fade in a gradient pattern, the outside should be the brightest, and the middle the lightest, in one second
5. Move the pattern from outside to inside for 3 seconds
6. Crossfade the pattern to another one that goes from top to bottom in one second
7. Move the pattern from top to bottom in 1 second
8. Crossfade to a pattern that goes from inside to outside in 1 second
9. Move the pattern from inside to outside for 3 seconds
10. Crossfade to a pattern that goes from to to bottom in 1 second
11. Move the pattern from to to bottom for 3 seconds
12. Fade out all leds in 1 second
13. Start over

You should report anything that isn't as you think it should be. Especially if some or all LEDs are not working, or if the patterns look wrong.

The pattern implementation is in tmk_visualizer/led_test.c
And the actual low level communication mostly in drivers/gdisp/IS31FL3731C

[tthayer]

Nothing is happening with the LEDs. The display backlight on the right-hand half cuts out after a minute and doesn't come back on until the keyboard is replugged in.

[fredizzimo]

Thanks for the report. Guess I have to re-check all the code.

But the fact that the backlight cuts off is very strange. It doesn't happen on mine with no leds installed.

Does the keyboard still work if the backlight is off? And does the LCD go blank too?

[tthayer]

Thanks for working on this! The keyboard continues to function just fine and the LCD continues to cycle through the layers display. 

Sent from Outlook for iPhone

_____________________________

From: fredizzimo [email protected]
Sent: Sunday, April 24, 2016 12:32
Subject: Re: [fredizzimo/infinity_ergodox] There's no LED support (#5)
To: fredizzimo/infinity_ergodox [email protected]
Cc: Tony Thayer [email protected], Mention [email protected]

Thanks for the report. Guess I have to re-check all the code.

But the fact that the backlight cuts off is very strange. It doesn't happen on mine with no leds installed.

Does the keyboard still work if the backlight is off? And does the LCD go blank too?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

[dervos]

Perhaps as a 'hello world' test you could hook one led port up to a volt meter. It would allow for easier initial testing and you'd be able to verify the test results of others.

[fredizzimo]

@tthayer, I found an issue, there was an off by one error in the code, which had been left by a refactoring I did.

The code was taking care of setting which LEDs are enabled to the hardware. The controller has two blocks, and only one of them is mapped to the keyboard LEDs. And of course the byte layout for those are interleaved, which means that no LEDs would get mapped. So that would explain why nothing shows up.

But this does not explain why the LCD backlight stops working. Although both are LEDs they have nothing to do with each other, and controlled by entirely different controllers. A memory corruption of some sort could be possible, but I doubt it, especially since most values should be re-written when you switch layers. I also think that this would happen on my board without LEDs as they should not affect the software at all.

Some kind of USB power issue could also be possible, if it detects that there's not enough power and shuts the LEDs off. But that also sounds very unlikely since the LEDs were not working and they shouldn't draw any power.

So for now, the most likely cause sounds like there might have been some kind of inference because of the incorrect mapping. So I suggest that you try my fix out.

@dervos Thanks for the suggestion, I guess a multimeter would show at least something, even if the LEDs are PWM controlled. I will try it, if this fix isn't enough, and there are other bugs. I don't want to open the keyboard if it's not necessary, as the acryllic is very hard to align right, at least on my keyboard. I have the first version, where they didn't fix the space between the thumb section, so I have less than one mm of space there.

[tthayer]

@fredizzimo The LEDs still don't power on but the right-hand LCD backlight stays lit now.

[fredizzimo]

@tthayer, thanks for the report, although it's completely the opposite from what I expected.

I will continue reviewing my own code, and test with a mult-meter before I ask you again.

[tthayer]

@fredizzimo No problem. I wish I was able to provide better feedback!

[fredizzimo]

@tthayer, I just pushed a new version, that forces all the LEds on directly from the driver, bypassing all visualization logic.

I get some readings when testing with the multimeter, but it varies from LED to led for some reason. Maybe you could test this to see if some of them light up?

[kejadlen]

@fredizzimo Unfortunately, I'm away from my Infinity until Thursday, but looking forward to helping out with testing and debugging once I get back.

[tthayer]

@fredizzimo That worked! They're all lit now. The right side now feels unresponsive and kind of slow though. Keys on the right side are also 'sticking'.

[fredizzimo]

@tthayer, Great. Do they all have the same brightness?

The unresponsiveness probably comes from the fact that the animation is still running in the background, and the same problem actually happens for the LCD screen too. I will have to profile and optimize some things.

@kejadlen, Don't worry, I have the help I need for now.

I will check if I can find the reason why it doesn't work through the visualizer. Probably not for too long though, as I have to go to bed quite soon.

[tthayer]

Yes, it appears they all have the same brightness.

[fredizzimo]

Hi @tthayer, @kejadlen.

I have fixed a couple of issues, and I think that the LED animation should be working now. Could you test it, please?

The brightness probably don't change as linearly as it should since I didn't apply any gamma calculations, as described in http://www.issi.com/WW/pdf/31FL3731C.pdf, read the "gamma correction" section. I'm not sure if I should apply that correction on the user side in the visualization code or, as a helper function that can be called, on the driver side, done automatically. Doing it on the user side would be more flexible, but I'm not sure if that kind of flexibility is needed. What do you think?

[kwkroll32]

@fredizzimo Do you still need help testing this? I just flashed the LED branch and it seems to be running the light test loop.

[fredizzimo]

@kwkroll32, thanks. Yes, @tthayer and @kejadlen haven't had time to test the latest version yet.

It would be nice if you could verify that everything is done in the correct order, like I described above. And that all LEDs light up exactly as described.

I'm also interested in the linearity of the brightness, since there's no gamma correction as described above. How bad is it?

Once everything is confirmed working, I guess I will disable the demo loop, and maybe replace it with a fixed brightness settings, or even all LEDs off. You should then be able to customize it the same way as the LCD screen. I will also merge it to the master at that point.

[tthayer]

I'll try it out this weekend.

[kwkroll32]

I uploaded a short video of the LED behavior. Hopefully you can tolerate the poor quality.
https://youtu.be/-M3CBAGVVEY

[fredizzimo]

@kwkroll32, thanks for the video.

It looks like gamma correction is definitely needed. And there also seems to be some bugs, for example at around 0:08, it looks like half of the keyboard stays fully lit, while the other half is off.

I will look at it maybe later today or tomorrow, but if some of you want to have a look too, the bug should be somewhere in the tmk_visualizer/led_test.c file.

[kejadlen]

@fredizzimo Having trouble updating ugfx:

fatal: reference is not a tree: e221a690616e20f87e0b0088baffdbd5427be862
Unable to checkout 'e221a690616e20f87e0b0088baffdbd5427be862' in submodule path 'tmk_visualizer/ugfx'
Failed to recurse into submodule path 'tmk_visualizer'

Edit: Nevermind, I figured out the problem - git was still using Tectu instead of fredizzimo for the remote.

[bremoran]

Once my LEDs arrive, I'm happy to help out with testing this feature out.

[fredizzimo]

Hi All,

I have hopefully fixed the gradient code. I don't know what I was thinking when I wrote the original function, it seemed like I had only done it halfway.

The effect should be much better now, with this fix applied. But to make sure it would be nice if someone could test it.

I still think the gamma correction is needed though, so I will implement that next. It will be done inside the driver so it should be completely transparent to the user.

[kwkroll32]

That looks a lot better; new video below. I had a hard time discerning what it was supposed to do last time. The only thing I'm not sure about is whether the horizontal crossfade is supposed to start in the middle column. You can tell that it starts one away from the middle column in both the left-->right and the right-->left fade.
https://youtu.be/lGJzUa6dBUE

[fredizzimo]

Thanks, that really confirms that brighness correction is needed.

I also think that there might be some problems with the top row, it doesn't look like it's having the same brightness as the other rows from the video.

Also the thumb cluster, seems to be acting strangely, the outer keys seems to be quite bright. And it gradient is supposed to go down below the main keyboard rows to the two thumb buttons, because I represent the keyboard as a 7x7 bitmap, where the two upper thumb keys are part of the bottom main keyboard row.

I thought that this gradient effect would be enough to diagnose that all LED works individually correctly, and it might be, when the brightness correction works. But I still think I have to add one step that lights up each led, one by one, for easier diagnosing.

[kwkroll32]

I've been reading the code and documentation to figure out how this works [having written lots of Python, but never working with C or hardware controllers]. One thing I noticed is that commenting out portions of the animation test in led_test.c can break the entire animation altogether. Do you know what that's about? I also commented out the corresponding lines from led_test.h, and reduced the number of frames to match the number of frame_functions.

In other words, how might I limit your animation test to [keyframe_fade_in_all_leds, keyframe_no_operation, keyframe_fade_out_all_leds]? I think that having these 3 as a sanity check, plus 1 complicated test like the crossfade, might make it easier for me to troubleshoot.

[ec1oud]

So the head of the led branch is now

afd5be1 Update visualizer

With that one, I don't see any LED animation, they are staying off. But I do see the LCD image toggling back and forth erratically. Sometimes much faster than usual.

[fredizzimo]

Hm, that's strange. I must have broken something when doing the emulator stuff.

I will check it, after I have made blitting support for this #11

[ec1oud]

But https://youtu.be/ocLzjHaLDtg is from ba6b411, seems OK.

Git bisect says afd5be1 is the first bad commit. Which doesn't make sense to me.

[fredizzimo]

Some quick update.

I have not completely found out what's wrong. But I found a few problems.

1. The erratic LCD image toggling is caused by the system somehow waiting for much longer than it should. It happens at the start of the LED animation cycle, where the LEDs are supposed to stay on for 1 second. At this time there's no fading going on, so the system can wait. It's supposed to wait for around 0.5 seconds (verified by prints), but it actually waits for 5 - 20 seconds. This also causes the following animations to toggle too fast, since the animation tries to catch up. This behavior is probably caused by the usage of the ugfx api fredizzimo/tmk_visualizer@d79e94a.
2. It seems to send the right values to the LEDs, so I don't undertand if the LEDs don't turn on at all. It also seems to be completely unrelated to the changes.
3. Each update step is taking much longer than supposed, and it's not related to the cos functions. Calculating those takes less than 1 ms. But the whole update, probably caused by either the LED or LCD transfer to the actual hardware takes over 200ms, which means that we can only get 5 FPS for the animations, so that's very bad. I need to check this further.

The difference between f7cedcd, which is working(?), and afd5be1, is the update of the tmk_visualizer submodule. They both actually points to the same commit message, but I amended a commit together, so they are actualy two changes. The only difference is that the following line in led_test.c was changed from

float normalized_index = 1.0f - (index / (num - 1) * two_pi);

to

 float normalized_index = (1.0f - index / (num - 1)) * two_pi;

Are you sure you ran git submodule update --recursive for each bisect test? Otherwise the tmk_visualizer submodule might not have been at the correct change.

Edit:
I think the animation now looks otherwise correct, expect that the bug that was in the amended commit bug is there. This causes the sine gradients to start from around the second key, rather than the first one.

[ec1oud]

I was doing the bisect build/test steps like this

git submodule sync --recursive; git submodule update --init --recursive; git clean -dfx; make && sudo make program

[fredizzimo]

Hi @ec1oud,

I have now fixed the problem, at least with the erratic LCD image toggling. There was a bug in either the ugfx library documentation or implementation, which says that the function takes a system tick parameter, but in reality it should be milliseconds. I will report that bug to them at some point.

Also due to an integer overflow caused by this, the behavior was not consistent, which could explain why it appeared to be working for some commits. BTW, I can see the problem in the video you posted. At the end of it, the layer bitmap stays on for around 5 seconds, and after that it flickers a bit.

I think that the missing LED animation could have been caused by the same issue, but I'm not sure.

[r2d2rogers]

I'm eager to try this, would it still be useful to have reports from the LED branch here, or should I wait for the QMK side?

[fredizzimo]

Thanks @r2d2rogers, yes it's still useful to have it tested here. The code that I have done here will be almost directly ported to QMK, so it would be nice confirm that everything works before that.

[r2d2rogers]

I was able to flash and see the led pattern progression. I'll get setup to post a YouTube video tonight so you can see if it's doing exactly what you expect.

[r2d2rogers]

Here is the video I promised. https://youtu.be/GXt1RlNkzQo

[fredizzimo]

Hi, @r2d2rogers

Thanks for the video. Unfortunately it showed the same problem as @ec1oud saw, and that I already fixed. So I checked the history, and it turned out that I had forgot to push the LED branch with the fixes to Github, I had only pushed the fixes in the visualizer submodule.

I just pushed it, so now it should hopefully work better.

[r2d2rogers]

Another video with the latest changes, let me know when you'd like me to use the qmk version.
-- Link removed, I think I missed the submodule update, reflashing and will post a new video.

[r2d2rogers]

Now I don't get key led's lit at all, but the LCD responds much faster for layer changes.

[fredizzimo]

Hm, that's very strange. I have to check what the problem could be.

If you want you can also try QMK, the changes were just merged, and the default layout should have the LEDs enabled.

To flash the keyboard you need to use make dfu-util

[fredizzimo]

@r2d2rogers, Have you tried re-plugging the keyboard a few times? Maybe the LEDs come only some of the times.

That's just guessing, but I really have no idea what's going on at the moment. And it also seems like the issue has been coming and going, with different versions. You can see that if your read the whole discussion in the issue.

I have never found a definite answer to the problem. But the fact that it seems to come and go, and stays constant over versions seems to indicate a memory corruption(unless you prove that theory wrong, by finding that it works some times and other times not) .

[r2d2rogers]

The LEDs haven't lit on multiple plug in attempts. I have tried direct to the PC, through the powered hub that has been working great so far, and one hand at a time. I have the QMK checked out and built on a linux machine, but I am on my Windows 10 workstation at work. I'm looking at the windows flash instructions for QMK (hex vs bin files) to attempt that next.

[fredizzimo]

@r2d2rogers,

If you run make dfu-util, it will automatically convert the elf to a bin file and flash it. If you can't flash on that machine, you can copy the generated bin file, and flash that on another machine.

[r2d2rogers]

Thank you, that's just what I needed to know.

It works. After I realized I forgot the master flag for which hand I plug into the computer. All LEDs working on QMK, with the fixed behavior of the LCD.

Looks like it's time to move over to the QMK for further issues.

[fredizzimo]

Great, although it doesn't really make sense that it works there, almost the same code should be used in both, which means that there's more debugging to be done...

Does the animation look right now?

And yes, the QMK port is starting to get ready. It already has basically all the features of this repository. The only thing missing are the unit tests, and the visualizing emulator.

I'm currently merging it with the EZ ergodox, so that all their keymaps can be used. And I also think I will make some improvements to the visualizer, before I will start recommending people to use that.

[r2d2rogers]

Actually, the LED animation isn't right, it's more of a fade in and out as a whole, not the ripple pattern.

Would me building with the MASTER=right flag affect the LED animation?

[fredizzimo]

I don't think it would.

I will soon have to go to bed, but tomorrow I will record a video of how it's supposed to look like. I can do that through the emulator, which renders everything on the PC. The emulator is a bit tricky to build though at the moment, so it's probably easier for me to make a video.

[r2d2rogers]

I pulled the most up to date version of the LED branch here and if I specify MASTER=right the LEDs in the keys do not light up. If I set it to left or don't specify anything (default left) they work correctly with good response from the layer display on the LCD when changing layers.

I will try right versus left on QMK and see what that does, but it will be later today.

[fredizzimo]

@r2d2rogers, thanks for the report. It will most likely be the same in QMK. It's probably an easy fix though.

And sorry, I completely forgot about the video, and can't do that until the weekend at the earliest. I'm at another computer right now, and I don't want to set it up for that.

[r2d2rogers]

I tried changing the submodules on the QMK to match what you are using here, but I still don't get more than the fade up and down on the LEDs on qmk, no rippling from edge to edge.

[fredizzimo]

I will check what's going on during the weekend.

[r2d2rogers]

Looks like we have the visualizer and LED stuff out of the loop on the latest versions of the QMK stuff. Is there anything I can poke at on getting the test stuff running with the latest sub-project modes? I now have two Infinity Ergodox sets at my disposal, so interating versions of the lighting while checking stability of the code is easy for me.

Currently I still have one set running the led branch here, and the other is on QMK with the subproject code for infinity ergodox.

[fredizzimo]

Hi @r2d2rogers,

Sorry for not answering earlier. Unfortunately I have been busy with qmk/qmk_firmware#591, but after that I will return to fixing the Visualizer.

The development speed is not that fast at the moment though, since I mostly have time only during the weekends, and maybe a few hours here and there during weekdays.