Raspberry and I2C LCD1602 with PCF8574T controller does not show text but only squares

[ciarlystreet]

Hi,
I'm probably doing something wrong, this is the connection scheme:

and this is the code:

const { RaspiIO } = require("raspi-io");
const five = require("johnny-five");

const board = new five.Board({
  io: new RaspiIO()
});

board.on("ready", function() {
  // I2C LCD, PCF8574
  const lcd = new five.LCD({ 
  controller: "PCF8574T"
});

  lcd.print("Hello");
});

but in the LCD I see only:

Can you help me understand if I'm wrong or if there is something wrong with the LCD?

Any advice is welcome 🙌

[ciarlystreet]

I don't know if this can help solve the problem, I just tried this tutorial to see if the LCD works and to make it work I had to replace I2C_ADDR = 0x3f with I2C_ADDR = 0x27 done the screen shows the writing but if I try the script nodejs continues to show nothing now not even the squares anymore.

[ciarlystreet]

I tried this script #1314 (comment) but it still doesn't work

const { RaspiIO } = require("raspi-io");
const five = require("johnny-five");

const board = new five.Board({
  io: new RaspiIO({
    enableI2C: true
  })
});

board.on("ready", function() {
  var lcd = new five.LCD({
    controller: "PCF8574T"
  });

  lcd.cursor(0, 0).print("10".repeat(8));
  lcd.cursor(1, 0).print("01".repeat(8));

  var isOn = true;

  setInterval(() => {
    isOn = !isOn;

    if (isOn) {
      lcd.on().backlight();
    } else {
      lcd.off().noBacklight();
    }
  }, 2000);
});

[fivdi]

Hi @ciarlystreet, after a 1602 LCD is powered on, the top row will display solid squares and the bottom row will be blank. This is the expected behaviour and indicates the LCD is being powered correctly.

The programs shown above also look like they should work.

What is the complete output of the following command when run from the command line on the Raspberry Pi?

i2cdetect -y -r 1

If i2cdetect shows that the LCD is at address 0x27 this is correct and what Johnny-Five expects.

When the Johnny-Five program is running what output does it display? Are any error messages displayed?

A lot of work was done on raspi-io recently and there are currently issues with I2C that are being resolved with this PR. The issue that you are seeing may be related to the current I2C issues in raspi-io.

It may be possible to workaround the issue by directly editing the file node_modules/johnny-five/lib/lcd.js on the Raspberry Pi and replacing this line of code:

  this.io.i2cWrite(this.address, this.memory);

with this:

  this.io.i2cWrite(this.address, [this.memory]);

If it still doesn't work after modifying node_modules/johnny-five/lib/lcd.js, you could try using pi-io rather than raspi-io until the I2C issues in raspio-io have been resolved.

[ciarlystreet]

Hi @fivdi,
I answer you questions:

• What is the complete output of the following command when run from the command line on the Raspberry Pi?

i2cdetect -y -r 1

This is the output:

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- 27 -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- -- 

• When the Johnny-Five program is running what output does it display? Are any error messages displayed?
  No no error is shown, it is shown only:

1571638531696 Available Raspi IO  
1571638531705 Connected Raspi IO  
1571638531715 Repl Initialized  

• Editing the file node_modules/johnny-five/lib/lcd.js
  Instead of showing "Hello" it shows:
  

• Using pi-io works properly, but does not print special characters created by the useChar(charCode|name) function.

Thanks 🙌

[fivdi]

@ciarlystreet I think the recent work on raspi-io may result in timing issues that were not there in the past so I created this issue. These timing issues are likely to be the cause of the issues that you are seeing with raspi-io.

I don't have an I2C LCD so I'll get one to see if I can figure out what the problem is with pi-io. beaglebone-io is likely to have the same issue.

[fivdi]

@nebrius ^^

[nebrius]

I realized what's happening and did a writeup at nebrius/j5-io#12 (comment).

tl;dr the sleep method called at https://github.com/rwaldron/johnny-five/blob/v1.3.1/lib/lcd.js#L422 and defined at https://github.com/rwaldron/johnny-five/blob/v1.3.1/lib/lcd.js#L13-L31 uses a while loop, which blocks the Node.js event loop and prevents the I2C writes from being completed.

To fix this, we'll need to rewrite this method to use a mechanism that does not block the event loop, probably with an async function and a new sleep method that looks something like:

async function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

and then we can await this method in the other calls. I have two questions though, and I'd like to hear what @dtex and @rwaldron think:

1. We would be bumping up the required version of Node.js to v8.x.x most likely, according to https://node.green/. I'm hugely in favor of this, as v8.x.x has already been end-of-lifed, and I don't think we should be supporting even older versions.
2. This blocking method will no longer be truly blocking, as the event loop will now be unblocked and available for other code (such as a web server, etc.) to do there thing here instead of later. This is a good thing IMO from a usability perspective, but there is a noticeable chance of regressions.

What do you all think?

[fivdi]

@ciarlystreet I had forgotten that cursor needs to be invoked after invoking useChar. See here.

The following should work:

const five = require('johnny-five');
const PiIO = require('pi-io');

const board = new five.Board({
  io: new PiIO()
});

board.on("ready", () => {
  const lcd = new five.LCD({
    controller: "PCF8574T"
  });

  lcd.useChar("heart");
  lcd.cursor(0, 0);
  lcd.print("Hello :heart:");
});

[dtex]

@nebrius

From the source 🤣

/**
 * This atrocity is unfortunately necessary.
 * If any other approach can be found, patches
 * will gratefully be accepted.
 */

Sleep uses microseconds or milliseconds so the solution will be a little trickier than just wrapping setTimeout. Maybe use setTimeout for ms calls, but for us calls we will need to check the time on every pass through the event loop. This has the side effect of pegging the CPU, but hopefully these are 5 - 50 microsecond bursts of intensity and are unlikely to set anything on fire.

1. We've already told the world we officially support LTS versions (Active and Maintenance). We should add a note to that effect in the Readme.

2. I don't think you have much to worry about from a regression standpoint since things that really do depend on order are all handled by callbacks except for those few instances which use the sleep method in exactly this manner. It's only used in Expander, LCD and in the Brat example file.

p.s. There are some supported platforms that don't make it through the event loop anywhere near fast enough for us to have sub-millisecond delays. On old pi's for example, we may end up waiting 20ms for a 5us call to complete. We need to check the device docs and make sure that's not a problem if, for example, we are needing to rapidly toggle a pin.

[ciarlystreet]

@ciarlystreet I had forgotten that cursor needs to be invoked after invoking useChar. See here.

The following should work:

const five = require('johnny-five');
const PiIO = require('pi-io');

const board = new five.Board({
  io: new PiIO()
});

board.on("ready", () => {
  const lcd = new five.LCD({
    controller: "PCF8574T"
  });

  lcd.useChar("heart");
  lcd.cursor(0, 0);
  lcd.print("Hello :heart:");
});

Thanks @fivdi 🙌

[nebrius]

 * This atrocity is unfortunately necessary.
 * If any other approach can be found, patches
 * will gratefully be accepted.

🤣 😭

Hmm...what about something like this?

async function sleepus(duration) {
  const startTime = process.hrtime();
  let deltaTime;
  let usWaited = 0;
  return new Promise((resolve) => {
    function tick() {
      if (usDelay > usWaited) {
        deltaTime = process.hrtime(startTime);
        usWaited = (deltaTime[0] * 1E9 + deltaTime[1]) / 1000;
        process.nextTick(tick) // IIRC, this runs as fast as the event loop runs, and can be sub millisecond in some cases
      } else {
        resolve();
      }
    }
  });
}

EDIT: when I get some free time, I'll start experimenting.

[nebrius]

OK, I tested and that approach is...not great. After bug fixes, I could only get to a highly-variable minimum delay time of 500-1000us.

So what about this: We updated the IO plugin API so that all of these I2C read/write methods return a Promise. In J5, we could then await these method calls before we call sleep, which would restore the existing order of operations on the RPi, while also making the timing stricter (currently it's actually possible for the delay to be less than the time called, since we start the sleep method before the bits have been written to the wire).

I'm not familiar with this part of the codebase, so do you see any gotcha's with this approach @dtex?

EDIT: FYI I just tested and await undefined acts as a NO-OP and just continues on, so it's backwards compatible with existing IO plugins that don't update their signatures.

[fivdi]

I'm not familiar with this part of the codebase, so do you see any gotcha's with this approach @dtex?

My apologies for intruding on the conversation here.

One gotcha is that await is only valid in async functions and using await outside an async function will result in a SyntaxError at runtime. If await is used to wait for a sleep operation to complete or to wait for a Promise returned by an I2C read/write operation to resolve/reject, then the function using await will have to become an async function. This would change the semantics of several LCD methods, including methods used by J5 users. For example, clear is a synchronous method today. If clear directly or indirectly invokes methods that use await, then clear would also have to be asynchronous.

To be honest, I think this is a difficult problem to solve and it goes beyond what we have looked at so far, for example, I have just realized that if the below code (which is unrelated to LCDs) is run using raspi-io it is highly likely that the digitalWrite will be performed and have completed before the two i2cWrites have started.

this.io.i2cWrite(address, [0x00, 0]);
this.io.i2cWrite(address, [0x01, 0]);
this.io.digitalWrite(pin, this.io.LOW);

On the other hand, the IO Plugins Specification specifies that data writing operations must be executed in order of instruction.

[nebrius]

Good points, hmmm...

I'm thinking now that I should switch from using manager specific queues (one for Serial, a different one for I2C, etc) to using a single global queue. This will fix the ordering issue.

Oh!! I could create a bigger boundary for this using worker threads. Each method exposed by J5-IO just puts a queue message into a queue running in the background thread (or separate process even), so that blocking the event loop won't stop Raspi IO from working properly. Honestly this makes it more firmata like anyways.

[fivdi]

I would expect both of these proposals to result in more timing and synchronous/asynchronous issues.

KISS and all will be good.

[fivdi]

In this case KISS means Keep it synchronous, stupid 😄

[nebrius]

Do keep in mind that every firmata operation is asynchronous. That some Raspi IO operations are/were synchronous is actually a deviation from the Arduino and other firmata based systems, even though the API would suggest otherwise.

It’ll be a chunk of work to convert to threads, but it will conceptually make the Raspberry Pi more in line with the other platforms, not less.

[fivdi]

Sure, I'll keep that in mind. It's also important to keep in mind that some J5 operations are orders of magnitude faster on a Raspberry Pi than on Notebook connected to an Arduino Uno via a serial port running at 57600 baud which is the default baud rate if I remember correctly. This results in very different timing characteristics.

But rather than speculating about what may or may not be the case I'd prefer if we took a look at a specific piece of Johnny-Five code. The below code reads the temperature and humidity from a SHT31D sensor.

        var readCycle = function() {
          // Page 10, Table 8
          // Send high repeatability measurement command
          io.i2cWrite(address, [
            this.REGISTER.MEASURE_HIGH_REPEATABILITY >> 8,
            this.REGISTER.MEASURE_HIGH_REPEATABILITY & 0xFF,
          ]);

          setTimeout(function() {
            io.i2cReadOnce(address, READLENGTH, function(data) {
              computed.temperature = uint16(data[0], data[1]);
              computed.humidity = uint16(data[3], data[4]);
              this.emit("data", computed);
              readCycle();
            }.bind(this));
          }.bind(this), 16);
        }.bind(this);

If you switch to an implementation based on worker threads, how will you guarantee that the i2cWrite has completed sending the required command to the SHT31D before the setTimeout starts?

[nebrius]

If you switch to an implementation based on worker threads, how will you guarantee that the i2cWrite has completed sending the required command to the SHT31D before the setTimeout starts?

There's no guarantee of that on any platform, so I don't think it matters. Indeed, it almost certainly will not have been sent to the SHT31D on an Arduino before setTimeout has been called

[fivdi]

Indeed, it almost certainly will not have been sent to the SHT31D on an Arduino before setTimeout has been called

As mentioned above I'd prefer not to speculate.

There's no guarantee of that on any platform, so I don't think it matters.

Fair enough 😄, then let me rephrase the question and ask you why is it likely to work?

[nebrius]

As mentioned above I'd prefer not to speculate.

OK, I'll rephrase: I guarantee that it won't have happened. For the value to have been sent to the sensor, these things have to happen:

1. We wind through J5 to eventually end up at node-serialport, calling write. This is a Node.js stream operation, which is not synchronous, as per https://nodejs.org/api/stream.html#stream_writable_write_chunk_encoding_callback, and we are not making use of the write completion callback.
2. The message gets put into a stream buffer within Node.js that will eventually get written to the fs device (once the event loop gets around to it).
3. Now that the message is the kernel driver buffer for the serial port, it will get moved to various hardware buffers. Which buffers and how many are dependent on the system the code is running on, but at minimum there will be a transmit/receive buffer for the USB controller hardware.
4. The message is sent out on the USB wire
5. The USB receive hardware buffer on the FTDI chip on the Arduino is filled with the message
6. The FTDI chip converts it to a serial message and puts it in the UART transmit hardware buffer
7. The serial message is sent on the UART board traces (on the wire) to the ATMEGA CPU
8. The message is put in the UART receive buffer in the ATEMGA CPU
9. A CPU interrupt fires that pulls the message out of the UART receive hardware buffer and makes it's way into Firmata on the Arduino
10. Firmata converts the message and sends it to the I2C transmit hardware buffer on the Arduino
11. The message goes out on the I2C wire
12. The I2C receive hardware buffer on the SHT31D is filled with the message
13. The processor on the SHT31D pulls the message out of the buffer and sticks it into the appropriate CPU register, and the device does it's thing.

That is, at minimum, 13 asynchronous steps that occur after i2cWrite is called. The setTimeout method will be called after step 1 or 2 is finished.

Fair enough 😄, then let me rephrase the question and ask you why is it likely to work?

Because firmata itself is a message passing protocol that sends messages from one thread (the main Node.js thread) to another "thread" (firmata running on the Arduino) over a serialized communication mechanism. The transmission protocol is different, as there are a lot fewer steps on the RPi than the Arduino (as outlined in the steps above) because there were fewer systems in play, but we're closer to simulating this because all operations are serialized and sent over a (virtual) wire.

This approach normalizes all operations and enforces an order of operations (because they sit in a receive queue), which was missing before.

[fivdi]

That is, at minimum, 12 asynchronous steps that occur after i2cWrite is called. The setTimeout method will be called after step 1 or 2 is finished.

Yes, this is what will happen, setTimeout starts waiting after step 1 or 2, and after the 16 milliseconds i2cRead will go through even more steps as it needs to send information back and everything automagically works. It's quite fascinating actually.

I guess, I'll have to wait and see how the threads based implementation goes on the Pi.

Good Luck 🍀

[nebrius]

I hit an architectural brick wall with the threads approach...super tl;dr another architectural decision I made (and don't regret and want to keep) makes using threads impossible, since they don't have true shared memory.

I'm going to dig into the J5 code and see if there's a way to get rid of the blocking code entirely in it.

[nebrius]

OK, I've done some digging and gotten a greater understanding of how sleep is used and where...so it can't be converted to async because it's used in the LCD constructor, and constructors can't be async.

I think the best approach now is to convert the LCD class to use a command queue system like Raspi IO uses under the hood. This will remove the need for sleep calls entirely (which would be replaced with a local sleep command in the queue) and the whole thing can become async and unblock the event loop.

I'll start working on that today

[nebrius]

throws in towel

Looks like updating the LCD class isn't straightfoward, and would require a pretty hefty rewrite. I give, I rewrote j5-io to use synchronous I2C writes. CPU usage increased by about 5-10% in my integration test app (I noticed it while reading from an MCP9808 sensor and writing to an SSD1306 LCD), but better for it to work than be fast.

@ciarlystreet can you try running 'npm update' and test again to see if it works?

[fivdi]

@ciarlystreet if it's still not working after updating please manually modify lcd.js as suggested in #1639 (comment) to see if it then works.

@nebrius if J5 calls i2cWrite with two numbers like this:

  this.io.i2cWrite(this.address, this.memory);

it looks like i2cWrite in j5-io will ignore it or do you see this differently?

[nebrius]

it looks like i2cWrite in j5-io will ignore it or do you see this differently?

Hmmm, yeah that does appear to be the case. This signature isn't documented though in the IO Plugin spec, and AFAIK Raspi IO never supported it.

[dtex]

Related firmata/firmata.js#198

[ciarlystreet]

@nebrius @fivdi I tried this script:

const { RaspiIO } = require("raspi-io");
const five = require("johnny-five");

const board = new five.Board({
  io: new RaspiIO()
});

board.on("ready", function() {
  // I2C LCD, PCF8574
  const lcd = new five.LCD({ 
  controller: "PCF8574T"
});

  lcd.print("Hello");
});

with:

  "dependencies": {
    "johnny-five": "^1.3.1",
    "raspi-io": "^10.0.4"
  }

And I kept seeing the squares.

Replacing, in the file node_modules/johnny-five/lib/lcd.js, this:

  this.io.i2cWrite(this.address, this.memory);

with this:

  this.io.i2cWrite(this.address, [this.memory]);

it works.
Also shows special characters, example:

  lcd.useChar("heart");
  lcd.cursor(0, 0);
  lcd.print("Hello :heart:");

[fivdi]

@ciarlystreet That's good news. We're nearly there. Thanks for the feedback.

@nebrius

This signature isn't documented though in the IO Plugin spec

Unfortunately the documentation is incomplete.

AFAIK Raspi IO never supported it.

Raspi IO supported it up until at least [email protected] here.

The use case mentioned by @dtex which, if I'm not mistaken, is "write a single byte to the specified register" was supported up until at least [email protected] here.

[nebrius]

Raspi IO supported it up until at least [email protected] here.

Huh, I always thought that was a hack for Remote IO or something, since firmata.js doesn't call Raspi IO when used directly in J5. TIL.

Turns out, that specific signature is in the code, but it's written as:

i2cWrite(address: number, register: number): void;

But the second argument is the payload, not the register, and the proper signature is:

i2cWrite(address: number, byte: number): void;

This is why I love TypeScript: it makes all this explicit and intentional, and it's impossible to have undocumented signatures 😁

I just published another version of j5-io with this fix.

@ciarlystreet can you run 'npm update' again and test to ensure this fix is correct?

[fivdi]

I tested this with the latest version of Raspi IO and everything now works as expected. I think this issue can be closed.

[dtex]

Awesome! Since @fivdi tested amd we haven't heard from @ciarlystreet in a while, I'm going to go ahead and close it. Feel free to re-open if necessary.

[nebrius]

Thanks for testing @fivdi!