New mqtt version with mosquitto

[bwired-nl]

Hi

Im using mqtt for a long time and is working very good
this weekend i installed a new raspberry with the new nodejs 0.12 and also your new Mqtt

I cant get it to work anymore?
If I use your new sample, mosquitto does not see any topics
The program runs without errors but no topics?
My mosquitto is running on another raspberry working good with other raspberrys with lower version of mqtt
Is there a possibility to install an older version of mqtt, so i can check?
any known issues for this?

Thanks
pieter

[itavy]

Hi,

take a look at #249 if you are using it over SSL.

if you want to check it with older versions you have 2 options:

1. use package.json file to specify what version you will use
2. npm install mqtt@"version"

if you need more help regarding your issue please post some sample code of what are you doing so we can help.

Best regards,
itavy

[mcollina]

Which example are you using that it is not working?

There might be a race condition problem in the example in the README. It seemed fixed, but please try:

var mqtt    = require('mqtt');
var client  = mqtt.connect('mqtt://test.mosquitto.org');

client.on('connect', function() {
  client.subscribe('presence');
  client.publish('presence', 'Hello mqtt');
});

client.on('message', function (topic, message) {
  // message is Buffer
  console.log(message.toString());
  client.end();
});

[bwired-nl]

Hi All

thanks for helping!
the new above sample works perfect on test.mosquitto

but i have mosquitto installed on a cubietruck (ubuntu)
i have installed ubuntu version mosquitto form here http://mosquitto.org/download/
My problem is that the old version of Mqtt is working perfect with this setting
but if i upgrade to last mqtt version not anymore

i have done the changes regarding connect
this is not working, could be that is see something wrong, not an expert :-)

var mqttbroker = { address: '10.0.0.47', port: '1883' };
var mqtt = require('mqtt');
var client = mqtt.connect(mqttbroker.port, mqttbroker.address);

client.on('connect', function() {
client.subscribe('presence');
client.publish('presence', 'Hello mqtt');
});

client.on('message', function (topic, message) {
// message is Buffer
console.log(message.toString());
client.end();
});

but also this not working
var client = mqtt.connect('10.0.0.47');
/home/pi/node_modules/mqtt/lib/connect/index.js:62
opts.protocol = opts.protocol.replace(/:$/, '');
^

and this not (this just ends the programs and gives back cursor)

var mqtt = require('mqtt')
, host = '10.0.0.47' // or localhost
, client = mqtt.connect();
// or , client = mqtt.connect({ port: 1883, host: host, keepalive: 10000});

client.subscribe('presence');
client.publish('presence', 'bin hier');
client.on('message', function (topic, message) {
console.log(message);
});
client.end();

[itavy]

hi,

connect constructor use 2 parameters:
connect(brockerUrl, opts)

look in lib/connect/index.js from line 53 to 58 for how to use those 2 parameters

your last example is correct, but you call client.end() before you actualy receive the message. move the client.end() inside the function for receive message and it should work

Best regards,
itavy

[bwired-nl]

Hi itavy

sorry, i had tested that also, did so many testing

this is not working also,
i get this i see nothing happening, justing starting....
pi@raspberrypi ~ $ node testmqtt.js
starting

but i have also this for monitoring, and nothing comes in there also?
C:\Program Files (x86)\mosquitto>mosquitto_sub -h 10.0.0.47 -v -t #

this is the code i use now, like i told before with old mqtt its running instantly with same setting

var mqtt    = require('mqtt')
  , host = '10.0.0.47' // or localhost
  , client = mqtt.connect();

console.log('starting');
client.subscribe('presence');
client.publish('presence', 'bin hier');
client.on('message', function (topic, message) {
  console.log(message);
  client.end();
});

[itavy]

hi,

i can tell you what i have done when i switched first time:

1. i put a lot of intermediate messages to see step by step where it hangs
2. i put mosquitto in debug mode and watched all messages
   maybe that will help because otherwise i don't see any flaw in your code

Regards,
Octavian Ionescu

[bbx10]

Hi,

I am usinge Ubuntu 14.04, mosquitto, and node mqtt on the same laptop. mosquitto_sub and mosquitto_pub on localhost work as expected but node mqtt does not.

node v.0.10.25
npm install mqtt

Here is the test code with log messages for every event.

var mqtt    = require('mqtt');
var client  = mqtt.connect('mqtt://127.0.0.1');

console.log('mqtt.connect');

client.on('connect', function () {
  console.log('on connect');
  client.subscribe('presence');
  client.publish('presence', 'Hello mqtt');
});

client.on('error', function(err) {
  console.log('on error', err);
});

client.on('close', function() {
  console.log('on close');
});

client.on('disconnect', function() {
  console.log('on disconnect');
});

client.on('reconnect', function() {
  console.log('on reconnect');
});

client.on('offline', function() {
  console.log('on offline');
});

client.on('message', function (topic, message) {
  // message is Buffer 
  console.log(message.toString());
  client.end();
});

The output looks like this:

mqtt.connect
on offline
on close
on reconnect
on offline
on close
on reconnect

If the URL is changed to mqtt://test.mosquitto.org, the test works fine. The output looks like this:

mqtt.connect
on connect
Hello mqtt
on close

I suspect this is a race problem since connecting to localhost must be < 1ms while connecting to test.mosquitto.org is probably > 50 ms depending on location.

I will try to work backwards from the offline event since it is emitted in a single place in the code.

Thanks,
bbx10node

[mcollina]

Which version of Mosquitto are you running locally?

Il giorno mer 18 mar 2015 alle ore 23:02 bbx10 [email protected]
ha scritto:

Hi,

I am usinge Ubuntu 14.04, mosquitto, and node mqtt on the same laptop.
mosquitto_sub and mosquitto_pub on localhost work as expected but node mqtt
does not.

node v.0.10.25
npm install mqtt

Here is the test code with log messages for every event.

var mqtt = require('mqtt');var client = mqtt.connect('mqtt://127.0.0.1');
console.log('mqtt.connect');

client.on('connect', function () {
console.log('on connect');
client.subscribe('presence');
client.publish('presence', 'Hello mqtt');
});

client.on('error', function(err) {
console.log('on error', err);
});

client.on('close', function() {
console.log('on close');
});

client.on('disconnect', function() {
console.log('on disconnect');
});

client.on('reconnect', function() {
console.log('on reconnect');
});

client.on('offline', function() {
console.log('on offline');
});

client.on('message', function (topic, message) {

// message is Buffer
console.log(message.toString());
client.end();
});

The output looks like this:

mqtt.connect
on offline
on close
on reconnect
on offline
on close
on reconnect

If the URL is changed to mqtt://test.mosquitto.org, the test works fine.
The output looks like this:

mqtt.connect
on connect
Hello mqtt
on close

I suspect this is a race problem since connecting to localhost must be <
1ms while connecting to test.mosquitto.org is probably > 50 ms depending
on location.

I will try to work backwards from the offline event since it is emitted in
a single place in the code.

Thanks,
bbx10node

—
Reply to this email directly or view it on GitHub
#257 (comment).

[bbx10]

Hi,

mosquitto 3.1 is the default version for Ubuntu 14.04.

I found the problem and has nothing to do with timing or race conditions. node mqtt requests protocolVersion 4 by default but mosquitto supports up to 3. And mosquitto does not like MQTT
for the protocolId.

The following messages appear in syslog when mosquitto logging is turned on.

mosquitto[6884]: Invalid protocol version 4 in CONNECT
mosquitto[6884]: Invalid protocol "MQTT" in CONNECT

The following works but I am not sure what the protocol ID should be. I copied the value that mosquitto_sub sends.

var client  = mqtt.connect('mqtt://127.0.0.1', {
  protocolId: 'MQIsdp',
  protocolVersion: 3 
});

I will take a look at mosquitto source code to see what the rules are for the protocol ID. Or just build from source code instead of using the version in the repo.

bbx10node

[bbx10]

Hi,

Installing mosquitto from the mosquitto dev PPA solves the problem as well. The version is 1.4-0mosquitto1. The example in the README works without changes except I changed the URL to 'mqtt://127.0.0.1'.

bbx10node

[mcollina]

Thanks for reporting and helping triaging this one. I updated the README accordingly.

[derekxm]

Thanks @bbx10. I ran into exactly the same problem. Your post helped immensely.

[matriawan]

@bbx10 and @mcollina i've same issue in localhost doesn't work, if install default MQTT package on ubuntu 14.04, solved problem by installing using this method, work like a charm
$sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
$sudo apt-get update
$sudo apt-get install mosquitto libmosquitto-dev mosquitto-clients
$sudo service mosquitto status
after installing the mosquitto packages, any configuration in "/etc/mosquitto"
PS: if doesn't work, make sure the internet connection without any proxy configuration

[jimbojw]

This problem is still around here a year later. According to my reading of the MQTT spec, the broker should return a CONNACK with return code 0x01 if the protocol level isn't specified. Is there any way the module can detect this and emit an 'error' event?

[mcollina]

@jimbojw I think it's fairly straightforward. Would you mind to send a PR?

[jimbojw]

Update: I spent a couple of hours looking into this, about at the limit I'm willing to sink. :)

I've noticed a couple of problems (by module):

• MQTT.js - The mocha tests in MQTT.js for the connect() method don't seem to actually fully test the connection (to me, they look like they just create connection objects of various sorts).
• MQTT.js - The createConnection() test does actually create a TCP connection to whatever's running on port 1883, but it calls it a success after the underlying net connection connects. That is, the createConnection() method does not actually wait for a connack before emitting connected. Also it's deprecated.
• mqtt-connection - I looked into the mqtt-connection module too. Tests there are failing right out the gate.
• mqtt-connection - All the tests seem to use protocol version 3 (MQTT 3.1) not 4 (MQTT 3.1.1).

Somewhere there should be a test that implements a minimal shim server that correctly connack's for version 3 but not 4, like old versions of Mosquitto do. It's not clear to me which package ought to contain this, my guess would be mqtt-connect.

I stand by my assertion though if the server connack's with return code 0x01, this should be interpreted by the client as an error condition that will not be resolved by reconnecting and should emit an 'error' event. It's not clear to me however which module ought to be responsible for this and how to implement a test for it.

Next steps:

• Create a bug in the appropriate module's github repo with all of this information.
• Implement a test that fails under the current behavior.
• Update relevant code to pass the test and submit a PR.

[mcollina]

So:

1. createConnection is going to be removed in v2, in favor of mqtt-connection. The whole idea of a 'connection' object is completely broken.
2. connect() (https://github.com/mqttjs/MQTT.js/blob/master/lib/connect/index.js#L132) is responsible for allocating a new MqttClient. The tests verifies the options  on which this is created. The actual tests are really spread in files, because of how this project evolved over time. Any consolidation there will be easy.
3. mqtt-connection tests failing: they were passing last time I checked. Maybe some library got updated, feel free to send a PR.
4. Yes.

You want to place the fix here: https://github.com/mqttjs/MQTT.js/blob/master/lib/client.js#L681-L701.
For the test, you can start a new server for a test in https://github.com/mqttjs/MQTT.js/blob/master/test/client.js, where you listen only to the 'connect' event, and reply with the given CONNACK return code.

[jimbojw]

TL;DR This is a problem with the MQTT protocol, not the mqtt Node module.

Thanks for the link to the code for _handleConnack. I dug into this a little further, and I think the problem is actually in the MQTT 3.1 protocol itself.

From the spec on CONNECT: http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connect (emphasis mine)

If the client sends an invalid CONNECT message, the server should close the connection. This includes CONNECT messages that provide invalid Protocol Name or Protocol Version Numbers. If the server can parse enough of the CONNECT message to determine that an invalid protocol has been requested, it may try to send a CONNACK containing the "Connection Refused: unacceptable protocol version" code before dropping the connection.

I investigated the bytes passed on the TCP connection between mqtt and old Mosquitto. As far as I can tell, Mosquitto is simply closing the connection in response to the invalid protocol version number, which the specification allows it to do.

From the client's perspective, the socket just closed unexpectedly. It has no way of knowing whether this was a network interruption or a wilful rejection of the CONNECT packet. Continuously attempting to reconnect may be the correct move here.

However, from the perspective of the user of the mqtt npm module, there is no difference between attempting to connect to a TCP port that's not listening at all, and one that accepts the TCP connection but rejects the CONNECT packet. There is no 'connected' event that signals that the underlying TCP connection was at least successful.

When connecting, there are two cases worth considering separately. One is the case where at some point in the past a successful CONNACK had been received. These cases will often be due to a connectivity issue that could be resolved in the future. For example if the broker restarts, or there's a hiccup in the client's network connectivity.

The other case is where at no point in the past has the client ever received a CONNACK. These cases are often (but not always) going to be due to incorrect configuration like wrong host/port, wrong protocol version, etc. This is the case with old Mosquitto.

Currently the mqtt module's strategy is optimized for the first case. In the case of a disconnect, we attempt reconnect ad infinitum.

It might be nice to have an optional "maximum first connect threshold" that stops trying to connect after some reasonable number of attempts if a CONNACK has never been received. This would help users triage configuration issues, but should NOT be the default since this would change the behavior of production systems which may already rely on the infinite reconnect strategy.

In any case, I do not have cycles to implement this functionality myself, and we may as well close this bug as working as intended.

[mcollina]

Thanks for investigating this, and providing a definite answer for this.
I'm closing this issue, and opening a new one for maxFirstConnects options.