Tcontrol support

[universam1]

Tcontrol should be supported as a backend.
Currently the communication required is a UDP frame.

The question @Zooms1 is, if the generic #3 HTTP approach would be possible to implement?

[universam1]

Work in progress to support Tcontrol

[Henielma]

Now I use to interface from the Brewometer to Tcontrol a UDP message to port 4968. The information string could be for example "T: 20.0 G: 1060.3 U: 3.65"
T: Temperature
G: Specific Gravity
U: Battery Voltage
Is it possible to extend the beer spindel software that it can send such a text string to UDP port 4968?

[Henielma]

The #3 solution possibly can also be used for Tcontrol.
Labview 8.6.1 has some support for Web Services like Read Postdata. I have not worked with it and I do not know if during the wait of 30 min the rest of the application keeps on running. Do you know of an example of such a http post so I can test if the combination with Labview is possible? Probably I can use temporary the Raspberry pi 2 to send a HTTP post. I will search for python scrips that send such data.

[Henielma]

Now my Wemos board is running and I uploaded the iSpindel immage I understand you screenshot better. Looks good your work in progress screenshot of the Wemos webpage. Server address is the IP port combined with the port number? What is the exact format of this field?

[universam1]

@Henielma
The UDP message is clear, we can create a sample implementation.

For you to test the genericHTTP, this one should be a valid test sample you could trigger from a linux box:

#3 (comment)

I'm not sure to put the port into the server field or make a separate input field. Probably the latter one.

[koffienl]

Keep in mind that UDP packets from/to ESP are not allways received (hence, UDP .. :P ).
For other project with UDP communication between ESP's I made the UDP repeats variable for the user. When the UDP packet is send, the repeats are taken and it is send as much as the repeats. The UDP packet itself contains a unique hash for that moment, so the receiver can filter out duplicate received packets.

[universam1]

Yes, I wouldn't choose UDP over TCP in this application. But if Tcontrol can't support other protocols we can support that too, but I would see it as the second best option.

[Henielma]

UDP packets can easily be lost because there is no check if it is received, I know. Unless that fact there is an advantage. Because it is only send and forget the power dissipation of the sender will be less than when an acknowledge protocol is used. This can be interesting for this application. When there is a good stable connection the disadvantage can be bigger than the advantage of UDP.

Nevertheless I agree we fist try to solve the connection to Tcontrol using POSTDATA or another possibility is TCP that is also supported by Labview 8.6 and this uses acknowledgement of reception. So one way or another we will get an connection.

This evening I will test the working with the new test version with HTTP.

[Henielma]

The last few day a lot of time spend to get something working with Labview 8.6.1 in combination with web services and I did not succeed to get good progress. Labview 8.6.1 is 8 years old and the web services was at that time new for Labview. This in combination with modern OS like Windows 10 makes it difficult for me to get it working together. And I foresee problems in the future when more Tcontrol uses want to use this iSpindel in combination with different Windows versions. So I prefer a more simple interface between the iSpindel and Tcontrol. UDP has the disadvantage that messages easily can get lost so the best solution is probably TCP.

In the configuration screen of the iSpindel only the IP address of the Tcontrol PC needs to be filled in. A fixed TCP port number can be used. For example 4969.

The transmitted data can for example be "T: 20.0 D: 70.53 U: 3.65"
T: Temperature
D: Degree of tilt
U: Battery Voltage

Does this fit into the iSpindel functionality in your view?

[universam1]

Okay I will make a proof of concept for you

[Henielma]

That is nice because I want to do the same test with the iSpindel as I do now with the Brewometer. In a test version of Tcontrol I used a part of the screen to give the information of the Brewometer and the calculated CO2 production (turquoise curve) according to the change of the SG (pink curve) during a fermentation. This can be compared to the purple curve that is the CO2 production measured with the balloon CO2 production sensor.

The calculated CO2 curve is much wilder then the measured cure but wen the SG calculation of the iSpindel has one or two decimals then I expect this will be much better.

[universam1]

OK I've implemented UDP on port 4968 I'm interested to see how it works! If it is reliable enough, the UDP implementation might outweighs the TCP or HTTP on Battery wearing a lot, so more often updates vs. lost data.
https://github.com/universam1/iSpindel/tree/master/bin/testing

Select TControl in the portal and the server name or ip.
Unfortunately I'm unable to test, hope it works

[Henielma]

Thanks for the implementation!
Just did a short test but I do not receive the data yet. Later this evening I will test a bit further and hopefully I can then see the send packages.

[Henielma]

[Henielma]

When during configuration I first select the last option and then the Tcontrol option I get two fields IP address and port number (4968). Using this configuration I get the following logging:

But listening on port 4968 does not give any data. I have not connected the DS18b20 sensor. Can that be the problem?

[universam1]

Hey I made a test setup and got it working, somehow. There were some bugs to solve.

In my setup, there was a huge loss with UDP about 80% which is not acceptable. Not totally sure if this could be improved, but I realized that the same setup on TCP just works great out of the box.

Switching to TCP adds almost no time to the transmission, so I wonder can you switch with TControl to TCP instead of UDP? Otherwise, the data frame is the very same.

If TCP is possible, the new firmware 3.3.1 should work for you!
Let me know how it goes.

[Henielma]

I am testing with version 3.3.2 the Tcontrol TCP function.

The Tcontrol application does not see any data yet but it seems that the iSpindel can send its data to my PC.

Does the iSpindel setup the port 4968 communication or has the Tcontrol application to do that?

[Henielma]

Yes it works! I do receive the TCP data from the iSpindel. Thanks for your effort.
Later I will post a screenshot of the Tcontrol screen with the iSpindel data.

[Henielma]

It is a robuust connection with the iSpindel. Last night I put the ESP board with accellero sensor using a power bank into my fermentation fridge on the first floor. Every 9 seconds there was an update received by the Wifi router in the living room downstairs.

Today I placed the ESP board on a steady place with a Tcontrol temperature sensor nearby. No sample filtering is added in Tcontrol for the accerosensor and still I see a almost flat curve (pink curve (value is degree / 3). The red curve is the measured Tcontrol temperature.

It would be nice if I can do a fermentation using the iSpindel. It seems a nice extra feature to the fermentation control software Tcontrol.

[Henielma]

For the SG calibration I want to add a field to the Tcontrol screen with the tilt using water. And the other calibration point will be the Yeast pitch SG. This will mostly be measured using a normal hydrometer or a Brix meter. This functionality I want to program the coming weeks.

[universam1]

Thanks for the update! This sounds like a success story. Let me know how things go on, when you agree we can declare the code then public.

[universam1]

For the SG calibration I want to add a field to the Tcontrol screen with the tilt using water. And the other calibration point will be the Yeast pitch SG.

Do you mean in the iSpindel? If so thats not a good idea, we were at that point already before. Because the calculation is way better done in high level Software like Tcontrol and not in the iSpindel. Consider every iSpindel will have a different formula and that would make it very complicated for the user. Instead, doing it in the Frontend is way easier to calibrate and you dont have to edit your iSpindel all the time

[Zooms1]

If there are specific instructions with setting up the iSpindel with Tcontrol, please let me know so I can incorporate instructions in the documents. Op 14 dec. 2016 20:32 schreef "universam1" <[email protected]>:

…

Thanks for the update! This sounds like a success story. Let me know how things go on, when you agree we can declare the code then public. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AXPqa_bBo5J1KsQkt8ksZHTKyi9TnhT-ks5rIERggaJpZM4LEt_U> .

[universam1]

Well just write from your experience, I think thats the very best description better than the me as the developer could do :)

[Henielma]

Do you mean in the iSpindel? If so thats not a good idea, we were at that point already before. Because the calculation is way better done in high level Software like Tcontrol and not in the iSpindel.

No in Tcontrol I mean. I fully agree to let the iSpindel send the tilt and not the SG.

Let me know how things go on, when you agree we can declare the code then public.

The only remark I have is that according to me you still have to select Tcontrol twice in the configuration mode to get the right IP address field. When that is solved the code can go public in my opinion.

If there are specific instructions with setting up the iSpindel with
Tcontrol, please let me know so I can incorporate instructions in the
documents.

Only select in the Configuration screen the Tcontrol option and fill in the IP address of the Tcontrol PC. The used port is fixed 4968 UDP. Thats all. The new version of Tcontrol will recognize this port and use the iSpindel information.

[universam1]

you still have to select Tcontrol twice in the configuration mode to get the right IP address field.

Yesterday I realized that some browser might have a problem or not compatible. I'm using Chrome, which one are you using? Okay I have to see if I can make it compatible.

The used port is fixed 4968 UDP.

Actually TCP

[universam1]

@Henielma Could you please try the fixed FW 3.4.2?

[Henielma]

I tested the version 3.4.2 but this version does not have the robuust TCP communication of version 3.3.2. Most of the time it missed a few messages and then it receives one.

Just to be sure I reloaded 3.3.2 and hier every message was received again. Then without closing Tcontrol uploaded 3.4.2 and directy the missing messages issue is back again.

[Henielma]

The flush version still misses a lot of samples.

The delay version is interesting. When the iSpindel is close (1 meter) to the wifi acces point no samples are lost anymore! The last update timer of Tcontrol variates between 23 and 25 seconds.

Next test is place the iSpindel in my fermentation fridge on the first floor. Three of the 21 samples are missed. The last update timer of Tcontrol variates from 23 till 32 seconds.

So this proves in my opinion that the TCP message repeat takes seconds extra time. Therefore it is needed to not switch off the iSpindel before the sample is transmitted correctly.

Is it possible to check in the iSpindel if the transmission buffer is empty before switching off? Or check a flag that the sample is transmitted correctly? Otherwise keep the iSpindel a (configurable) time on before going to sleep. The last option makes it more difficult for uses to configure it correctly for their situation.

Sending a string or character back from Tcontrol to the iSpindel is also a possibility but will consume more power then checking inside the iSpindel if the TCP protocol really has send the sample.

[universam1]

Unfortunately delay is not an option for us. See a normal run is less than 3s, adding a delay of 5s means reducing battery life of about 60%!!
So a response is by far less expensive and the better way.

But thinking twice about your comment it dawns on me. The 1.5.4 SDK introduces RF calibration that should be done at first. Guess the older SDK uses some good values and your pcb has issues without calibration. That would explain why you have better results at closer distance. You are the first one reporting this. Things to investigate further.

[Henielma]

I made a testversion of Tcontrol that sends back the text "Tcontrol" after receiving a sample. Probably you can make a version that not goes to sleep until this reaction is received or maximal waits for example 10 seconds for this reacion.

RF calibration is that something to program inside the ESP or is it something I have to do with this ESP module I have?

[Henielma]

I made the test version with responding with the text "Tcontrol" and then close the connection and tested again with the flush and delay version. Flush still looses samples but the delay version does not anymore even not in the fermentation fridge. At least 21 samples are received without one missing. De Last update times with configuration of 5 seconds is between 13 and 16 seconds.

[universam1]

Hi, okay this version is interesting, it does the response check plus retry 4x if not answered within 3s. So please monitor what happens in your setup and report. Thanks!
iSpindel.ino.response.zip

[Henielma]

Hi, it seems to work the response check when Tcontrol responses. When I remove the response in Tcontrol the iSpindel repeats the message at least 10 times and after that not anymore. So the normal behavior works well now, thanks. But the repeating and the tray to send a next sample does not work yet I think. Because when one sample is missed after the waiting time the next sample should be picked up.

[Henielma]

Now I tested again without sending a reply. At least 23 retries and after that he iSpindel resets to the unconfigured state and wants to be configured again.

[universam1]

Can you send me the logs as text file, would be Bert helpful. Thanks

[Henielma]

Here a normal transmission

[Henielma]

capture.txt

Here the logfile with the repeats and starting the configure mode
Open it with Wordpad

[universam1]

Okay thanks for the capture log file, that helps alot!!

Found a little bug while it just ran in the loop, it should only 4 times!
attaching fixed one:

iSpindel.ino.fixed.zip

Now I know why it reseted completely and went back to config mode. Because of the above bug the heap overrun.
That was a good catch!

Nevertheless, it shows that TC need up to 1000ms+ to answer! This could be the reason why we had many dropped frames, probably your application answers too slowly. Then the response is the only chance we have.
On the other occasions it couldnt connect at all. I really wonder if you have Wifi or ethernet issues.

[Henielma]

The Tcontrol application runs in a 1 second loop so the waiting time for the iSpindel can vary between 100 and 1000 ms. I just tried to reduce this time in Tcontrol but this is not easy to change at the moment. Probably something for me to improve later.

The fixed version works well in good and bad weather situations. This is very good!

During a normal received reply the iSpindel gets to sleep after 4 till 10 Waits of 100 ms.
When the iSpindel does not receive a good reply it repeats 3 times and waits after each repeat 30 waits (3 seconds).
This works very promising, thanks for all your effort!

[Henielma]

After almost one day test sending every 10 seconds a sample my powerbank battery is empty. The pink curve are the received tilt samples. The tilt value is divided by three. The result appears to be very constant. The iSpindel software works now fine with the Tcontrol software. I am happy with it.

[universam1]

Like to hear this news, great that this works now reliable. Your SW looks also very nice and sofisticated.
Close this issue when you are ok.

[Henielma]

It is okay for me to close this issue. Thanks for the TCP implementation in the very nice iSpindel.

[Henielma]

I am now working on a new version of Tcontrol and during testing I found out that the first sample of the iSpindel after power up gives 0.00 instead of the right angle. It is not a big issue of-course. The next sample is okay.

[Henielma]

I already solved it in Tcontrol by ignoring a sample with as tilt 0.00 so it is no problem for Tcontrol anymore. The change of getting a real tilt of exact 0.00 is neglect-able.

[universam1]

good point I'll check that!

[universam1]

I can reproduce it once in 10 times or so. Is it reproducible for you?

[Henielma]

In my situation it occurs almost every time after connecting and powering the USB cable to the PC. Keep in mind that for my test situation I power the D1 board via the USB cable without a battery.

[universam1]

That might be the reason, could you double check that with the finished iSpindel?
Thank you

[Henielma]

I just checked it with by powering my test setup with a USB battery pack instead of a PC. Then the first sample has a correct tilt. I have switched off and on the battery pack and all these 10 times the first sample is correct. So it seems that it appears mostly when during powering up the PC connects with the serial port via USB. So according to me it is not a big issue anymore.

[universam1]

@Henielma If you want to digg into the RF calibration have a look this:
https://github.com/esp8266/Arduino/blob/master/cores/esp8266/core_esp8266_phy.c#L242

You could try to set it to 3 to request a calibration.

[Henielma]

It is now 1 ( // 1: RF init only do TX power control CAL, others using RF CAL data in flash, it takes about 20ms for RF init) and the wifi communication works fine so I do not see a reason for trying further improvement. The battery life seems pretty good now. The iSpindel is already running 1 month sending every 10 minutes a sample and the battery is still 3.91 Volt.

[Henielma]

Yesterday I released the first Tcontrol version (V065) with iSpindel support. Thanks to the good support of universam1 the result has become a robuust solution.

The green curve is the SG determined by the CO2 production sensor and the turquoise curve is the SG determined by the iSpindel. These two curves are now almost the same!

In the manual of Tcontrol I also made a chapter for using and configuring the iSpindel. This manual can be found on my site at the end of the following page:
https://sites.google.com/site/henielma/tcontrol

[jakeelee]

Hi!
Really nice work, just got mine up and running!
Is there any plans to make some integration to Brewpi? or brewpiless?

[universam1]

support has not been requested for these systems, but it might be even easier to add support on their side to read the generic-HTTP or generic-TCP implementation.