Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syncrobit #60 and #58 ecc selection #2

Closed
invictus737 opened this issue Jan 27, 2023 · 150 comments
Closed

Syncrobit #60 and #58 ecc selection #2

invictus737 opened this issue Jan 27, 2023 · 150 comments

Comments

@invictus737
Copy link

Hello,

some V1 CM4 based Syncrobit miners are having the ECC address not standard and some extra handling is required for that special case. Please add at the minim the posibility to custom specify the i2c address (60 or 58) and the slot (0,1,2,3).

Regards

@shawaj
Copy link
Member

shawaj commented Jan 29, 2023

Do you have a source for this information of the different i2c bus and slot? This shouldn't be the case as it's part of the audit from helium foundation.

We currently have no way to dynamically assign the bus address or slot as it is pulled in from the hardware definitions in hm-pyhelper

You can already customise this to your needs but I appreciate it is not the simplest process if you aren't familiar with python packages etc

@invictus737
Copy link
Author

invictus737 commented Jan 29, 2023 via email

@shawaj
Copy link
Member

shawaj commented Jan 29, 2023

Hmm very odd. I guess they must be some custom version of the chip.

Do you know which slot the key is in?

Lastly, could you go to https://hub.balena.io/organizations/nebraltd/fleets/HELIUM-syncrobit and click "get started" and add one of your devices to this that have the ECC on address 58 instead of 60? And once done, can you let me know?

Thanks!

@shawaj
Copy link
Member

shawaj commented Feb 6, 2023

Any joy with this @invictus737 ?

@invictus737
Copy link
Author

Hi,
sorry not easy during winter to reach to those locations. In the mean time please add i2c-tools package or instruct how to add package. Required to drill down to the right i2c address and also https://github.com/helium/gateway-mfr-rs to test the right slot number where the keys are stored.

@shawaj
Copy link
Member

shawaj commented Feb 6, 2023

How are you accessing them then? Are they on BalenaCloud?

@invictus737
Copy link
Author

invictus737 commented Feb 6, 2023 via email

@shawaj
Copy link
Member

shawaj commented Feb 6, 2023

@invictus737 so you are not using our firmware?

@UrsoXone
Copy link

I am using your firmware and I do have one Syncrobit, which after flashing your firmware is showing on dashboard the ECC is not detected... may be this fella falls into the @invictus737 description. How can I check it?

@shawaj
Copy link
Member

shawaj commented Feb 11, 2023

@UrsoXone can you add gh_shawaj to have access to your fleet?

Then I can take a look

@UrsoXone
Copy link

UrsoXone commented Feb 11, 2023 via email

@shawaj
Copy link
Member

shawaj commented Feb 11, 2023

@UrsoXone Can you share a screenshot of the diagnostics? Or the diagnostics JSON download?

@UrsoXone
Copy link

UrsoXone commented Feb 11, 2023 via email

@shawaj
Copy link
Member

shawaj commented Feb 11, 2023

@UrsoXone there was nothing attached?

@shawaj
Copy link
Member

shawaj commented Feb 11, 2023

For my future reference, this is your device UUID - https://dashboard.balena-cloud.com/devices/99e4bc8a25781c87d8473d270d1f54d0

Screenshot 2023-02-11 at 12 50 11

I2c is on 0x58 as in above. I will see if I can get an update pushed over the weekend if I have any time to play around with it.

Could you leave this device turned on @UrsoXone ?

@UrsoXone
Copy link

Yes!
Please let me know the progress...I am afraid, I have more like that :). Thanks for the support and thanks @invictus737 for the trigger..

@jd3rul
Copy link

jd3rul commented Feb 11, 2023

Hej shawaj !
I am using your firmware (latest version) on 1 of my Syncrobits.
One of which after flashing your firmware is showing on dashboard the ECC is not detected...
How can I check it the memory slot?
Is it possible to release a fw image for this specific sample of units?

@UrsoXone
Copy link

UrsoXone commented Feb 11, 2023 via email

@mg198
Copy link

mg198 commented Feb 12, 2023

i have the same issue, flashed eMMC and now it is showing the issue below.

Overall Status: Errors Found
Firmware Version | 2023.02.07.0-5 (b3cc38c)
Variant | Syncrobit Hotspot
Hardware Serial Number | 100000009f8b4a8c

Radio Operational | Awaiting Location Assertion
ECC Detected | False

{"AN":null,"BA":"HELIUM-SYNCROBIT","BALENA_DEVICE_TYPE":["raspberrypicm4-ioboard"],"BN":"","BT":true,"BUTTON":27,"CELLULAR":false,"CONTAINS_FCC_IDS":[],"CONTAINS_IC_IDS":[],"CPU_ARCH":"aarch64","E0":"E4:5F:01:4C:A3:81","ECC":false,"ECCOB":true,"FCC_IDS":[],"FR":"N/A (Non Nebra Miner)","FRIENDLY":"Syncrobit Hotspot","FW":"2023.02.07.0-5","IC_IDS":[],"ID":"ebc5e7a6690fd7e487f823b2b289e7c6","KEY_STORAGE_BUS":"/dev/i2c-1","LOR":false,"LTE":false,"MAC":"wlan0","OK":null,"PF":false,"PK":null,"RE":"UN123","RESET":17,"SPIBUS":"spidev0.0","STATUS":22,"SUPPORTED_MODELS":["Syncrobit Hotspot"],"SWARM_KEY_URI":"ecc://i2c-1:96?slot=0","TYPE":"Full","VA":"COMP-SYNCROBITCM4","W0":"46:A4:11:40:B7:D9","firmware_short_hash":"b3cc38c","last_updated":"19:13 UTC 12 Feb 2023","serial_number":"100000009f8b4a8c"}

@UrsoXone
Copy link

UrsoXone commented Feb 12, 2023 via email

@mg198
Copy link

mg198 commented Feb 12, 2023

{"AN":null,"BA":"HELIUM-SYNCROBIT","BALENA_DEVICE_TYPE":["raspberrypicm4-ioboard"],"BN":"","BT":true,"BUTTON":27,"CELLULAR":false,"CONTAINS_FCC_IDS":[],"CONTAINS_IC_IDS":[],"CPU_ARCH":"aarch64","E0":"E4:5F:01:4C:A3:81","ECC":false,"ECCOB":true,"FCC_IDS":[],"FR":"N/A (Non Nebra Miner)","FRIENDLY":"Syncrobit Hotspot","FW":"2023.02.07.0-5","IC_IDS":[],"ID":"ebc5e7a6690fd7e487f823b2b289e7c6","KEY_STORAGE_BUS":"/dev/i2c-1","LOR":false,"LTE":false,"MAC":"wlan0","OK":null,"PF":false,"PK":null,"RE":"UN123","RESET":17,"SPIBUS":"spidev0.0","STATUS":22,"SUPPORTED_MODELS":["Syncrobit Hotspot"],"SWARM_KEY_URI":"ecc://i2c-1:96?slot=0","TYPE":"Full","VA":"COMP-SYNCROBITCM4","W0":"46:A4:11:40:B7:D9","firmware_short_hash":"b3cc38c","last_updated":"19:13 UTC 12 Feb 2023","serial_number":"100000009f8b4a8c"}

@shawaj
Copy link
Member

shawaj commented Feb 12, 2023

@shawaj
Copy link
Member

shawaj commented Feb 12, 2023

@UrsoXone and @mg198 can you please provide the name of your device from helium network and a link from the helium explorer?

@mg198
Copy link

mg198 commented Feb 12, 2023

@UrsoXone
Copy link

UrsoXone commented Feb 12, 2023 via email

@mg198
Copy link

mg198 commented Feb 12, 2023

I have it with me now. let me know if you need anything additional. It has been offline for a long time, 7 months. i have another syncrobit from the same batch but don't have physical access to it at the moment

@shawaj
Copy link
Member

shawaj commented Feb 12, 2023

Iast time you got direct access to it.

@UrsoXone I know, but I want to make sure that it is the same public key as expected that's why I need the name from onboarding

@mg198
Copy link

mg198 commented Feb 12, 2023

Okay perfect! Yes I have it with me and have access to it at any moment

@UrsoXone
Copy link

UrsoXone commented Feb 12, 2023 via email

@jd3rul
Copy link

jd3rul commented Mar 29, 2023

root@6255f448c59a:/opt# i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: 60 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
image

@shawaj
Copy link
Member

shawaj commented Mar 29, 2023

Hmm very strange @jd3rul - can you try issuing the command ./python-dependencies/hm_pyhelper/gateway_mfr_aarch64 --device ecc://i2c-1:96?slot=0 key

And increase the number before key up from 0 until the output matches your animal name

(For reference, the 60 shown in the output is 0x60 - which is hexadecimal address 60 - which is decimal 96 which is why we use 96 in the command mentioned above)

@jd3rul
Copy link

jd3rul commented Mar 29, 2023

Hm. Looks like it should have read it ... It worked right away in slot 0

image

@shawaj
Copy link
Member

shawaj commented Mar 29, 2023

@jd3rul what release is it running?

@jd3rul
Copy link

jd3rul commented Mar 29, 2023

@shawaj
Firmware Version | 2023.02.07.0-25 (d2dd598)

@shawaj
Copy link
Member

shawaj commented Mar 29, 2023

That explains it. The -26 update fixed this issue :-)

@jd3rul
Copy link

jd3rul commented Mar 29, 2023

@shawaj is there a force update check in diagnostics page ?

@shawaj
Copy link
Member

shawaj commented Mar 29, 2023

@jd3rul are you running on BalenaCloud or on our open fleet image?

@jd3rul
Copy link

jd3rul commented Mar 29, 2023

@shawaj I deployed using this method, posted up:

Navigate to the bottom of this page: https://github.com/NebraLtd/helium-syncrobit
Click on Deploy with balena - this will open your BalenaCloud page
Deploy to existing fleet - select the fleet with your Syncrobit miners:

@shawaj
Copy link
Member

shawaj commented Mar 30, 2023

@shawaj I deployed using this method, posted up:

Navigate to the bottom of this page: https://github.com/NebraLtd/helium-syncrobit Click on Deploy with balena - this will open your BalenaCloud page Deploy to existing fleet - select the fleet with your Syncrobit miners:

@jd3rul ok so the way to force the update will be just to repeat the process mentioned and it should then build a new release and you will get the -26 release ☺️

@jd3rul
Copy link

jd3rul commented Mar 30, 2023

Aweright!! Animal name appears in there.
image

I try now with the problematic one.

@jd3rul
Copy link

jd3rul commented Mar 30, 2023

@shawaj Ok, so I found one at 58x3 slot.
Now its the time to override swarm key location, but i see the command was deleted here from posts.
image

LE: Found it . Its SWARM_KEY_URI_OVERRIDE and value ecc://i2c-1:88?slot=3 . It worked, thanks!

@shawaj
Copy link
Member

shawaj commented Mar 30, 2023

@jd3rul great stuff

@jd3rul
Copy link

jd3rul commented Mar 31, 2023

@shawaj The first one I connected, appeared Active in explorer.helium pretty fast.
On the other hand, the 2nd one (which had slot 3 / 58 ) is online for many hours but still shows Inactive in explorer.
Is it a way to force some activity on chain ?

@shawaj
Copy link
Member

shawaj commented Mar 31, 2023

@jd3rul you probably need to reboot the device after you have entered the correct key slot

It should start working quite quickly once you find the correct keyslot

@jd3rul
Copy link

jd3rul commented Mar 31, 2023

So its on for 4+ hours, and still not showing online in explorer
I'll leave it like that more hours.

These messages keep repeating in logs:
diagnostics INFO:root:Diagnostics shipping not requested, skipping.
diagnostics INFO:root:Diagnostics complete

LE: Ok it worked! It finally witnessed. Nice!

@jd3rul
Copy link

jd3rul commented Apr 2, 2023

image
I cannot connect to this miner dashboard (local ip), and balena displays this.
I attached also diagnostics. Did you encounter this before?

helium-syncrobit_withered-street-02.04.23_14_27_07_(+0300).txt

@shawaj
Copy link
Member

shawaj commented Apr 2, 2023

This is the error:


<diagnostics> 2023-04-02 07:09:39,337 - [ERROR] - hm_pyhelper.miner_param - (ecc_diagnostic.py).perform_test -- /opt/python-dependencies/hw_diag/diagnostics/ecc_diagnostic.py:(43) - cannot access local variable 'ecc_location' where it is not associated with a value

<diagnostics> Traceback (most recent call last):

<diagnostics>   File "/opt/python-dependencies/hw_diag/diagnostics/ecc_diagnostic.py", line 21, in perform_test

<diagnostics>     ecc_tests = get_gateway_mfr_test_result()

<diagnostics>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 207, in get_gateway_mfr_test_result

<diagnostics>     return run_gateway_mfr("test")

<diagnostics>            ^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 71, in wrapper_lock_ecc

<diagnostics>     raise ex

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 60, in wrapper_lock_ecc

<diagnostics>     raise ex

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 57, in wrapper_lock_ecc

<diagnostics>     value = func(*args, **kwargs)

<diagnostics>             ^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 32, in run_gateway_mfr

<diagnostics>     command = get_gateway_mfr_command(sub_command, slot=slot)

<diagnostics>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 167, in get_gateway_mfr_command

<diagnostics>     get_ecc_location()

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 150, in get_ecc_location

<diagnostics>     if not ecc_location:

<diagnostics>            ^^^^^^^^^^^^

<diagnostics> UnboundLocalError: cannot access local variable 'ecc_location' where it is not associated with a value

<diagnostics> ERROR:hm_pyhelper.miner_param:cannot access local variable 'ecc_location' where it is not associated with a value

<diagnostics> Traceback (most recent call last):

<diagnostics>   File "/opt/python-dependencies/hw_diag/diagnostics/ecc_diagnostic.py", line 21, in perform_test

<diagnostics>     ecc_tests = get_gateway_mfr_test_result()

<diagnostics>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 207, in get_gateway_mfr_test_result

<diagnostics>     return run_gateway_mfr("test")

<diagnostics>            ^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 71, in wrapper_lock_ecc

<diagnostics>     raise ex

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 60, in wrapper_lock_ecc

<diagnostics>     raise ex

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/lock_singleton.py", line 57, in wrapper_lock_ecc

<diagnostics>     value = func(*args, **kwargs)

<diagnostics>             ^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 32, in run_gateway_mfr

<diagnostics>     command = get_gateway_mfr_command(sub_command, slot=slot)

<diagnostics>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 167, in get_gateway_mfr_command

<diagnostics>     get_ecc_location()

<diagnostics>   File "/opt/python-dependencies/hm_pyhelper/miner_param.py", line 150, in get_ecc_location

<diagnostics>     if not ecc_location:

<diagnostics>            ^^^^^^^^^^^^

<diagnostics> UnboundLocalError: cannot access local variable 'ecc_location' where it is not associated with a value

I can see where it's coming from so will push an update to try and fix this. However, it is essentially because it is not finding the correct key slot.

@shawaj
Copy link
Member

shawaj commented Apr 2, 2023

image I cannot connect to this miner dashboard (local ip), and balena displays this. I attached also diagnostics. Did you encounter this before?

helium-syncrobit_withered-street-02.04.23_14_27_07_(+0300).txt

@jd3rul is this the same miner that you said was working before or a different one?

Also, can you add gh_shawaj as a developer user on your fleet on BalenaCloud? Then I can take a look at it quickly

@jd3rul
Copy link

jd3rul commented Apr 2, 2023

@shawaj It was not working before. Its recently added and having troubles.
I just added you to the fleet.

@7ChrisMM4
Copy link

Thanks @7ChrisMM4 🙏

@shawaj @WayneNebra - Sorry, this fell off the map due to some extenuating circumstance. I am following up as my device still shows ECC Error. Looks like a lot of progress has been made for some. Can I get the cliff notes and direction on how to fix?

@7ChrisMM4
Copy link

ticket 410466

this is in relation to ticket 410466 @shawaj @WayneNebra

@shawaj
Copy link
Member

shawaj commented Apr 26, 2023

Closing this as fixed by NebraLtd/hm-diag#560 and NebraLtd/helium-miner-software#753

For anyone reading this, the summary is:

  • 0x58 miners almost exclusively have their ECC key on slot 2. We have fixed all issues with these keys that would cause them to show as not working on diagnostics
  • there are a small number (less than 50) Syncrobit units that have 0x58 ECC with slot that isn't 2. Most of these are on slot3 but a few on others. We can manually override these to the correct slot. If you have this issue with the ECC key still showing an error in diagnostics after the v1.0.2-6 release then please reach out to support at https://support.nebra.com and we can fix the issue directly for you. If you have this issue and are running on BalenaCloud you need to follow the process described here... Syncrobit #60 and #58 ecc selection #2 (comment)
  • 0x60 all have their ECC key on slot 0

So we think this should all be resolved

@shawaj shawaj closed this as completed Apr 26, 2023
@7ChrisMM4
Copy link

7ChrisMM4 commented Apr 26, 2023 via email

@invictus737
Copy link
Author

invictus737 commented Apr 26, 2023 via email

@shawaj
Copy link
Member

shawaj commented Apr 28, 2023

You guys are ROCKSTARS!!!! Thank you for your support with this.

On Wed, Apr 26, 2023 at 4:44 AM Aaron Shaw @.> wrote: Closed #2 <#2> as completed. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6KCUC7V6TZMN3FOLEGUNJDXDEDARANCNFSM6AAAAAAUIYU5XM . You are receiving this because you were mentioned.Message ID: @.>
-- Chris

No probs @7ChrisMM4

All we ask in return is for a couple 5 star reviews on Trustpilot and Google at the links below :-)

@mg198
Copy link

mg198 commented Dec 16, 2023

Hey sorry to repost here, i just reflashed my syncrobit via em4 from the jumper board (nebra firmware with belenta on MAC), i cannot recall how to update the emmc to slot 3 as it is both showing offline on belenta cloud BUT i can see it locally on the nebra dashboard but it says: Radio Operational: Awaiting Location Assertion (this should already be asserted)
ECC Detected: greeen check mark

[Download Diagnostics Info for Support:

{"AN":null,"BA":"HELIUM-SYNCROBIT","BALENA_DEVICE_TYPE":["raspberrypicm4-ioboard"],"BN":"","BT":true,"BUTTON":27,"CELLULAR":false,"CONTAINS_FCC_IDS":[],"CONTAINS_IC_IDS":[],"CPU_ARCH":"arm64","E0":"E4:5F:01:4C:A3:81","ECC":true,"ECCOB":true,"FCC_IDS":[],"FR":"N/A (Non Nebra Miner)","FRIENDLY":"Syncrobit Hotspot","FW":"v1.3.0","IC_IDS":[],"ID":"a2fafd6579d5ecf4ad2297eeadf57239","LOR":false,"LTE":false,"MAC":"wlan0","OK":null,"ONBOARDING_KEY_URI":["ecc://i2c-1:96?slot=0","ecc://i2c-1:88?slot=15"],"PF":false,"PK":null,"RE":"UN123","RESET":17,"SPIBUS":"spidev0.0","STATUS":null,"STATUS_NEOPIXEL":{"ACTIVE_HIGH":true,"CHANNEL":0,"DMA":10,"LED_COUNT":6,"PIN":18},"SUPPORTED_MODELS":["Syncrobit Hotspot"],"SWARM_KEY_URI":["ecc://i2c-1:96?slot=0","ecc://i2c-1:88?slot=2"],"TYPE":"Full","VA":"COMP-SYNCROBITCM4","W0":"0E:99:B1:7F:C5:AD","commercial_fleet":false,"config_version":"a198b83","diagnostics_passed":false,"diagnotics_version":"6d56740","errors":["gateway_pubkey","gateway_pubkey","RE","gateway_region"],"firmware_short_hash":"c2dd87a","gateway_pubkey":"<_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = "DNS resolution failed for helium-miner:4467: C-ares status is not ARES_SUCCESS qtype=A name=helium-miner is_balancer=0: DNS server returned answer with no data"\n\tdebug_error_string = "UNKNOWN:DNS resolution failed for helium-miner:4467: C-ares status is not ARES_SUCCESS qtype=A name=helium-miner is_balancer=0: DNS server returned answer with no data {created_time:"2023-12-16T15:48:19.877172644+00:00", grpc_status:14}"\n>","gateway_region":"<_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = "DNS resolution failed for helium-miner:4467: C-ares status is not ARES_SUCCESS qtype=A name=helium-miner is_balancer=0: DNS server returned answer with no data"\n\tdebug_error_string = "UNKNOWN:DNS resolution failed for helium-miner:4467: C-ares status is not ARES_SUCCESS qtype=A name=helium-miner is_balancer=0: DNS server returned answer with no data {grpc_status:14, created_time:"2023-12-16T15:48:20.121942161+00:00"}"\n>","gatewayrs_version":"5428419","last_updated":"15:48 UTC 16 Dec 2023","multiplexer_version":"a1fc09b","nebra_fleet":true,"packet_forwarder_version":"5c3c90a","serial_number":"100000009f8b4a8c"}

@jd3rul
Copy link

jd3rul commented Dec 16, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.