Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add bottom - next/previous page in CAPTURE #1002

Closed
i6cbi opened this issue Jul 8, 2024 · 2 comments
Closed

add bottom - next/previous page in CAPTURE #1002

i6cbi opened this issue Jul 8, 2024 · 2 comments

Comments

@i6cbi
Copy link

i6cbi commented Jul 8, 2024

No description provided.

@i6cbi
Copy link
Author

i6cbi commented Jul 8, 2024

Hi Miha.
i don't know pull request
i added possibilty to go next or previous page even at the bottom of the page without to go back to the top

raspberry-noaa-v2/webpanel/App/Views/Captures
after line 102

{% endfor %}
<link rel="stylesheet" type="text/css" href="/assets/css/pagination.css">
<nav aria-label="page" id="pagination" class="mb-0">
  {% include('Captures/pagination.html') %}
{% endblock %}

``

@creas002
Copy link
Contributor

New feature request is now included in aarch-support in this commit. 370013b

@i6cbi i6cbi closed this as completed Jul 14, 2024
MihajloPi added a commit that referenced this issue Jul 31, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>
MihajloPi added a commit that referenced this issue Aug 1, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>
MihajloPi added a commit that referenced this issue Aug 1, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>
MihajloPi added a commit that referenced this issue Aug 1, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* fixes #1025

* integration for ELEKTRO-L3

* fix wrong log call

* fix thumbnail

* add missing section in noaa-v2.conf.js2 template

* add ELEKTRO_L3_FREQ in common.sh

* also show images of each channel, not only the composites

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: MihajloPi <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
MihajloPi added a commit that referenced this issue Aug 5, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for Meteor when decoding with SatDump

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>
MihajloPi added a commit that referenced this issue Aug 7, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

#1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a007035
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for Meteor when decoding with SatDump

* Update README.md (#1037)

* Update core.yml - Simply getting OS Release (#1038)

Update core.yml - Simply getting OS Release 

Goal was to work on as many as Debian Distro's as possible even if they are using local repo's

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>
MihajloPi added a commit that referenced this issue Sep 12, 2024
…d many more. (#1070)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* aarch64/arm64 support and general fixes (#1029)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for SatDump Meteor images

* Fixing filling issue for Meteor when decoding with SatDump

* Update README.md (#1037)

* Update core.yml - Simply getting OS Release (#1038)

Update core.yml - Simply getting OS Release 

Goal was to work on as many as Debian Distro's as possible even if they are using local repo's

* Update README.md (#1043)

* Update rn2_uninstall.sh (#1045)

Commenting out systemctl disable service because disable causes it to remain disabled even after the service is removed and reinstalled by Ansible 
	
Commenting out the disable command has no impact since it will be removed when removing packages

* Update verification.sh (#1046)

Ooops... I had hard coded /home/richard , fixed with ${HOME}

* Beta development - Setting SDR Device ID, Fixed Receive scripts to pass PPM Offset, Fixed dependancies to work on OS Lite versions (#1049)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Fix to be tested

* Fix to be tested for RTL-SDR PPM correction (#1051)

* Fixing filling issue for SatDump Meteor images

* Fix to be tested

* Beta development - modified receive scripts to pass PPM & Offset based on receiver type (#1052)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Reverting PPM changes to receive scripts

ppm_correction should only be used for RTPSDR and RTL-TCP, there may be other receivers that can use it but unknown

* Update noaa-v2.conf.j2

This was left out of ansible/roles/common/templates/noaa-v2.conf.j2 template file which caused an error for anyone enabling it.

# Coronal mass ejection activity
ENABLE_CORONAL={{ enable_coronal|lower }}

* modified receive scripts to pass PPM & Offset based on receiver type

Modified receivenoaa.sh & receive_meteor.sh scripts to pass PPM & Offset based on receiver type.

Also added a log statement so everybody can see whats getting sent to satdump from output.log

/home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E

# I added setting FREQ_OFFSET=""  when its not an rtlsdr receiver to prevent the Offset being passed to satdump

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--gain"
  ppm_correction="--ppm_correction"
else
  gain_option="--general_gain"
  FREQ_OFFSET=""
fi

I tested it for NOAA 18 and you can see PPM and Offset was passed to satdump as desired:

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7454 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
INFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm was passed

14:08:43 - 13/08/2024] (D) Parameters :
[14:08:43 - 13/08/2024] (D)    - align_timestamps : true
[14:08:43 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:08:43 - 13/08/2024] (D)    - autocrop_wedges : true
[14:08:43 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:08:43 - 13/08/2024] (D)    - buffer_size : 1000000
[14:08:43 - 13/08/2024] (D)    - frequency : 137912500
[14:08:43 - 13/08/2024] (D)    - gain : 49.6
[14:08:43 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:08:43 - 13/08/2024] (D)    - ppm_correction : 0
[14:08:43 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--general_gain"
  FREQ_OFFSET=""
else
  gain_option="--gain"
  ppm_correction="--ppm_correction"
fi

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7452 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --general_gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
^CINFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm & offset was not passed

[14:11:17 - 13/08/2024] (D) Parameters :
[14:11:17 - 13/08/2024] (D)    - align_timestamps : true
[14:11:17 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:11:17 - 13/08/2024] (D)    - autocrop_wedges : true
[14:11:17 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:11:17 - 13/08/2024] (D)    - buffer_size : 1000000
[14:11:17 - 13/08/2024] (D)    - frequency : 137912500
[14:11:17 - 13/08/2024] (D)    - general_gain : 49.6
[14:11:17 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:11:17 - 13/08/2024] (D)    - samplerate : 1024000

I modified receive_metoer.sh and performed the same testing

/home/richard/raspberry-noaa-v2/scripts/receive_meteor.sh "METEOR-M2 4" METEOR-M2-4-20240813-194754 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723578474 3 83 Northbound W

I tested it for M2-4  and you can see PPM and Offset was passed to satdump as desired:

INFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7462 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9000e6   --gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm was passed

[14:32:22 - 13/08/2024] (D) Parameters :
[14:32:22 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:32:22 - 13/08/2024] (D)    - buffer_size : 1000000
[14:32:22 - 13/08/2024] (D)    - cadu_size : 8192
[14:32:22 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:32:22 - 13/08/2024] (D)    - derandomize : true
[14:32:22 - 13/08/2024] (D)    - fill_missing : true
[14:32:22 - 13/08/2024] (D)    - frequency : 137900000
[14:32:22 - 13/08/2024] (D)    - gain : 49.6
[14:32:22 - 13/08/2024] (D)    - nrzm : true
[14:32:22 - 13/08/2024] (D)    - ppm_correction : 0
[14:32:22 - 13/08/2024] (D)    - rs_dualbasis : false
[14:32:22 - 13/08/2024] (D)    - rs_i : 4
[14:32:22 - 13/08/2024] (D)    - rs_type : "rs223"
[14:32:22 - 13/08/2024] (D)    - rs_usecheck : true
[14:32:22 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

NFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7463 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9000e6   --general_gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm & offset was not passed

[14:35:06 - 13/08/2024] (D) Parameters :
[14:35:06 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:35:06 - 13/08/2024] (D)    - buffer_size : 1000000
[14:35:06 - 13/08/2024] (D)    - cadu_size : 8192
[14:35:06 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:35:06 - 13/08/2024] (D)    - derandomize : true
[14:35:06 - 13/08/2024] (D)    - fill_missing : true
[14:35:06 - 13/08/2024] (D)    - frequency : 137900000
[14:35:06 - 13/08/2024] (D)    - general_gain : 49.6
[14:35:06 - 13/08/2024] (D)    - nrzm : true
[14:35:06 - 13/08/2024] (D)    - rs_dualbasis : false
[14:35:06 - 13/08/2024] (D)    - rs_i : 4
[14:35:06 - 13/08/2024] (D)    - rs_type : "rs223"
[14:35:06 - 13/08/2024] (D)    - rs_usecheck : true
[14:35:06 - 13/08/2024] (D)    - samplerate : 1024000

* Ensure ownership/permissions are set correctly for key scripts

* Ensure ownership/permissions are set correctly for key scripts

---------

Co-authored-by: MihajloPi - YU4MPI <[email protected]>

* Changing config directory ownership as root

* Removing whitespaces

* Ooops...

* Allow odd timezones in sun.py (#1056)

* fixes #1021

* fixes #1055

* Fix deprecation warning in heatmap.py (#1059)

* fixes #1021

* fixes deprecation message for getsize and ANTIALIAS (will be removed in Pillow 10)

* Beta development - Update dependencies/yml - added become: yes for .config ownership (#1057)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/a…
MihajloPi added a commit that referenced this issue Sep 12, 2024
* Bumping version

* 64-bit Bookworm support for Raspberry Pi, SatDump update, bugfixes and many more. (#1070)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* aarch64/arm64 support and general fixes (#1029)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for SatDump Meteor images

* Fixing filling issue for Meteor when decoding with SatDump

* Update README.md (#1037)

* Update core.yml - Simply getting OS Release (#1038)

Update core.yml - Simply getting OS Release 

Goal was to work on as many as Debian Distro's as possible even if they are using local repo's

* Update README.md (#1043)

* Update rn2_uninstall.sh (#1045)

Commenting out systemctl disable service because disable causes it to remain disabled even after the service is removed and reinstalled by Ansible 
	
Commenting out the disable command has no impact since it will be removed when removing packages

* Update verification.sh (#1046)

Ooops... I had hard coded /home/richard , fixed with ${HOME}

* Beta development - Setting SDR Device ID, Fixed Receive scripts to pass PPM Offset, Fixed dependancies to work on OS Lite versions (#1049)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Fix to be tested

* Fix to be tested for RTL-SDR PPM correction (#1051)

* Fixing filling issue for SatDump Meteor images

* Fix to be tested

* Beta development - modified receive scripts to pass PPM & Offset based on receiver type (#1052)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Reverting PPM changes to receive scripts

ppm_correction should only be used for RTPSDR and RTL-TCP, there may be other receivers that can use it but unknown

* Update noaa-v2.conf.j2

This was left out of ansible/roles/common/templates/noaa-v2.conf.j2 template file which caused an error for anyone enabling it.

# Coronal mass ejection activity
ENABLE_CORONAL={{ enable_coronal|lower }}

* modified receive scripts to pass PPM & Offset based on receiver type

Modified receivenoaa.sh & receive_meteor.sh scripts to pass PPM & Offset based on receiver type.

Also added a log statement so everybody can see whats getting sent to satdump from output.log

/home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E

# I added setting FREQ_OFFSET=""  when its not an rtlsdr receiver to prevent the Offset being passed to satdump

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--gain"
  ppm_correction="--ppm_correction"
else
  gain_option="--general_gain"
  FREQ_OFFSET=""
fi

I tested it for NOAA 18 and you can see PPM and Offset was passed to satdump as desired:

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7454 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
INFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm was passed

14:08:43 - 13/08/2024] (D) Parameters :
[14:08:43 - 13/08/2024] (D)    - align_timestamps : true
[14:08:43 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:08:43 - 13/08/2024] (D)    - autocrop_wedges : true
[14:08:43 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:08:43 - 13/08/2024] (D)    - buffer_size : 1000000
[14:08:43 - 13/08/2024] (D)    - frequency : 137912500
[14:08:43 - 13/08/2024] (D)    - gain : 49.6
[14:08:43 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:08:43 - 13/08/2024] (D)    - ppm_correction : 0
[14:08:43 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--general_gain"
  FREQ_OFFSET=""
else
  gain_option="--gain"
  ppm_correction="--ppm_correction"
fi

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7452 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --general_gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
^CINFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm & offset was not passed

[14:11:17 - 13/08/2024] (D) Parameters :
[14:11:17 - 13/08/2024] (D)    - align_timestamps : true
[14:11:17 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:11:17 - 13/08/2024] (D)    - autocrop_wedges : true
[14:11:17 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:11:17 - 13/08/2024] (D)    - buffer_size : 1000000
[14:11:17 - 13/08/2024] (D)    - frequency : 137912500
[14:11:17 - 13/08/2024] (D)    - general_gain : 49.6
[14:11:17 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:11:17 - 13/08/2024] (D)    - samplerate : 1024000

I modified receive_metoer.sh and performed the same testing

/home/richard/raspberry-noaa-v2/scripts/receive_meteor.sh "METEOR-M2 4" METEOR-M2-4-20240813-194754 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723578474 3 83 Northbound W

I tested it for M2-4  and you can see PPM and Offset was passed to satdump as desired:

INFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7462 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9000e6   --gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm was passed

[14:32:22 - 13/08/2024] (D) Parameters :
[14:32:22 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:32:22 - 13/08/2024] (D)    - buffer_size : 1000000
[14:32:22 - 13/08/2024] (D)    - cadu_size : 8192
[14:32:22 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:32:22 - 13/08/2024] (D)    - derandomize : true
[14:32:22 - 13/08/2024] (D)    - fill_missing : true
[14:32:22 - 13/08/2024] (D)    - frequency : 137900000
[14:32:22 - 13/08/2024] (D)    - gain : 49.6
[14:32:22 - 13/08/2024] (D)    - nrzm : true
[14:32:22 - 13/08/2024] (D)    - ppm_correction : 0
[14:32:22 - 13/08/2024] (D)    - rs_dualbasis : false
[14:32:22 - 13/08/2024] (D)    - rs_i : 4
[14:32:22 - 13/08/2024] (D)    - rs_type : "rs223"
[14:32:22 - 13/08/2024] (D)    - rs_usecheck : true
[14:32:22 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

NFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7463 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9000e6   --general_gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm & offset was not passed

[14:35:06 - 13/08/2024] (D) Parameters :
[14:35:06 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:35:06 - 13/08/2024] (D)    - buffer_size : 1000000
[14:35:06 - 13/08/2024] (D)    - cadu_size : 8192
[14:35:06 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:35:06 - 13/08/2024] (D)    - derandomize : true
[14:35:06 - 13/08/2024] (D)    - fill_missing : true
[14:35:06 - 13/08/2024] (D)    - frequency : 137900000
[14:35:06 - 13/08/2024] (D)    - general_gain : 49.6
[14:35:06 - 13/08/2024] (D)    - nrzm : true
[14:35:06 - 13/08/2024] (D)    - rs_dualbasis : false
[14:35:06 - 13/08/2024] (D)    - rs_i : 4
[14:35:06 - 13/08/2024] (D)    - rs_type : "rs223"
[14:35:06 - 13/08/2024] (D)    - rs_usecheck : true
[14:35:06 - 13/08/2024] (D)    - samplerate : 1024000

* Ensure ownership/permissions are set correctly for key scripts

* Ensure ownership/permissions are set correctly for key scripts

---------

Co-authored-by: MihajloPi - YU4MPI <[email protected]>

* Changing config directory ownership as root

* Removing whitespaces

* Ooops...

* Allow odd timezones in sun.py (#1056)

* fixes #1021

* fixes #1055

* Fix deprecation warning in heatmap.py (#1059)

* fixes #1021

* fixes deprecation message for getsize and ANTIALIAS (will be removed in Pillow 10)

* Beta development - Update dependencies/yml - added become: yes for .config ownership (#1057)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $H…
MihajloPi added a commit that referenced this issue Sep 20, 2024
… // Inspire-Sat 7 having been retired (#1073) (#1074)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* aarch64/arm64 support and general fixes (#1029)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for SatDump Meteor images

* Fixing filling issue for Meteor when decoding with SatDump

* Update README.md (#1037)

* Update core.yml - Simply getting OS Release (#1038)

Update core.yml - Simply getting OS Release 

Goal was to work on as many as Debian Distro's as possible even if they are using local repo's

* Update README.md (#1043)

* Update rn2_uninstall.sh (#1045)

Commenting out systemctl disable service because disable causes it to remain disabled even after the service is removed and reinstalled by Ansible 
	
Commenting out the disable command has no impact since it will be removed when removing packages

* Update verification.sh (#1046)

Ooops... I had hard coded /home/richard , fixed with ${HOME}

* Beta development - Setting SDR Device ID, Fixed Receive scripts to pass PPM Offset, Fixed dependancies to work on OS Lite versions (#1049)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Fix to be tested

* Fix to be tested for RTL-SDR PPM correction (#1051)

* Fixing filling issue for SatDump Meteor images

* Fix to be tested

* Beta development - modified receive scripts to pass PPM & Offset based on receiver type (#1052)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Reverting PPM changes to receive scripts

ppm_correction should only be used for RTPSDR and RTL-TCP, there may be other receivers that can use it but unknown

* Update noaa-v2.conf.j2

This was left out of ansible/roles/common/templates/noaa-v2.conf.j2 template file which caused an error for anyone enabling it.

# Coronal mass ejection activity
ENABLE_CORONAL={{ enable_coronal|lower }}

* modified receive scripts to pass PPM & Offset based on receiver type

Modified receivenoaa.sh & receive_meteor.sh scripts to pass PPM & Offset based on receiver type.

Also added a log statement so everybody can see whats getting sent to satdump from output.log

/home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E

# I added setting FREQ_OFFSET=""  when its not an rtlsdr receiver to prevent the Offset being passed to satdump

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--gain"
  ppm_correction="--ppm_correction"
else
  gain_option="--general_gain"
  FREQ_OFFSET=""
fi

I tested it for NOAA 18 and you can see PPM and Offset was passed to satdump as desired:

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7454 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
INFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm was passed

14:08:43 - 13/08/2024] (D) Parameters :
[14:08:43 - 13/08/2024] (D)    - align_timestamps : true
[14:08:43 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:08:43 - 13/08/2024] (D)    - autocrop_wedges : true
[14:08:43 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:08:43 - 13/08/2024] (D)    - buffer_size : 1000000
[14:08:43 - 13/08/2024] (D)    - frequency : 137912500
[14:08:43 - 13/08/2024] (D)    - gain : 49.6
[14:08:43 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:08:43 - 13/08/2024] (D)    - ppm_correction : 0
[14:08:43 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--general_gain"
  FREQ_OFFSET=""
else
  gain_option="--gain"
  ppm_correction="--ppm_correction"
fi

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7452 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --general_gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
^CINFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm & offset was not passed

[14:11:17 - 13/08/2024] (D) Parameters :
[14:11:17 - 13/08/2024] (D)    - align_timestamps : true
[14:11:17 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:11:17 - 13/08/2024] (D)    - autocrop_wedges : true
[14:11:17 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:11:17 - 13/08/2024] (D)    - buffer_size : 1000000
[14:11:17 - 13/08/2024] (D)    - frequency : 137912500
[14:11:17 - 13/08/2024] (D)    - general_gain : 49.6
[14:11:17 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:11:17 - 13/08/2024] (D)    - samplerate : 1024000

I modified receive_metoer.sh and performed the same testing

/home/richard/raspberry-noaa-v2/scripts/receive_meteor.sh "METEOR-M2 4" METEOR-M2-4-20240813-194754 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723578474 3 83 Northbound W

I tested it for M2-4  and you can see PPM and Offset was passed to satdump as desired:

INFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7462 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9000e6   --gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm was passed

[14:32:22 - 13/08/2024] (D) Parameters :
[14:32:22 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:32:22 - 13/08/2024] (D)    - buffer_size : 1000000
[14:32:22 - 13/08/2024] (D)    - cadu_size : 8192
[14:32:22 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:32:22 - 13/08/2024] (D)    - derandomize : true
[14:32:22 - 13/08/2024] (D)    - fill_missing : true
[14:32:22 - 13/08/2024] (D)    - frequency : 137900000
[14:32:22 - 13/08/2024] (D)    - gain : 49.6
[14:32:22 - 13/08/2024] (D)    - nrzm : true
[14:32:22 - 13/08/2024] (D)    - ppm_correction : 0
[14:32:22 - 13/08/2024] (D)    - rs_dualbasis : false
[14:32:22 - 13/08/2024] (D)    - rs_i : 4
[14:32:22 - 13/08/2024] (D)    - rs_type : "rs223"
[14:32:22 - 13/08/2024] (D)    - rs_usecheck : true
[14:32:22 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

NFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7463 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9000e6   --general_gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm & offset was not passed

[14:35:06 - 13/08/2024] (D) Parameters :
[14:35:06 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:35:06 - 13/08/2024] (D)    - buffer_size : 1000000
[14:35:06 - 13/08/2024] (D)    - cadu_size : 8192
[14:35:06 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:35:06 - 13/08/2024] (D)    - derandomize : true
[14:35:06 - 13/08/2024] (D)    - fill_missing : true
[14:35:06 - 13/08/2024] (D)    - frequency : 137900000
[14:35:06 - 13/08/2024] (D)    - general_gain : 49.6
[14:35:06 - 13/08/2024] (D)    - nrzm : true
[14:35:06 - 13/08/2024] (D)    - rs_dualbasis : false
[14:35:06 - 13/08/2024] (D)    - rs_i : 4
[14:35:06 - 13/08/2024] (D)    - rs_type : "rs223"
[14:35:06 - 13/08/2024] (D)    - rs_usecheck : true
[14:35:06 - 13/08/2024] (D)    - samplerate : 1024000

* Ensure ownership/permissions are set correctly for key scripts

* Ensure ownership/permissions are set correctly for key scripts

---------

Co-authored-by: MihajloPi - YU4MPI <[email protected]>

* Changing config directory ownership as root

* Removing whitespaces

* Ooops...

* Allow odd timezones in sun.py (#1056)

* fixes #1021

* fixes #1055

* Fix deprecation warning in heatmap.py (#1059)

* fixes #1021

* fixes deprecation message for getsize and ANTIALIAS (will be removed in Pillow 10)

* Beta development - Update dependencies/yml - added become: yes for .config ownership (#1057)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs ha…
MihajloPi added a commit that referenced this issue Oct 26, 2024
* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

* aarch64/arm64 support and general fixes (#1029)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Sunsetting legacy Debian 32 bit and Debian 64 bit support

* aarch64 support (#1031)

* Add toggle for coronal mass GIFs

* aarch64 support

Fixed typo in README
Modified dependencies.yml to support 64-bit Predict and Satdump
New script strach_perms.sh to correct /run/user/${UID} after reboot
New script set_os_config_kernel8.sh to modify kernel so wxtoimg will run
New DEB file for 64-bit predict
New DEB file for 64-bit satdump

* SatDump update for x64 machines

* Modified tools & dependancies

Modified dependancies in support of :
wxtoimg libraries on ARM64
wxtoimg symbolic link for ARM64
predict on ARM64

Modified tasks in support of :
wxtoimg kernel8
Fixing /run/user/uid permissions on reboot

* Disabled - name: copy RTL-SDR udev rule

We have an intermittant conflict from dualing UDEV rule files. They are identifical, but sometimes cause Satdump not to be able to open RTL-SDR receiver when executed as a non-root user.

/usr/lib/udev/rules.d/60-librtlsdr0.rules is installed from OSMON's it clone RTL-SDR software" , which puts /lib/udev/rules.d/60-librtlsdr0.rules in place

* rework

* Added --fill_missing option to NOAA & METEOR receive scripts

* undo "fill_missing" option from NOAA as its for GOES and METEOR only

* Hmmm somehow `--fill_missing` got lost when I switched to SatDump live decoding

* Adding updated SatDump for armhf built by Richard

* Providing support for any Debian based distro

* Setting automatic deletion to false to store 3 days worth of audio for archiving and debugging

* Fixing the log for the breakpoint

* Fixing the log for the break point

* Preventing invalid receiver_type and satellite decoders from going past the install script

* Adding fix to prevent Gtk-WARNING

Modified schedule.sh to unset XTERM variables which would cause NOAA annotations to fail when passes were scheduled via VNC using lxterminal.

* Widening the scope of variables to  omit when scheduling passes

* Added WAYFIRE to omit

* Update receive_meteor.sh

Backing out --fill_missing to avoid merge conflict with parent

* Update receive_noaa.sh - Backing out changes to avoid merge conflict

* add bottom - next/previous page in CAPTURE

https://github.com/jekhokie/raspberry-noaa-v2/issues/1002

* Ooops... Thanks Jerome!

* Updating comments

* Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images

Modified polar_plot.py to resolve warning when generating AZ/EL & Direction images.

Example of warnings...

UserWarning: marker is redundantly defined by the 'marker' keyword argument and the fmt string "o" (-> marker='o'). The keyword argument will take precedence.
  p.plot(az_at_max_elev, max_elev, 'o', marker="*", markersize=12, label="Max El")

* Filter out wxtoimg error messages for known/acceptable issue

Since the source code to WXTOIMG is not available I used gdc (w/o symbols) and strace and see what is causing the 'free invalid pointer' error. I determined that it happens on application clean up after successfully processing the map overlay file into the final image. Everything is working correctly except the clean up of a 3rd memory location which is failing because it was already removed by the kernel I believe.

strace snippet:
munmap(0xef6b8000, 20533248)            = 0
munmap(0xee323000, 20533248)            = 0
munmap(0xecf8e000, 20533248)            = 0
writev(2, [{iov_base="free(): invalid pointer", iov_len=23}, {iov_base="\n", iov_len=1}], 2) = 24

The first two munmap calls worked and the third munmap fails as its already deallocated.

The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. The region is also automatically unmapped when the process is terminated. On the other hand, closing the file descriptor does not unmap the region.

At this point I'm happy to know it successfully allocated those memory slots for processing

mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xef6b8000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xee323000
mmap2(NULL, 20533248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xecf8e000

 but it was freeing up memory on the last address that it threw the error. This modification to receive_noaa.sh is to filter out the 'free invalid pointer' and 'Aborted' messages before its written out to the RN2 log

* Filter out wxtoimg error messages for known/acceptable issue

* Modified dependencies to support 32-bit wxtoimg on arm64 & amd64

* Modified core.yml to support various flavors of Linux

* Removed wxtoimg 320bit support for AMD64 since we have a DEB for it already

* Update README.md

* Tuning Meteor M2-4 maps in MeteorDemod

* Define XDG_RUNTIME_DIR with proper permissions

Define XDG_RUNTIME_DIR with proper permissions

* Introduction of Verification Tool and RN2 uninstall script (#1019)

* Introduction of verification tool

The verification tool can be used to help identify RN2 installation/configuration issues which may potentially prevent proper functioning f capture/decode/processing of APT telemetry data.

Execute the verification script by passing the required argument [ quick | full ]

$HOME/raspberry-noaa-v2/scripts/tools/verification.sh

Argument required:  ./verification.sh quick    or    ./verification.sh full
                        (~ 1 minute)                       (~ 5 minutes)

#########
# NOTES #
#########

# Dryrun of binaries includes executing :

  nxing web page returned 200 OK status to confirm Web Portal is up.

  satdump live capture for 1 second to ensure it runs without error.

  wxmap generates an overlay map image which can be found       : $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-map-output.png

  wxtoimg generates MCIR enhanced image which can be founnd     :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/wxtoimg-mcir-output.jpg

  meteordemod -h is executed to ensure it runs without error.

# When FULL mode is choosen meterdemod fully decodes a staged cadu file :

  meteordemod generates a full set of images which can be found :  $HOME/raspberry-noaa-v2/scripts/tools/verification_tool/test_files/tmp

* Set 755 on verification.sh

Set 755 on verification.sh

* Ensure meteordemond working directory exists

* Set 755 on build_permissions_list.sh

* Update README.md

Introduction of new Verification Tool

* Update README.md

Reformatting Troubleshooting section from lastchange

* Update README.md

* Script to remove RN2 installation

* updated to shutdown nginx before removal

* stop php service

* Corrected php service name to stop when removing RN2

* further cleanup of services

* Tweaked some of the logging messages

* updated to remove rtl-sdr package

* install script does not install RTL-SDR when it finds source in /tmp

* Update caution_uninstall_rn2.sh

* Clean up the library cache so any dangling items are cleared & RTL-SDR changes since it is not a APT installed package (#1020)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Delete db backup files that are older than 3 days

* Removing old PHP thumbnail selection for Meteor (which didn't work)

* Squashed commit of the following:

commit 806663a83a01c91a74f214a15d9b2d7b2500f1e7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:41:41 2024 -0400

    Corrected variable name

commit cd010e0053f8dbe32421e65bd12dae501f879978
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 07:15:51 2024 -0400

    Since verifcation tool is added to install script, conditionally execute from upgrade script

commit 8c6ea4e755f5ff469b1463eacb10f6a5f9cb70b7
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Tue Jul 30 06:38:18 2024 -0400

    Removed /etc/modprobe.d/rtlsdr.conf

commit a5b7821479bf74f9b7c0d0a67998424fde30e642
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 21:24:29 2024 -0400

    Updated to remove reboot dependency and execute Verification Tool

    Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

commit 3c50ae76316a823174ced50e0d68c1bbf2189949
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:17:12 2024 -0400

    Added error handling and Improved text in echo for moving RN2 tree

commit b375303f75d62f22d852cfd18162fc9bebe1fcdd
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 17:07:59 2024 -0400

    Moved git clone command to start of script

    Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

commit 5552f61a62d15f7f9413a6b8b1d99336a2a9c135
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 12:03:14 2024 -0400

    Stop and RN2 AT processes that might be running during uninstall

    # kill any RN2 AT processes as part of clean up, otherwise danglers
    # may have the RTL-SDR open and verification dry run test for satdump will fail

commit edd0f4ef45b20aceeccc708729023fb5d9afbc8e
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:36:23 2024 -0400

    Corrected /srv restore/Unstage permissions

commit 7d91bd16a8cb249e3519b474ea0561624f7d82b4
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 11:18:53 2024 -0400

    Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

commit 8c0c56d2ba6ead61105450204862fa665116bf9b
Merge: a007035 bb0d497
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:49:02 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit a00703542ffb999d18f5fe423971e96782408066
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:48:21 2024 -0400

    Added tee -a  for appending to log file

commit bb0d497e96c28a9e260439e3bc46e11766a644d9
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:27:38 2024 -0400

    Resolve merge conflict

commit a83fda3a95650a5906deeb03b8e8a1851a257ce9
Merge: acb9fc5 3fd34b1
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:16:48 2024 -0400

    Merge branch 'aarch64-support' of https://github.com/creas002/raspberry-noaa-v2 into aarch64-support

commit acb9fc53b8462b90561532eed28bb110efe4a6f2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:14:45 2024 -0400

    Script was rename to rn2_uninstall.sh and moved to .rm2_utils

commit 3fd34b17f5d79f00e2ec9bb24ade1cc958bf3ee2
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 10:08:22 2024 -0400

    Update README.md - In-Situ Upgrade

commit 4f97fae0760b4fd1f0d02078bda68a8d495cc161
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:55:59 2024 -0400

    Introduction of In-situ RN2 upgrade

    #          1) Backup/Stage RN2 key directories
    #          2) Uninstall existing RN2 installation
    #          3) git clone new repository
    #          4) Restore/UnStage RN2 key directories
    #          5) install_and_upgrade
    #          6) verification tool

commit b252da5d5319a318af317b0eb5b124733f5da073
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Mon Jul 29 09:10:58 2024 -0400

    Fix -  DB Backup directory not cleaned up #1021

commit ad5555d760191e5e4e37cf7508b5d6f57de5e2cb
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 11:43:43 2024 -0400

    RTL-SDR changes since it is not a APT installed package

commit d967486f8a9f5e1e98f7bacffdf995af52974121
Author: Richard Creasey - AI4Y <[email protected]>
Date:   Sun Jul 21 09:22:53 2024 -0400

    Clean up the library cache so any dangling items are cleared

* Introduction of RN2 In-Situ Upgrade (#1024)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log to support.sh (#1028)

* Clean up the library cache so any dangling items are cleared

* RTL-SDR changes since it is not a APT installed package

* Fix -  DB Backup directory not cleaned up #1021

* Introduction of In-situ RN2 upgrade

#          1) Backup/Stage RN2 key directories
#          2) Uninstall existing RN2 installation
#          3) git clone new repository
#          4) Restore/UnStage RN2 key directories
#          5) install_and_upgrade
#          6) verification tool

* Update README.md - In-Situ Upgrade

* Script was rename to rn2_uninstall.sh and moved to .rm2_utils

* Resolve merge conflict

* Added tee -a  for appending to log file

* Script was renamed to rn2_uninstall.sh and moved to .rn2_utils

* Corrected /srv restore/Unstage permissions

* Stop and RN2 AT processes that might be running during uninstall

# kill any RN2 AT processes as part of clean up, otherwise danglers
# may have the RTL-SDR open and verification dry run test for satdump will fail

* Moved git clone command to start of script

Moved git clone command to start of script before destructive tasks are performed in case user passed a bad URL or Branch name

* Added error handling and Improved text in echo for moving RN2 tree

* Updated to remove reboot dependency and execute Verification Tool

Verification script had 1 failure after Fresh OS install and install_and_upgrade.sh executed. Reboot would resolve the USB udev permission issue, but reloading udev rules and trigger resolves it without a reboot.  I left the code block to execute verification tool on fresh installs, so it only happens the first time. Users can call Verification Tool anytime the need to such as part of support.sh script.

* Removed /etc/modprobe.d/rtlsdr.conf

* Since verifcation tool is added to install script, conditionally execute from upgrade script

* Corrected variable name

* Added git config, verification tool and tee to log

---------

Co-authored-by: Sasszem <[email protected]>
Co-authored-by: Richard Creasey - AI4Y <[email protected]>
Co-authored-by: Jérôme <[email protected]>

* Rebooting the Pi automatically after the first install

* Fixing filling issue for SatDump Meteor images

* Fixing filling issue for Meteor when decoding with SatDump

* Update README.md (#1037)

* Update core.yml - Simply getting OS Release (#1038)

Update core.yml - Simply getting OS Release 

Goal was to work on as many as Debian Distro's as possible even if they are using local repo's

* Update README.md (#1043)

* Update rn2_uninstall.sh (#1045)

Commenting out systemctl disable service because disable causes it to remain disabled even after the service is removed and reinstalled by Ansible 
	
Commenting out the disable command has no impact since it will be removed when removing packages

* Update verification.sh (#1046)

Ooops... I had hard coded /home/richard , fixed with ${HOME}

* Beta development - Setting SDR Device ID, Fixed Receive scripts to pass PPM Offset, Fixed dependancies to work on OS Lite versions (#1049)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Fix to be tested

* Fix to be tested for RTL-SDR PPM correction (#1051)

* Fixing filling issue for SatDump Meteor images

* Fix to be tested

* Beta development - modified receive scripts to pass PPM & Offset based on receiver type (#1052)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not used.

As part of this clean up the verification tool permissions list has this directory and is sub directory removed from being checked.

* Removed unneeded task

* Reverting PPM changes to receive scripts

ppm_correction should only be used for RTPSDR and RTL-TCP, there may be other receivers that can use it but unknown

* Update noaa-v2.conf.j2

This was left out of ansible/roles/common/templates/noaa-v2.conf.j2 template file which caused an error for anyone enabling it.

# Coronal mass ejection activity
ENABLE_CORONAL={{ enable_coronal|lower }}

* modified receive scripts to pass PPM & Offset based on receiver type

Modified receivenoaa.sh & receive_meteor.sh scripts to pass PPM & Offset based on receiver type.

Also added a log statement so everybody can see whats getting sent to satdump from output.log

/home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E

# I added setting FREQ_OFFSET=""  when its not an rtlsdr receiver to prevent the Offset being passed to satdump

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--gain"
  ppm_correction="--ppm_correction"
else
  gain_option="--general_gain"
  FREQ_OFFSET=""
fi

I tested it for NOAA 18 and you can see PPM and Offset was passed to satdump as desired:

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7454 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
INFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm was passed

14:08:43 - 13/08/2024] (D) Parameters :
[14:08:43 - 13/08/2024] (D)    - align_timestamps : true
[14:08:43 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:08:43 - 13/08/2024] (D)    - autocrop_wedges : true
[14:08:43 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:08:43 - 13/08/2024] (D)    - buffer_size : 1000000
[14:08:43 - 13/08/2024] (D)    - frequency : 137912500
[14:08:43 - 13/08/2024] (D)    - gain : 49.6
[14:08:43 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:08:43 - 13/08/2024] (D)    - ppm_correction : 0
[14:08:43 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

if [[ "$receiver" == "rtlsdr" ]]; then
  gain_option="--general_gain"
  FREQ_OFFSET=""
else
  gain_option="--gain"
  ppm_correction="--ppm_correction"
fi

richard@raspberrypi:~$ /home/richard/raspberry-noaa-v2/scripts/receive_noaa.sh "NOAA 18" NOAA-18-20240813-153343 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723563223 3 32 Southbound E
INFO : The system has enough space to store a NOAA pass on RAM
INFO : Free : 7452 ; Required : 50
INFO : Direction Southbound
INFO : Cropping SatDump NOAA images enabled
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at  MHz via SatDump live pipeline
INFO : /usr/bin/satdump live noaa_apt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9125e6 --satellite_number 18 --sdrpp_noise_reduction   --general_gain 49.6  --autocrop_wedges --start_timestamp 1723563313 --save_wav  --timeout 3
^CINFO : Files recorded
INFO : Resampling down audio
INFO : Producing pristine image
INFO : Bulding pass map

Satdump log shows ppm & offset was not passed

[14:11:17 - 13/08/2024] (D) Parameters :
[14:11:17 - 13/08/2024] (D)    - align_timestamps : true
[14:11:17 - 13/08/2024] (D)    - audio_samplerate : 50000.0
[14:11:17 - 13/08/2024] (D)    - autocrop_wedges : true
[14:11:17 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:11:17 - 13/08/2024] (D)    - buffer_size : 1000000
[14:11:17 - 13/08/2024] (D)    - frequency : 137912500
[14:11:17 - 13/08/2024] (D)    - general_gain : 49.6
[14:11:17 - 13/08/2024] (D)    - max_crop_stddev : 3500
[14:11:17 - 13/08/2024] (D)    - samplerate : 1024000

I modified receive_metoer.sh and performed the same testing

/home/richard/raspberry-noaa-v2/scripts/receive_meteor.sh "METEOR-M2 4" METEOR-M2-4-20240813-194754 /home/richard/raspberry-noaa-v2/tmp/orbit.tle 1723578474 3 83 Northbound W

I tested it for M2-4  and you can see PPM and Offset was passed to satdump as desired:

INFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7462 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6 --ppm_correction 0 --frequency 137.9000e6   --gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm was passed

[14:32:22 - 13/08/2024] (D) Parameters :
[14:32:22 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:32:22 - 13/08/2024] (D)    - buffer_size : 1000000
[14:32:22 - 13/08/2024] (D)    - cadu_size : 8192
[14:32:22 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:32:22 - 13/08/2024] (D)    - derandomize : true
[14:32:22 - 13/08/2024] (D)    - fill_missing : true
[14:32:22 - 13/08/2024] (D)    - frequency : 137900000
[14:32:22 - 13/08/2024] (D)    - gain : 49.6
[14:32:22 - 13/08/2024] (D)    - nrzm : true
[14:32:22 - 13/08/2024] (D)    - ppm_correction : 0
[14:32:22 - 13/08/2024] (D)    - rs_dualbasis : false
[14:32:22 - 13/08/2024] (D)    - rs_i : 4
[14:32:22 - 13/08/2024] (D)    - rs_type : "rs223"
[14:32:22 - 13/08/2024] (D)    - rs_usecheck : true
[14:32:22 - 13/08/2024] (D)    - samplerate : 1024000

# Since I don't have another SDR receiver type, I flipped the logic to test that neither --ppm_correction or Offset values were passed to setdump using:

NFO : The system has enough space to store a Meteor pass on RAM
INFO : Free : 7463 ; Required : 10
INFO : Direction Northbound
INFO : I'll flip this image pass because FLIP_METEOR_IMG is set to true and PASS_DIRECTION is Northbound
INFO : Recording /home/richard/raspberry-noaa-v2 via rtlsdr at 137.9000 MHz using SatDump record
INFO : /usr/bin/satdump live meteor_m2-x_lrpt . --source rtlsdr --samplerate 1.024e6   --frequency 137.9000e6   --general_gain 49.6   --fill_missing --timeout 3
INFO : Removing old bmp, gcp, and dat files
INFO : Running MeteorDemod to demodulate OQPSK file, rectify (spread) images, create heat map and composites and convert them to JPG
INFO : Waiting for files to close

Satdump log shows ppm & offset was not passed

[14:35:06 - 13/08/2024] (D) Parameters :
[14:35:06 - 13/08/2024] (D)    - baseband_format : "cf32"
[14:35:06 - 13/08/2024] (D)    - buffer_size : 1000000
[14:35:06 - 13/08/2024] (D)    - cadu_size : 8192
[14:35:06 - 13/08/2024] (D)    - constellation : "oqpsk"
[14:35:06 - 13/08/2024] (D)    - derandomize : true
[14:35:06 - 13/08/2024] (D)    - fill_missing : true
[14:35:06 - 13/08/2024] (D)    - frequency : 137900000
[14:35:06 - 13/08/2024] (D)    - general_gain : 49.6
[14:35:06 - 13/08/2024] (D)    - nrzm : true
[14:35:06 - 13/08/2024] (D)    - rs_dualbasis : false
[14:35:06 - 13/08/2024] (D)    - rs_i : 4
[14:35:06 - 13/08/2024] (D)    - rs_type : "rs223"
[14:35:06 - 13/08/2024] (D)    - rs_usecheck : true
[14:35:06 - 13/08/2024] (D)    - samplerate : 1024000

* Ensure ownership/permissions are set correctly for key scripts

* Ensure ownership/permissions are set correctly for key scripts

---------

Co-authored-by: MihajloPi - YU4MPI <[email protected]>

* Changing config directory ownership as root

* Removing whitespaces

* Ooops...

* Allow odd timezones in sun.py (#1056)

* fixes #1021

* fixes #1055

* Fix deprecation warning in heatmap.py (#1059)

* fixes #1021

* fixes deprecation message for getsize and ANTIALIAS (will be removed in Pillow 10)

* Beta development - Update dependencies/yml - added become: yes for .config ownership (#1057)

* Setting SDR Device ID

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md

* Update setting_sdr_source_id.md - fixed mispelling

* Update setting_sdr_source_id.md - Fixed 2nd typo

* Update setting_sdr_source_id.md - Fixed more typo's (SMH)

* Update setting_sdr_source_id.md - Reformatting

* Update setting_sdr_source_id.md

* beta-development - Pass missing --ppm_correction parameter to satdump

Tested PPM Offset was passed and processed by satdump correctly when receive scripts were called from AT jobs with settings.yml values of :

noaa_15_freq_offset: -2
noaa_18_freq_offset: -1
noaa_19_freq_offset: 0
meteor_m2_3_freq_offset: 1
meteor_m2_4_freq_offset: 2

* beta-development - bug fix - OS Lite versions Bookworm 64 & Bullseye 32

OS Lite versions Bookworm 64 & Bullseye 32 fail on fresh OS install

I have tested this fix on Bookworm 64-bit full & lite as well as Bullseye 32-bit Full/lite

OS Lite versions do not have .config, but when its created as as part of the .config/meteordemod task below
The .config ownership of top directory is root:root 755 when this happens and it needs to be target_user:target_user 700

* Clean up Bogus config/annotation directory created under $HOME

RN2 installs have been placing a second config/annotation directory under ${HOME} which is not us…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants