Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug]: UCP unable to connect to the node server (xhr poll error) #576

Open
va7wv opened this issue Dec 3, 2024 · 11 comments
Open

[bug]: UCP unable to connect to the node server (xhr poll error) #576

va7wv opened this issue Dec 3, 2024 · 11 comments
Labels
bug Something isn't working triage Triage

Comments

@va7wv
Copy link

va7wv commented Dec 3, 2024

FreePBX Version

FreePBX 17

Issue Description

This is a new install using the official bash script on Debian. Have tried disabling firewall, restarting mysqld, nothing seems to change condition. Error popup on UCP login page indicates "xhr poll error". Mariadb is running but service shows error. How do I begin to troubleshoot this? Everything else works fine, but I would like to have a working UCP.

Operating Environment

+---------------------+------------+---------+-------------+-----------+
| Module | Version | Status | License | Signature |
+---------------------+------------+---------+-------------+-----------+
| accountcodepreserve | 17.0.0.1 | Enabled | GPLv2 | Sangoma |
| adv_recovery | 17.0.4 | Enabled | Commercial | Sangoma |
| allowlist | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| amd | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| announcement | 17.0.2.1 | Enabled | GPLv3+ | Sangoma |
| api | 17.0.1.6 | Enabled | AGPLv3+ | Sangoma |
| areminder | 17.0.3.10 | Enabled | Commercial | Sangoma |
| arimanager | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| asterisk-cli | 17.0.2 | Enabled | GPLv3+ | Sangoma |
| asteriskinfo | 17.0.2 | Enabled | GPLv3+ | Sangoma |
| backup | 17.0.5.62 | Enabled | GPLv3+ | Sangoma |
| blacklist | 17.0.1.2 | Enabled | GPLv3+ | Sangoma |
| broadcast | 17.0.2 | Enabled | Commercial | Sangoma |
| builtin | | Enabled | | Unsigned |
| bulkhandler | 17.0.5 | Enabled | GPLv3+ | Sangoma |
| calendar | 17.0.4.20 | Enabled | GPLv3+ | Sangoma |
| callaccounting | 17.0.5 | Enabled | Commercial+ | Sangoma |
| callback | 17.0.2.1 | Enabled | GPLv3+ | Sangoma |
| callerid | 17.0.1 | Enabled | Commercial | Sangoma |
| callforward | 17.0.1.7 | Enabled | AGPLv3+ | Sangoma |
| calllimit | 17.0.1.2 | Enabled | Commercial | Sangoma |
| callrecording | 17.0.3.8 | Enabled | AGPLv3+ | Sangoma |
| callwaiting | 17.0.3.4 | Enabled | GPLv3+ | Sangoma |
| cdr | 17.0.4.29 | Enabled | GPLv3+ | Sangoma |
| cdrpro | 17.0.3.24 | Enabled | Commercial | Sangoma |
| cel | 17.0.2.10 | Enabled | GPLv3+ | Sangoma |
| certman | 17.0.3.13 | Enabled | AGPLv3+ | Sangoma |
| cidlookup | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| conferences | 17.0.3.2 | Enabled | GPLv3+ | Sangoma |
| conferencespro | 17.0.1.7 | Enabled | Commercial | Sangoma |
| configedit | 17.0.1.4 | Enabled | AGPLv3+ | Sangoma |
| contactmanager | 17.0.5.12 | Enabled | GPLv3+ | Sangoma |
| core | 17.0.18.7 | Enabled | GPLv3+ | Sangoma |
| cos | 17.0.1.1 | Enabled | Commercial | Sangoma |
| customappsreg | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| customcontexts | 17.0.1.3 | Enabled | GPLv2+ | Sangoma |
| dashboard | 17.0.4.6 | Enabled | AGPLv3+ | Sangoma |
| daynight | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| dictate | 17.0.1.2 | Enabled | GPLv3+ | Sangoma |
| directory | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| disa | 17.0.6 | Enabled | AGPLv3+ | Sangoma |
| donotdisturb | 17.0.2.3 | Enabled | GPLv3+ | Sangoma |
| dynroute | 17.0.3.2 | Enabled | GPLv3+ | Sangoma |
| endpoint | 17.0.1.94 | Enabled | Commercial | Sangoma |
| extensionroutes | 17.0.1 | Enabled | Commercial | Sangoma |
| extensionsettings | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| fax | 17.0.3.4 | Enabled | GPLv3+ | Sangoma |
| faxpro | 17.0.1.19 | Enabled | Commercial | Sangoma |
| featurecodeadmin | 17.0.2 | Enabled | GPLv3+ | Sangoma |
| filestore | 17.0.2.33 | Enabled | AGPLv3 | Sangoma |
| findmefollow | 17.0.4.10 | Enabled | GPLv3+ | Sangoma |
| firewall | 17.0.1.30 | Enabled | AGPLv3+ | Sangoma |
| framework | 17.0.19.21 | Enabled | GPLv2+ | Sangoma |
| hotelwakeup | 17.0.1.6 | Enabled | GPLv2 | Sangoma |
| iaxsettings | 17.0.1 | Enabled | AGPLv3 | Sangoma |
| infoservices | 17.0.1.1 | Enabled | GPLv2+ | Sangoma |
| ivr | 17.0.8 | Enabled | GPLv3+ | Sangoma |
| languages | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| logfiles | 17.0.3.3 | Enabled | GPLv3+ | Sangoma |
| manager | 17.0.6 | Enabled | GPLv2+ | Sangoma |
| miscapps | 17.0.3 | Enabled | GPLv3+ | Sangoma |
| miscdests | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| missedcall | 17.0.2 | Enabled | GPLv3+ | Sangoma |
| music | 17.0.5 | Enabled | GPLv3+ | Sangoma |
| oracle_connector | 17.0.1.1 | Enabled | Commercial | Sangoma |
| outcnam | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| outroutemsg | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| paging | 17.0.3 | Enabled | GPLv3+ | Sangoma |
| pagingpro | 17.0.1.6 | Enabled | Commercial | Sangoma |
| parking | 17.0.2.5 | Enabled | GPLv3+ | Sangoma |
| parkpro | 17.0.1.4 | Enabled | Commercial | Sangoma |
| pbxmfa | 17.0.2.3 | Enabled | Commercial+ | Sangoma |
| phpinfo | 17.0.1 | Enabled | GPLv2+ | Sangoma |
| pinsets | 17.0.3.2 | Enabled | GPLv3+ | Sangoma |
| pinsetspro | 17.0.2 | Enabled | Commercial | Sangoma |
| pm2 | 17.0.3.3 | Enabled | AGPLv3+ | Sangoma |
| pms | 17.0.2.30 | Enabled | Commercial | Sangoma |
| presencestate | 17.0.2.4 | Enabled | GPLv3+ | Sangoma |
| printextensions | 17.0.1.2 | Enabled | GPLv3+ | Sangoma |
| queueprio | 17.0.1.4 | Enabled | GPLv3+ | Sangoma |
| queues | 17.0.1.13 | Enabled | GPLv2+ | Sangoma |
| queuestats | 17.0.1.7 | Enabled | Commercial | Sangoma |
| qxact_reports | 17.0.3 | Enabled | Commercial | Sangoma |
| recording_report | 17.0.3.8 | Enabled | Commercial | Sangoma |
| recordings | 17.0.2.3 | Enabled | GPLv3+ | Sangoma |
| restapps | 17.0.1.27 | Enabled | Commercial | Sangoma |
| ringgroups | 17.0.2.6 | Enabled | GPLv3+ | Sangoma |
| sangomaconnect | 17.0.1.43 | Enabled | Commercial | Sangoma |
| sangomacrm | 17.0.1.17 | Enabled | Commercial | Sangoma |
| sangomartapi | 17.0.2.18 | Enabled | Commercial | Sangoma |
| setcid | 17.0.1.2 | Enabled | GPLv3+ | Sangoma |
| sipsettings | 17.0.6.9 | Enabled | AGPLv3+ | Sangoma |
| sipstation | 17.0.3.4 | Enabled | Commercial | Sangoma |
| sms | 17.0.1.15 | Enabled | Commercial | Sangoma |
| smsplus | 17.0.3 | Enabled | Commercial | Sangoma |
| soundlang | 17.0.4.1 | Enabled | GPLv3+ | Sangoma |
| superfecta | 17.0.3.5 | Enabled | GPLv2+ | Sangoma |
| sysadmin | 17.0.2.6 | Enabled | Commercial | Sangoma |
| timeconditions | 17.0.1.18 | Enabled | GPLv3+ | Sangoma |
| tts | 17.0.1.1 | Enabled | GPLv3+ | Sangoma |
| ttsengines | 17.0.1 | Enabled | AGPLv3 | Sangoma |
| ucp | 17.0.4.23 | Enabled | AGPLv3+ | Sangoma |
| userman | 17.0.6.30 | Enabled | AGPLv3+ | Sangoma |
| vmblast | 17.0.2 | Enabled | GPLv3+ | Sangoma |
| vmnotify | 17.0.1.7 | Enabled | Commercial | Sangoma |
| voicemail | 17.0.5.26 | Enabled | GPLv3+ | Sangoma |
| voicemail_report | 17.0.1.1 | Enabled | Commercial | Sangoma |
| voipinnovations | 17.0.1.4 | Enabled | Commercial | Sangoma |
| vqplus | 17.0.1.17 | Enabled | Commercial | Sangoma |
| weakpasswords | 17.0.1 | Enabled | GPLv3+ | Sangoma |
| webcallback | 17.0.4 | Enabled | Commercial | Sangoma |
| webrtc | 17.0.2.2 | Enabled | GPLv3+ | Sangoma |
+---------------------+------------+---------+-------------+-----------+

Relevant log output

2024-12-02 15:20 -08:00: There was an error with MySQL Connection
2024-12-02 15:21 -08:00: SqlError: (conn:39, no: 45009, SQLState: 08S01) socket has unexpectedly been closed
2024-12-02 15:21 -08:00:     at module.exports.createFatalError (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/misc/errors.js:90:10)
2024-12-02 15:21 -08:00:     at Connection.socketErrorHandler (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/connection.js:1193:20)
2024-12-02 15:21 -08:00:     at Socket.emit (node:events:529:35)
2024-12-02 15:21 -08:00:     at endReadableNT (node:internal/streams/readable:1400:12)
2024-12-02 15:21 -08:00:     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-12-02 15:21 -08:00:   sqlMessage: 'socket has unexpectedly been closed',
2024-12-02 15:21 -08:00:   sql: null,
2024-12-02 15:21 -08:00:   fatal: true,
2024-12-02 15:21 -08:00:   errno: 45009,
2024-12-02 15:21 -08:00:   sqlState: '08S01',
2024-12-02 15:21 -08:00:   code: 'ER_SOCKET_UNEXPECTED_CLOSE'
2024-12-02 15:21 -08:00: }
2024-12-02 15:21 -08:00: Error: connect ECONNREFUSED 127.0.0.1:3306
2024-12-02 15:21 -08:00:     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1549:16)
2024-12-02 15:21 -08:00:  From event:
2024-12-02 15:21 -08:00:     at /var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/connection.js:142:13
2024-12-02 15:21 -08:00:     at new Promise (<anonymous>)
2024-12-02 15:21 -08:00:     at Connection.connect (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/connection.js:131:12)
2024-12-02 15:21 -08:00:     at Object.createConnection (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/promise.js:37:17)
2024-12-02 15:21 -08:00:     at connect2database (/var/www/html/admin/modules/ucp/node/lib/freepbx.js:74:25)
2024-12-02 15:21 -08:00:     at /var/www/html/admin/modules/ucp/node/lib/freepbx.js:35:4
2024-12-02 15:21 -08:00:     at ChildProcess.exithandler (node:child_process:414:7)
2024-12-02 15:21 -08:00:     at ChildProcess.emit (node:events:517:28)
2024-12-02 15:21 -08:00:     at maybeClose (node:internal/child_process:1098:16)
2024-12-02 15:21 -08:00:     at ChildProcess._handle.onexit (node:internal/child_process:303:5) {
2024-12-02 15:21 -08:00:   errno: -111,
2024-12-02 15:21 -08:00:   code: 'ECONNREFUSED',
2024-12-02 15:21 -08:00:   syscall: 'connect',
2024-12-02 15:21 -08:00:   address: '127.0.0.1',
2024-12-02 15:21 -08:00:   port: 3306,
2024-12-02 15:21 -08:00:   fatal: true,
2024-12-02 15:21 -08:00:   sqlState: 'HY000'
2024-12-02 15:21 -08:00: }
2024-12-02 15:21 -08:00: There was an error with MySQL Connection

● mariadb.service - MariaDB 10.11.6 database server
     Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-12-02 15:21:35 PST; 17h ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
    Process: 245858 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
    Process: 245860 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
    Process: 245862 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_P>
    Process: 245932 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
    Process: 245936 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS)
   Main PID: 245921 (mariadbd)
     Status: "Taking your SQL requests now..."
      Tasks: 23 (limit: 4644)
     Memory: 256.8M
        CPU: 3min 26.170s
     CGroup: /system.slice/mariadb.service
             └─245921 /usr/sbin/mariadbd

Dec 02 15:21:35 freepbx mariadbd[245921]: 2024-12-02 15:21:35 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
Dec 02 15:21:35 freepbx mariadbd[245921]: 2024-12-02 15:21:35 0 [Note] /usr/sbin/mariadbd: ready for connections.
Dec 02 15:21:35 freepbx mariadbd[245921]: Version: '10.11.6-MariaDB-0+deb12u1'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 12
Dec 02 15:21:35 freepbx systemd[1]: Started mariadb.service - MariaDB 10.11.6 database server.
Dec 02 15:21:35 freepbx /etc/mysql/debian-start[245938]: Upgrading MySQL tables if necessary.
Dec 02 15:21:35 freepbx mariadbd[245921]: 2024-12-02 15:21:35 0 [Note] InnoDB: Buffer pool(s) load completed at 241202 15:21:35
Dec 02 15:21:35 freepbx /etc/mysql/debian-start[245949]: Checking for insecure root accounts.
Dec 02 15:21:35 freepbx /etc/mysql/debian-start[245953]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables
Dec 03 09:06:42 freepbx mariadbd[245921]: 2024-12-03  9:06:42 255 [Warning] Aborted connection 255 to db: 'asterisk' user: 'freepbxuser' host: 'localhost' (Got an error reading communication packet>
Dec 03 09:06:42 freepbx mariadbd[245921]: 2024-12-03  9:06:42 248 [Warning] Aborted connection 248 to db: 'asterisk' user: 'freepbxuser' host: 'localhost' (Got an error reading communication packet>
@va7wv va7wv added bug Something isn't working triage Triage labels Dec 3, 2024
@kguptasangoma
Copy link
Member

hi @va7wv first of all confirm your mariadb service is running properly ?

then check why [127.0.0.1:3306](http://127.0.0.1:3306/) getting connection refused to this IP and port? Please note If you have ipv6 enabled then ensure localhost is resolving to 127.0.0.1.

@va7wv
Copy link
Author

va7wv commented Dec 3, 2024

● mariadb.service - MariaDB 10.11.6 database server
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; preset: enabled)
Active: active (running) since Tue 2024-12-03 09:26:49 PST; 6min ago
Docs: man:mariadbd(8)
https://mariadb.com/kb/en/library/systemd/
Main PID: 725 (mariadbd)
Status: "Taking your SQL requests now..."
Tasks: 30 (limit: 4644)
Memory: 283.0M
CPU: 2.663s
CGroup: /system.slice/mariadb.service
└─725 /usr/sbin/mariadbd

Dec 03 09:26:49 freepbx mariadbd[725]: 2024-12-03 9:26:49 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
Dec 03 09:26:49 freepbx mariadbd[725]: 2024-12-03 9:26:49 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
Dec 03 09:26:49 freepbx mariadbd[725]: 2024-12-03 9:26:49 0 [Note] Server socket created on IP: '127.0.0.1'.
Dec 03 09:26:49 freepbx mariadbd[725]: 2024-12-03 9:26:49 0 [Note] InnoDB: Buffer pool(s) load completed at 241203 9:26:49
Dec 03 09:26:49 freepbx mariadbd[725]: 2024-12-03 9:26:49 0 [Note] /usr/sbin/mariadbd: ready for connections.
Dec 03 09:26:49 freepbx mariadbd[725]: Version: '10.11.6-MariaDB-0+deb12u1' socket: '/run/mysqld/mysqld.sock' port: 3306 Debian 12
Dec 03 09:26:49 freepbx systemd[1]: Started mariadb.service - MariaDB 10.11.6 database server.
Dec 03 09:26:49 freepbx /etc/mysql/debian-start[961]: Upgrading MySQL tables if necessary.
Dec 03 09:26:49 freepbx /etc/mysql/debian-start[992]: Checking for insecure root accounts.
Dec 03 09:26:49 freepbx /etc/mysql/debian-start[1006]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables

@va7wv
Copy link
Author

va7wv commented Dec 3, 2024

IPV6 disabled and not used.

@kguptasangoma
Copy link
Member

May be please try fwconsole restart to ensure all the services starting properly.

just thinking about below error

SqlError: (conn:39, no: 45009, SQLState: 08S01) socket has unexpectedly been closed
2024-12-02 15:21 -08:00:     at module.exports.createFatalError (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/misc/errors.js:90:10)
2024-12-02 15:21 -08:00:     at Connection.socketErrorHandler (/var/www/html/admin/modules/ucp/node/node_modules/mariadb/lib/connection.js:1193:20)
2024-12-02 15:21 -08:00:     at Socket.emit (node:events:529:35)
2024-12-02 15:21 -08:00:     at endReadableNT (node:internal/streams/readable:1400:12)
2024-12-02 15:21 -08:00:     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-12-02 15:21 -08:00:   sqlMessage: 'socket has unexpectedly been closed',
2024-12-02 15:21 -08:00:   sql: null,
2024-12-02 15:21 -08:00:   fatal: true,
2024-12-02 15:21 -08:00:   errno: 45009,
2024-12-02 15:21 -08:00:   sqlState: '08S01',
2024-12-02 15:21 -08:00:   code: 'ER_SOCKET_UNEXPECTED_CLOSE'
2024-12-02 15:21 -08:00: }
2024-12-02 15:21 -08:00: Error: connect ECONNREFUSED 127.0.0.1:3306

@va7wv
Copy link
Author

va7wv commented Dec 3, 2024

No go, unfortunately. I have attempted to restart the system several times as well. Appreciate the suggestion. I am not against spinning up another fresh Debian 12 VM and transferring the settings. Literally all I did was run the official script and setup the IVR though.

@va7wv
Copy link
Author

va7wv commented Dec 4, 2024

Started a new VM with the newest official bash script, transferred the configuration using Backup/restore and same problem.

@kguptasangoma
Copy link
Member

are you seeing any node socket error again on the new system ?

How about just fresh system without backup / restore, does that work fine for you ? just thinking if its due to your system environment/VM issue Vs config issue ?

@va7wv
Copy link
Author

va7wv commented Dec 4, 2024

poll_error

I will try another install in a fresh VM without the backup/restore and report back.

@kguptasangoma
Copy link
Member

yes and if you face any issue on fresh system without your configuration then can you please send me access details to [email protected] (if possible) so i can try to ssh to your system to check what is going on.

thanks

@va7wv
Copy link
Author

va7wv commented Dec 4, 2024

Confirmed. Fresh VM, new install. Added one extension, changed the user password. Go to UCP and immediately fails with same xhr error. No other settings changed at all.
Screenshot_20241204_152751

@kguptasangoma
Copy link
Member

Hi @va7wv is it possible for you to give me ssh access , so i can see what is going on? Send me credentials to [email protected]. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Triage
Projects
None yet
Development

No branches or pull requests

2 participants