Better handling of big log files to avoid crash of the viewer #719

arnowelzel · 2022-05-17T14:16:13Z

Steps to reproduce

Set loglevel to in config/config.php to 0 ('loglevel' => 0).
Execute many actions which will trigger a log message to get written.
Open the log viewer when the logfile has more than 60000 entires.

Expected behaviour

The log viewer displays the log.

If the log is too big to be displayed, truncate to the newest n entries (where n is a value which can be handled by the frontend - for example 1000 or 5000) or paginate the log.

Actual behaviour

The log viewer crashes, so the page will get blank. It's not clear when exactly this happens. I have seen this with a logfile which counts about 62000 entries, but I believe the actual limit is lower and also depends on what browser and runtime environment you use.

Server configuration

Operating system: Ubuntu 20.04

Web server: Apache 2.4.41-4ubuntu3.10

Database: MariaDB 10.5.15

PHP version: 7.4

Nextcloud version: 24.0.0

Where did you install Nextcloud from: nextcloud.com

List of activated apps:

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - admin_audit: 1.14.0
  - apporder: 0.15.0
  - bookmarks: 10.3.1
  - calendar: 3.3.0
  - circles: 24.0.0
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.1.0
  - contactsinteraction: 1.5.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - deck: 1.7.0
  - drawio: 1.0.2
  - external: 4.0.0
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_fulltextsearch: 24.0.0
  - files_fulltextsearch_tesseract: 24.0.0
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - forms: 2.5.0
  - fulltextsearch: 24.0.0
  - fulltextsearch_elasticsearch: 24.0.0
  - gpxpod: 4.3.0
  - keeweb: 0.6.9
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - nextcloud_announcements: 1.13.0
  - notes: 4.3.1
  - notifications: 2.12.0
  - oauth2: 1.12.0
  - onlyoffice: 7.3.4
  - password_policy: 1.14.0
  - phonetrack: 0.7.0
  - photos: 1.6.0
  - polls: 3.6.1
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - quota_warning: 1.14.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - spreed: 14.0.1
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - tasks: 0.14.4
  - text: 3.5.1
  - theming: 1.15.0
  - theming_customcss: 1.11.0
  - twofactor_backupcodes: 1.13.0
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - viewer: 1.8.0
  - workflowengine: 2.6.0
Disabled:
  - encryption: 2.2.0
  - files_external: 1.15.0
  - recommendations: 0.4.0
  - user_ldap
  - weather_status: 1.3.0

Nextcloud configuration:

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.0x0c.de"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "skeletondirectory": "",
        "overwrite.cli.url": "https:\/\/nextcloud.0x0c.de",
        "dbtype": "mysql",
        "version": "24.0.0.12",
        "installed": true,
        "htaccess.RewriteBase": "\/",
        "maintenance": false,
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "knowledgebaseenabled": false,
        "theme": "",
        "simpleSignUpLink.shown": false,
        "loglevel": 2,
        "log_authfailip": true,
        "mysql.utf8mb4": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_smtpauthtype": "LOGIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "trashbin_retention_obligation": "auto, 30",
        "app_install_overwrite": [
            "apporder",
            "deck",
            "keeweb",
            "appointments",
            "drawio",
            "bookmarks",
            "spreed"
        ],
        "has_rebuilt_cache": true,
        "onlyoffice": {
            "verify_peer_off": true
        },
        "default_language": "de",
        "default_phone_region": "DE",
        "check_data_directory_permissions": false,
        "log_rotate_size": 52428800
    }
}

Client configuration

Browser:
Vivaldi 5.2.2623.46 (Stable channel) (64-Bit)

Operating system:
Windows 10

Logs

Nextcloud log (data/owncloud.log)

(not relevant)

Browser log

react-dom.production.min.js:216 Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at t.findDOMNode (react-dom.production.min.js:289:284)
    at t.transition (CSSTransitionGroupChild.js:129:30)
    at t.r.componentWillEnter (CSSTransitionGroupChild.js:97:15)
    at t.s.performEnter (TransitionGroup.js:83:19)
    at TransitionGroup.js:192:21
    at Array.forEach (<anonymous>)
    at t.componentDidUpdate (TransitionGroup.js:191:17)
    at co (react-dom.production.min.js:219:502)
    at xl (react-dom.production.min.js:259:160)
    at t.unstable_runWithPriority (scheduler.production.min.js:18:343)

Uncaught (in promise) Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at t.findDOMNode (react-dom.production.min.js:289:284)
    at t.transition (CSSTransitionGroupChild.js:129:30)
    at t.r.componentWillEnter (CSSTransitionGroupChild.js:97:15)
    at t.s.performEnter (TransitionGroup.js:83:19)
    at TransitionGroup.js:192:21
    at Array.forEach (<anonymous>)
    at t.componentDidUpdate (TransitionGroup.js:191:17)
    at co (react-dom.production.min.js:219:502)
    at xl (react-dom.production.min.js:259:160)
    at t.unstable_runWithPriority (scheduler.production.min.js:18:343)

The text was updated successfully, but these errors were encountered:

adriangalbincea · 2022-05-23T08:37:46Z

I have the same issue, logging page is not loading anything, just spinning forever...

danir-de · 2022-06-01T13:14:54Z

We can trigger the issue when changing log settings or enabling auto refresh.

crazycorndog · 2022-09-06T23:35:32Z

I had the very same issue, logging page was not loading anything, just spinning forever...

Nextcloud 24(.0.4)

the logfile was 24M big, with 4554 lines:

-rw-r----- 1 nextcloud nextcloud 24M 18. Aug 00:18 /var/log/nextcloud/nextcloud.log
cat /var/log/nextcloud/nextcloud.log | wc -l
4554

I renamed the original logfile and touch'ed and chown'ed a new logfile, now the logging page is loading just fine (and saying that there are no log entries yet) but the logger doesnt seem to write anything to the new logfile now, even with debug loglevel enabled, the file is still empty

edit: while executing occ maintenance:mimetype:update-js an error happened in background and got logged in the log file, but still no debug level entries or alike

strugee · 2022-09-23T01:25:34Z

Note: the underlying React error message is Unable to find node on an unmounted component. (mostly adding this so that if folks look up the full React error string and then search GitHub for it, this issue will show up).

luxifr · 2023-01-01T02:40:03Z

same here with

www-data@2d5e879532e7:~/html/data$ ls -lh nextcloud.log
-rw-r----- 1 www-data www-data 3.0M Jan  1 02:39 nextcloud.log
www-data@2d5e879532e7:~/html/data$ wc -l nextcloud.log
2752 nextcloud.log

Antreesy · 2024-09-17T09:40:59Z

Quickly checked at Nextcloud 28 with 10k+ rows in a logfile, requests to logfile are chunked by 50 rows,

no visible freezes with 3k lines shown.
local file opened with 10k lines - quite slow, but manageable

Frontend handling can be improved with virtual scroller 🚀

https://www.npmjs.com/package/vue-virtual-scroller

mritzmann mentioned this issue May 17, 2022

Logging does not load #52

Closed

arnowelzel mentioned this issue May 23, 2022

Error opening log in 24 #699

Closed

joshtrichards added bug 0. Needs triage labels Aug 19, 2023

Antreesy added 1. to develop good first issue javascript and removed 0. Needs triage labels Sep 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better handling of big log files to avoid crash of the viewer #719

Better handling of big log files to avoid crash of the viewer #719

arnowelzel commented May 17, 2022 •

edited

Loading

adriangalbincea commented May 23, 2022

danir-de commented Jun 1, 2022

crazycorndog commented Sep 6, 2022 •

edited

Loading

strugee commented Sep 23, 2022

luxifr commented Jan 1, 2023

Antreesy commented Sep 17, 2024 •

edited

Loading

Better handling of big log files to avoid crash of the viewer #719

Better handling of big log files to avoid crash of the viewer #719

Comments

arnowelzel commented May 17, 2022 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Nextcloud log (data/owncloud.log)

Browser log

adriangalbincea commented May 23, 2022

danir-de commented Jun 1, 2022

crazycorndog commented Sep 6, 2022 • edited Loading

strugee commented Sep 23, 2022

luxifr commented Jan 1, 2023

Antreesy commented Sep 17, 2024 • edited Loading

Frontend handling can be improved with virtual scroller 🚀

arnowelzel commented May 17, 2022 •

edited

Loading

crazycorndog commented Sep 6, 2022 •

edited

Loading

Antreesy commented Sep 17, 2024 •

edited

Loading