documentserver:flush not working, missing file owner (workaround provided)

[danielegobbetti]

Hi!
With up-to-date versions of nextcloud (18.0.2) and documentserver_community (0.1.5) (and onlyoffice app 4.1.4) we experience some weird behavior when the online editor is used by an user that is not the owner of the file.

What happens is that the doc_$documentid folder is not populated with the files owner and fileid, thus making the occ documentserver:flush procedure stop with an error like:

OCP\Files\NotFoundException: /appdata_INSTANCE/documentserver_community/doc_26750874/owner

It took a lot of time to debug and a (quick and dirty) workaround is provided below. I believe this should be fixed because it yields to documents which are persistently out-of-date on the filesystem and a constantly growing oc_documentserver_changes table in the database. Apologies for the long issue, but I would like to share the workaround with other users affected by this bug.

The workaround is to build a map between the documentid and the fileid, in order to do so we need the instance-id (from the config.php file) and the ETag for each affected file. The ETag is available in the oc_filecache table, where the fileid is also specified.

In order to reconstruct how to build the mapping, I observed that the onlyoffice app is able to return the documentid from the fileid (see controller/editorcontroller.php and documentservice.php )

The owner might then be retrieved from the oc_share table (but there are no direct hits if a whole directory was shared and not the specific file, there are possibily other places where this information is available but I am not an expert in the nextcloud DB). But even if it's wrong, the saveChanges function is able to find the file anyway, hence the owner file may contain any user known to nextcloud.

The workaround we are using is to export all the files and etags with this query:

SELECT DISTINCT of.fileid, of.etag, of.path, os.uid_owner FROM oc_filecache of 
JOIN oc_share os ON (of.fileid=os.file_source)  
INTO OUTFILE '/tmp/files.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';

then a very basic php script that mimics the procedures linked above is used:

    $csv = array_map('str_getcsv', file('files.csv'));
    $instanceId = YOUR_INSTANCE_ID_HERE;
    foreach ($csv as $line) {
        $expected_key = $instanceId."_".$line[1];

        if (strlen($expected_key) > 20) {
            $expected_key = crc32( $expected_key);
        }
        $key = preg_replace("[^0-9-.a-zA-Z_=]", "_", $expected_key);
        $key = substr($key, 0, min(array(strlen($key), 20)));

        echo 'ls doc_'.$key.' && echo -n "'.$line[3].'" > doc_'.$key.'/owner && chown www-data:www-data doc_'.$key."/owner\n";
        echo 'ls doc_'.$key.' && echo -n "'.$line[0].'" > doc_'.$key.'/fileid && chown www-data:www-data doc_'.$key."/fileid\n";
    }

Execute this php and save the output to a file (in my case /tmp/mappings.txt), then go the documenserver_community folder within your installation (the one containing lots of "doc_documentid" files) and do something like this:

ls | while read dir; do grep $dir /tmp/mappings.txt; done

Now review the rows that are outputted to the console and - if you are fine with their content - execute them: the effect is to populate each directory with the two missing files.
Finally you will be able to execute occ documentserver:flush without issues.

[pongraczi]

Hi, please check my detailed debug, maybe the same source of the problem: #170 (comment)

[github-actions]

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 60 days. Thank you for your contribution!

Retaking repo management include inheritance of old, not treated, and probablty obsolete issues, this is why it was decided to mark issues as stale.

[github-actions]

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.