Integrity constraint violation: 1062 Duplicate entry ... for key 'fs_storage_path_hash'

[Vectrex]

When upgrading from 8.0.x to 8.1.0 I first came across the above error.

• did a complete reinstall of owncloud (empty database, empty data folder, system and configuration Ubuntu 14.04/APCu/MySQL/Files encoded)
• Upload via Ubuntu desktop client 1.8.4
• approx. 1100 files/2.3GiB uploaded
• all files end up on the server (including the ones which violate the index constraints)
• the latter ones don't get the "original" timestamp
• desktop client subsequently tries to re-synchronize ad infinitum
• owncloud.log entry for such a file:
  [...]

{"reqId":"njukFKIzFcc9RFrw4Mms","remoteAddr":"80.110.106.44","app":"webdav","message":"Exception: {\"Message\":\"An exception occurred while executing 'INSERT INTO `oc_filecache` (`mimepart`,`mimetype`,`mtime`,`size`,`etag`,`storage_mtime`,`permissions`,`encrypted`,`parent`,`path_hash`,`path`,`name`,`storage`) SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM `oc_filecache` WHERE `storage` = ? AND `path_hash` = ? HAVING COUNT(*) = 0' with params [\\\"5\\\", \\\"12\\\", 1437054304, 864444, \\\"09b9ce806d4a881f5efa810c05fb7391\\\", 1437054304, 27, 1, \\\"49\\\", \\\"a5797727e322ab6bae4935144e56c704\\\", \\\"files\\\\\\\/media\\\\\\\/fonts\\\\\\\/Glober.7z\\\", \\\"Glober.7z\\\", \\\"1\\\", \\\"1\\\", \\\"a5797727e322ab6bae4935144e56c704\\\"]:\\n\\nSQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-a5797727e322ab6bae4935144e56c704' for key 'fs_storage_path_hash'\",\"Code\":0,\"Trace\":\"
#0 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(116): Doctrine\\\\DBAL\\\\Driver\\\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n
#1 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(996): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOMySql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'INSERT INTO `oc...', Array)\\n
#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/db\\\/connection.php(151): Doctrine\\\\DBAL\\\\Connection->executeUpdate('INSERT INTO `oc...', Array, Array)\\n
#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/db\\\/adapter.php(93): OC\\\\DB\\\\Connection->executeUpdate('INSERT INTO `*P...', Array)\\n
#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/db\\\/connection.php(190): OC\\\\DB\\\\Adapter->insertIfNotExist('*PREFIX*filecac...', Array, Array)\\n
#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/cache\\\/cache.php(322): OC\\\\DB\\\\Connection->insertIfNotExist('*PREFIX*filecac...', Array, Array)\\n
#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/cache\\\/scanner.php(225): OC\\\\Files\\\\Cache\\\\Cache->put('files\\\/media\\\/fon...', Array)\\n
#7 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/cache\\\/scanner.php(188): OC\\\\Files\\\\Cache\\\\Scanner->addToCache('files\\\/media\\\/fon...', Array, -1)\\n
#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/connector\\\/sabre\\\/file.php(208): OC\\\\Files\\\\Cache\\\\Scanner->scanFile('files\\\/media\\\/fon...')\\n
#9 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/connector\\\/sabre\\\/directory.php(113): OC\\\\Connector\\\\Sabre\\\\File->put(Resource id #26)\\n
#10 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1053): OC\\\\Connector\\\\Sabre\\\\Directory->createFile('Glober.7z', Resource id #26)\\n
#11 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(513): Sabre\\\\DAV\\\\Server->createFile('media\\\/fonts\\\/Glo...', Resource id #26, NULL)\\n
#12 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n
#13 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n
#14 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(469): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n
#15 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n
#16 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files\\\/appinfo\\\/remote.php(83): Sabre\\\\DAV\\\\Server->exec()\\n
#17 \\\/var\\\/www\\\/owncloud\\\/remote.php(132): require_once('\\\/var\\\/www\\\/ownclo...')\\n
#18 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractMySQLDriver.php\",\"Line\":66}","level":4,"time":"2015-07-16T13:45:04+00:00"}
[...]

[LukasReschke]

@icewind1991 Any idea?

[ghost]

Ref: #13934

[vengefulpunk]

I am still getting this error too and it is not just for uploads this is occurring on downloads too

[tikund]

I'm getting a very similar error but not with large files, getting the error with files less than 1 Mbyte, using the Windows sync client version 1.8.4

[swadey]

i get the same issue. Is there a workaround?

[ghost]

@swadey See the linked issue for example: #13934 (comment)

[swadey]

@RealRancor I read that thread but it's not clear what the workaround/solution actually is. Is it necessary to change the tx_isolation setting?

[ghost]

@swadey First run the query as described in the linked issue:

SELECT @@GLOBAL.tx_isolation, @@tx_isolation;

to see if this is causing your issue.

[swadey]

@RealRancor Thanks, I did that and, both settings are at their defaults: REPEATABLE-READ

[ghost]

Ok, than it think its unrelated to the question for the transaction isolation level in the linked issue.

[pandur2000]

I have this problem, too, with win desktop 2.0.1 build 5446 & 8.1.3.0. Small files, larger files, totally random. SELECT @@GLOBAL.tx_isolation, @@tx_isolation: REPEATABLE-READ

Update: oc_filecache table to MyISAM seems to have worked.

[soundzgoood]

I get the exact same error with a qnap system, all kinds of files ( about every one out of twenty)
Owncloud 8.1.3 (Linux)
client 1.8.4 (Linux and also Windows)
PHP 5.5.27 (cli) (built: Jul 15 2015 14:16:42)

This is very annoying since you have to delete the error producing files on the server and rename them on your local system in order to get rid of the infinite loops.

[soundzgoood]

is this #19068 related? seems that setting the upload and download limit to 800kbytes/s is helping. trying this right now...

[soundzgoood]

yep, that did it! limiting upload and download speed will make the errors vanish. Automatic upload and download speed also works.

[PVince81]

Try setting "filesystem_check_changes" to 0 in config.php, it should solve those conflicts.

In 8.1.3 and 8.2 the transactional file locking feature will prevent such situations to happen: https://doc.owncloud.org/server/8.1/admin_manual/configuration_files/files_locking_transactional.html

[rcdailey]

I'm running the latest daily build of the Windows client from 11/10/2018 and I am constantly getting this error. Usually comes along with a "500 Internal Server Error" message.