file:scan --all causing ldap connection lost (timeout)

[jaceq]

Steps to reproduce

1. Setup owncloud with ldap , have ldap connection timeout (on ldap or LoadBalancer)
2. Run occ files:scan --all (I have 77 users and 48 folders)

Expected behaviour

If LDAP connection is lost it should be re-initiated and whole process not dropped in the middle

Actual behaviour

It seems ldap bind is done at the beginning of scan and then search queries are executed... but if scan takes a long time connection dies in the middle and then on next user whole scan dies with following error:

Exception while scanning: Lost connection to LDAP server.
#0 /var/www/owncloud/apps/user_ldap/lib/LDAP.php(257): OCA\User_LDAP\LDAP->postFunctionCall()
#1 /var/www/owncloud/apps/user_ldap/lib/LDAP.php(178): OCA\User_LDAP\LDAP->invokeLDAPMethod('read', Resource id #596, 'uid=someuser.last...', '(|(memberof=cn=...', Array)
#2 /var/www/owncloud/apps/user_ldap/lib/Access.php(175): OCA\User_LDAP\LDAP->read(Resource id #596, 'uid=someuser.last...', '(|(memberof=cn=...', Array)
#3 /var/www/owncloud/apps/user_ldap/lib/User_LDAP.php(219): OCA\User_LDAP\Access->readAttribute('uid=someuser.last...', '', '(|(memberof=cn=...')
#4 /var/www/owncloud/apps/user_ldap/lib/User_LDAP.php(274): OCA\User_LDAP\User_LDAP->userExistsOnLDAP(Object(OCA\User_LDAP\User\User))
#5 /var/www/owncloud/apps/user_ldap/lib/User_LDAP.php(323): OCA\User_LDAP\User_LDAP->userExists('someuser.lastejon')
#6 /var/www/owncloud/lib/private/User/User.php(251): OCA\User_LDAP\User_LDAP->getHome('someuser.lastejon')
#7 /var/www/owncloud/lib/private/Files/Mount/LocalHomeMountProvider.php(41): OC\User\User->getHome()
#8 /var/www/owncloud/lib/private/Files/Config/MountProviderCollection.php(97): OC\Files\Mount\LocalHomeMountProvider->getHomeMountForUser(Object(OC\User\User), Object(OC\Files\Storage\StorageFactory))
#9 /var/www/owncloud/lib/private/Files/Filesystem.php(432): OC\Files\Config\MountProviderCollection->getHomeMountForUser(Object(OC\User\User))
#10 /var/www/owncloud/lib/private/Files/Filesystem.php(370): OC\Files\Filesystem::initMountPoints('someuser.lastejon')
#11 /var/www/owncloud/lib/private/legacy/util.php(226): OC\Files\Filesystem::init('someuser.lastejon', '/someuser.lastejo...')
#12 /var/www/owncloud/lib/private/Files/Utils/Scanner.php(82): OC_Util::setupFS('someuser.lastejon')
#13 /var/www/owncloud/lib/private/Files/Utils/Scanner.php(156): OC\Files\Utils\Scanner->getMounts('/someuser.lastejo...')
#14 /var/www/owncloud/apps/files/lib/Command/Scan.php(158): OC\Files\Utils\Scanner->scan('/someuser.lastejo...')
#15 /var/www/owncloud/apps/files/lib/Command/Scan.php(226): OCA\Files\Command\Scan->scanFiles('someuser.lastejon', '/someuser.lastejo...', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#16 /var/www/owncloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/owncloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/owncloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /var/www/owncloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/owncloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/owncloud/lib/private/Console/Application.php(146): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/owncloud/console.php(102): OC\Console\Application->run()
#23 /var/www/owncloud/occ(11): require_once('/var/www/ownclo...')
#24 {main}

Server configuration

Operating system:
Debian 8.5

Web server:
Nginx + php7.0-fpm
Database:
Postgres
PHP version:
7.0
ownCloud version: (see ownCloud admin page)
9.1.2
Updated from an older ownCloud or fresh install:
Update
Where did you install ownCloud from:
Owncloud apt repo

[PVince81]

@jvillafanez maybe need to somehow automatically reconnect to the LDAP server after a specific time.

[PVince81]

Note: this issue will disappear once we have a central user table #23558 which will remove the need to connect to LDAP when iterating over all users.

[jaceq]

There are few choices to tackle this:
1 - Bind to ldap, query, close connection - repeat for every user
OR
2 - Bind to ldap at the beginning but then before using connection have php equivalent of

try:
  <query ldap>
except:
  <re-bind>
  <query ldap>

Doing rebind on some timer seems complicated and unreliable.

[jvillafanez]

There is the additional problem of LDAP pagination. Reconnecting blindly will cause issues.

I think the easiest workaround would be fetch all the LDAP users and scan their files one by one.

[jaceq]

I am not a php person but from other interactions with ldap (python / perl) I never had problems with pagination... so unless this would be a php specific thing I can't think of a problem.

[butonic]

No longer uses leap since the introduction of the account table

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.