Cifs external storage : utf-8 problem ?

[ghost]

Hello !

I mounted a Synology NAS as external storage (cifs) on my OwnCloud 7.0.1. I get troubles with some files not showing into OC. After some googling + seeking this forum, I can't find something similar.

I ask there before opening a bug to see if anybody else got in trouble with same problem.

What I did :

• removing / re-doing the attachement
• checking the rights (user, login...)
• creating a cron job to force cache sync (occ files:scan --all ...)
  ...
  No way.

Finaly, I think I found the problem in "cache scanner". If a file in the dir currently scanning presents a "non ascii" char (like éàè...), the "$path" varible is corrupted.

The dir contains these files :
01-AO DSE - CG_Conditions_Generales_AVASAD_V1.0 (1e copie).pdf
02-AO DSE - CG_Conditions_Génerales_AVASAD_V1.0 (2e copie).pdf <= é !!
03-AO DSE - CG_Conditions_Génerales_AVASAD_V1.0 (3e copie).pdf <= é !!

When scanner runs, the log shows :
{"reqId":"53fb3d872d417","app":"OC\Files\Cache\Scanner","message":"!!! Path '2014/RC/OC-problem/03-AO DSE - CG_Conditions_G 01-AO DSE - CG_Conditions_Generales_AVASAD_V1.0 (1e copie).pdf' is not readable !!!","level":0,"time":"2014-08-25 13:43:35","method":"PROPFIND","url":"/remote.php/webdav/QuodAdmin%40NAS/2014/RC/OC-problem/"}

The "$path" shows incomplete file name "03-AO DSE - CG_Conditions_G" interrupted just before the "é" and continues with the filename of the first file "01-AO DSE - CG_Conditions_Generales_AVASAD_V1.0 (1e copie).pdf".

Actualy, I get many similar messages with "accents" into their name and the files are not showing through OC...

Any idea ? Misconfigured OC ?

Thanks in advance for your help !

My configuration :

• Own server, Debian Wheezy
• OC 7.0.1
• MySQL 5.5.38
• PHP 5.4.4
• APC 3.1.13
• Apache2 2.2.22

My config.php file

'ocb92391edc3', 'passwordsalt' => '*** Removed for security reasons', 'trusted_domains' => array ( 0 => '192.168.30.67', 1 => 'cloud.quodapps.ch', 2 => 'timide', ), 'datadirectory' => '/var/spool/owncloud/quod', 'dbtype' => 'mysql', 'version' => '7.0.1.1', 'dbname' => 'quodcloud', 'dbhost' => 'localhost', 'dbtableprefix' => 'oc_', 'dbuser' => 'oc_oc_admin_quod', 'dbpassword' => '*** Removed for security reasons', 'installed' => true, 'mail_from_address' => 'quodcloud', 'mail_smtpmode' => 'php', 'mail_domain' => 'quod.ch', 'mail_smtpname' => 'masc', 'mail_smtppassword' => '*** Removed for security reasons', 'apps_paths' => array ( 0 => array ( 'path' => '/var/www/owncloud/apps', 'url' => '/apps', 'writable' => false, ), 1 => array ( 'path' => '/var/www/owncloud/apps2', 'url' => '/apps2', 'writable' => true, ), ), 'logdateformat' => 'Y-m-d H:i:s', 'log_rotate_size' => '10485760', 'loglevel' => '0', 'appstoreenabled' => false, 'appcodechecker' => false, 'updatechecker' => false, 'enable_previews' => true, 'theme' => 'quod', 'preview_libreoffice_path' => '/usr/bin/soffice', 'forcessl' => true, );

[timw4mail]

Looking at the database, the file path is correct, but the name is truncated at the 'offending' character.

There's also another case, such as Chinese/Japanese characters being replaced with question marks in the name.

There seems to be issues with the functionality that saves the name of the file from the scanner, and it is clearly not safe for anything other than ASCII characters.

[timw4mail]

Other related issues: #3190, #4513, #9076.
#8043 seems to be a possible solution.

[ghost]

Here is a post about similar problem (?)

external Storage SMB/CIFS special character

Postby schuhlos » Sat Oct 25, 2014 10:12 pm
Hi there,

i had a problem with the configuration. I am using oC 7.0.2 with Debian Linux. It works without any problem but if I mount an external storage with SMB/CIFS I have access on it but I cannot sync any files which contains special characters eg ä,ö,ü,ß . I allways get an error, but if I save this files on the local datastore on Debian Linux it works fine. The external storage is a Windows 2008R2 Server and i have full access on the shared folder.

Webclient -> Upload on external storage -> error
Webclient -> Upload on local datastorage -> ok
Windows OC client -> Upload on external storage -> error
Windows OC client -> Upload on local datastorage -> ok

I googled all without any solution. But now i found a workaround. The problem is the en_EN.UTF-8 which is hardcoded in smb.php in the directory /var/www/owncloud/apps/files_external/3rdparty/smb4php . I changed this to de_DE.UTF-8 and now I can sync with special characters.

I hope someone of the developer can check this.

Greetings

Chris

[DeepDiver1975]

I googled all without any solution. But now i found a workaround. The problem is the en_EN.UTF-8 which is hardcoded in smb.php in the directory /var/www/owncloud/apps/files_external/3rdparty/smb4php . I changed this to de_DE.UTF-8 and now I can sync with special characters.

Nice catch - we had a hard dependency to en_EN.UTF-8 back in the past, which we relaxed to a list of UTF-8 locales - but it looks like smb still requires en_EN.UTF-8

[PVince81]

@icewind1991 what do you think ?

[Phillinger]

@Marcatquod thanks for the workarround, great work!

[DeepDiver1975]

With owncloud 8.1 we use a different library to connect to smb/cifs shares - this should be fixed - please reopen if issue still persists. THX

[Phillinger]

With owncloud 8.1 on Debian Wheezy the error is back. Folders/Files with umlauts are not displayed correctly and can't be opened. Since the configuration of the app has changed, the workarround as mentioned above doesn't work anymore.

Got it fixed for me by changing the following file:
/var/www/owncloud/apps/files_external/3rdparty/icewind/smb/src/Server.php

There, i changed the const LOCALE = 'en_US.UTF-8'; to const LOCALE = 'de_DE.UTF-8';

[RobinMcCorkell]

cc @icewind1991

[PVince81]

Please try 9.1.0beta2. If it doesn't work, try setting the new "encoding compatibility" mode in the external storage's advanced settings (cog icon) and then run "occ files:scan --all" once.

If the problem persists, please open.