Sync client (1.2.5) ignores symlink files?

[cthielen]

I have a ~/todo.txt that I symlinked to ~/owncloud/todo.txt but the sync client doesn't pick it up.

I did a touch ~/owncloud/test just to ensure the sync client was working and that file did sync.

[danimo]

You are right. symlinks are not synched. This is by design, as synching them can lead to very unexpected behavior. The bug is that non-synched files are not reported, which is also already reported and will be fixed after 1.3.0 is out.

[alick]

It seems that Dropbox will sync the symlinked filed though, according to http://lifehacker.com/5154698/sync-files-and-folders-outside-your-my-dropbox-folder .

@danimo, Can you show me the link to the issue that unsynced files are not reported? I gave a search but didn't find it. I am using client 1.3.0, and when I test with symlink files, I cannot see reports that they are the not synced.

[danimo]

@alick9188 Unsynced file will be reported in 1.4. You can test with rc1 at http://owncloud.org/sync-clients/#testing.

[gbraad]

@danimo can you define "very unexpected behavior"? I can understand it 'it can lead to' issues when having to deal with recursion. I am not so sure what the 'by design' is.

[danimo]

@gbraad Recursion is the first reason, the second is that neither the server nor the webdav protocol (which is used for syncing) have any notion of a link, neither soft nor hard. This means that other clients will not be able to benefit from that information and receive a copy.

The real question here is: What are the use cases? When reading up on the topic, both for ownCloud and other syncing software like DropBox, some people explicitly do want this behavio because they want to sync directories outside of the (client) base directory. However with ownCloud it's possible to define multiple syncs, one per (server) subfolder, which can be associated with subdirectories on the server. This works well without symlinks and should is the recommended way for now.

[gbraad]

@danimo, thanks for the detailed description. However I do not believe this lack is a real issue, as you can never reconstruct it completely in the same way (since you have no idea where the symlink points to). You would have to treat it as a normal nested hierarchy from the point where the symlink is encountered. Because when people after the sync pull the files outside of the syncfolder, you do not have to have this reflect in all the different places you sync to (as it is purely a local issue; mostly due to storage needs?). But as you say, the notion of multiple syncs exists, this could be a workaround to accomplish something similar. Thanks again.

[alick]

@danimo, I indeed tried the multiple syncs setting, and find that the directory will show up in both outside the directory and in the ownCloud directory. So I end up with two same sets of files on the client. To achieve storage saving, the only thing I can do is to symlink the outside directory to the inner one. Is it so?

[fiftyheight]

In my case, I just want to sync some subfolders of my mp3 folder. Multiple syncs doesn't seems to be a solution for that, as it must copy or move the subfolders in a dedicated synced folder and this is not what i want to do.
Following symlinks as with a regular folder will be a very welcome solution, even if this is not the default and activated by an hidden option

[lcsiki]

I would like symlinks as well if would work, the desired content can be stored in one place.

[oernii]

i'd love symlinks too

[salinasv]

Yeah, I am facing the same issue. I understand the recursive issue but I think there could be a compromise to allow people to sync folders without needing to duplicate the space used by them in the local hard drive.

What about follow symlinks up to X depth? that could be done by the client.

[danimo]

@salinasv The main problem is that the sever is not capable of storing symlinks. Feel free to open an issue in core about that.

[longshanks197]

@danimo Thanks for following up on this issue and I understand the hesitation on allowing this function. It seems this was considered at one time as there is a setting in the ocsync.conf file for "sync_symbolic_links" that is marked as "NOT IN USE". This appears to be syncing the symlink on the server, which is not the intent here. I too am looking at the option to allow symlinks in a sync folder as well and I would not expect the server to store the symlinks but to store them as a normal directory structure.

@fiftyheight has a good example of how this would be helpful. If you want to share 50 folders in your mp3 archive, making 50 different sync profiles is a pain. Throwing some symlinks in a folder and syncing the folder is much easier.

My example is with the Linux Steam client. Not all games have a cloud sync for save games but some games I play on my desktop and on my laptop when I travel. I would like to have a SaveGame directory on my laptop and a SaveGame directory on my desktop. I would then symlink between the SaveGame directory to the individual save game folders in Steam on the laptop and desktop separately.The server would see it as a collection of folders in the SaveGame folder, not symlinks. The client would follow the symlinks and present the server as folders or files (using the linked inode values for changes, not the symlink).

@salinasv This would be a good idea. There is already a setting in the ocsync.conf file for max_depth. This is for total folder depth. I would imagine that setting a symlink limit for the client would be reasonable. The client is going to see the structure and know which file is a symlink and which is a file. Have the client stop following symlinks if it sees too many (2 links should be a good default) symlinks in one file path.

This puts a lot of burden on the client to evaluate the file structure but it should be possible. I would not want to touch the core to add this function.

[BigBrotherLovesYou]

I would like to have that behavior too, at least as an option. My sync strategy is create one sync dir, throw in symlinks as needed. Used that for years successfully with dropbox, but as i use owncloud for my contacts already i thought to use the sync client too.
So +1 for following symlinks \o/

[jbogdani]

+1 for symlinks

[danjohn]

Ok, I understand, that mirall/OCSync is not a backup solution, but only a "mirroring service". Accepted ;).

How will it behave, when there are h/s-links on the OC target? (grown home.dir structure, copied by, say, rsync)? Will OC respect that and sync new files, or is that just a perfect way to chaos?

[freelock]

Hi,

+1 on symlinks -- the way Dropbox handles them is by dereferencing them. This allows you to symlink other existing directories into the OC directory, and on the server/other copies, it would be as if the files in the symlinked directory were actually in a simple subdirectory.

I think that would be a good option for many of us...

[fieldse]

Issue opened here:
owncloud/core#6771

Please go here to +1 this idea

[cannycartographer]

This issue may have been brought up elsewhere but another use case for symlinks would be to sync one particular file from a folder, without syncing the rest of it.
Personally I'd actually prefer to be able to just 'share'/ 'sync' that file - maybe even from the file manager,but being able to create a symlink to it from the owncloud folder would have been a workaround.

[ricardoamaro]

+1 on symlinks - anyone in the *nix world makes heavy use of them... no syncing symlinks is a big issue.

[ericbodden]

+1 too. not syncing symlinks sucks

[Morganlej]

+1 for symlinks. By dereferencing at client side. As it seem everybody here assume it should in the first place. As said links are much used feature for people that really use *nix. This is a reason we still use dropbox even for in-house files.

Use example: i want a current project shared to my laptop or a customer. I then just drag a link from that folder to the owncloud shared folder, done. Easily done on several projects. When i remove that link, content get deleted on wherever it was shared.

Now withoput this feature i need to move that folder with content to OC shared folder and symlink it back to where it really belongs (so local programs etc stil use files in the original place) then remember to move back.

[microchip8]

+1 for symlinks! :)

[vliegeois]

From what I see of ownCloud, this option is not needed since ownCloud is more powerful than Dropbox.
Indeed, you are not required to sync the root folder and to have it appear as a local folder.
So for instance, you are not obliged to have a ownCloud folder. This is the default but it's not obligatory.
Instead, remove the sync of the root folder and add manually the sync of some specific folders like the Documents folder. The name of this folder on the cloud can be different from the name on your machine.
Now, whenever you add a file to your Documents folder, it will be synchronised and you don't have to see that on the cloud this Documents folder is stored with another name in the ownCloud root.

[rolandixor]

This is a pretty critical feature, especially in cases where you have a defined structure on the client that would need to be duplicated just to sync it.

[eBug]

+1

[r00t23]

+1 for symlinks!

[captoriginal]

+1 for symlinks, too!

[rolandixor]

I wonder if Nextcloud will support Symlinks. Either way, they've done us a disservice, because now we must migrate :/

[dragotin]

ladies and gents.... please think again about the symlink thing. It makes things more complicated, more fragile, because link targets can vanish, loop and such, and as #665 (comment) says, there are other options.

@rolandixor must migrate to what? Where is the client download button for nextcloud? We should not go this way in the bugtracker tough, rather let the socmedia peepz go on ranting...

[Morganlej]

For us who are used to symlinks the benefits are obvious.
We who use symlinks on ana daily basis use them because they solve a lot, easily, and we know how to use them.
As always, there are ways to foul up even without symlinks.
Symlinks is widely used, have been for long, time, and to say that specifically owncloud users can not handle that is not nice!

[danimo]

@Morganlej This was not meant as an insult. It is a result of a careful weighing of usability concerns. We're more than happy for a design document that proposes how to support symlinks in a non-confusing way for everyone, or at least "power users". So far we hadn't had any convincing suggestions, but I'm not saying they do not exist.

[cannycartographer]

Hi,
I would suggest one option, although it wouldn't solve everything that users after here, would be to allow users to select an individual file to be synced to the folder, for use cases where users don't want to move the file into a synced folder.

[scolebrook]

@Morganlej I use ownCloud to sync my ~/.ssh directory between multiple computers. Not just for keys but for the config file that defines shortcuts for all the servers I manage. I have two accounts setup in the ownCloud app. One syncs my entire account in the regular way, but I use selective sync to not sync the ssh folder from the server.

Then I have a second account setup in the client, but it's the same username and password so the same account server side. It just pairs the ssh folder in my account on the server with ~/.ssh. This is an alternative to using a symlink to to link ~/.ssh into my ~/ownCloud folder.

To me, it's cleaner. I don't need the symlink to tell ownCloud to sync a given local folder with a given server folder. That functionality is already built into ownCloud. But this is only a helpful solution if that's the only reason why you have a symlink. If you need symlinks in your file system for other software then this probably won't help you. But perhaps it might help others.

Keep in mind that in order for ownCloud to support symlinks, it has to be intuitive and simple to people who don't know what a file system is (99% of computer users despite using one every day) let alone what symlinks are. That's the challenge here, to make something that doesn't set up for failure those who are less skilled.

[codethief]

@scolebrook Why would those 99% of computer users use symlinks in the first place? I don't see the problem here. Just introduce the option to follow symlinks for power users that know what they're doing.

[scolebrook]

@codethief In light of the fact that ownCloud can already achieve the same functionality without the need for symlinks at all, there's the question of why spend the time to develop and maintain an additional option that achieves the same result via the filesystem rather than the settings window.

I'm not saying that there isn't a good reason to develop the feature in addition to what's already there. I'm just saying that in a project like this with limited resources there needs to be a good reason to take resources away from other features. If those features have a bigger benefit for more people then they should be developed first. Or vice versa.

And if it's determined that this needs to be developed ahead of other things then there are the considerations of exactly how to implement it. It's not a case of simply saying "just introduce the option". How will it work? How will it look in the ui? How will ownCloud detect and avoid loops of links? Will there be a limit on how deep the traversal of links should go? What happens if a target of a link disappears? Should ownCloud delete all files from the server that were behind that symlink? How will ownCloud handle symlinks to targets within the synced local ownCloud directory? What if a link target it removed due to selective sync on one computer, but still exists on another?

Symlinks work on one computer and they solve that problem well. ownCloud works across multiple computers and devices with multiple accounts on multiple servers and there are some fundamental differences here and every single one of them needs careful consideration and thorough testing. That's a lot of time to take away from other features to duplicate functionality that can be achieved with a couple of mouse clicks.

As @danimo said, this really starts with a design document. So someone who wants this feature developed needs to take some time to think through these scenarios and put together how they think this feature could work. Then as a community we can work through what if's and iterate on that design until something solid is produced. Only then have we reached square one and can consider actual implementation.

[codethief]

@scolebrook Thanks for your response! I understand a lot better now.

[doits]

edit: moved to #1440 because that seems to be the proper issue