extend item data structure to better deal with large xattr-like things?

[enkore]

Round-up of #51 #1337 #1342

• xattrs (Linux, OSX, BSD) are currently stored in-line in the items, as a dictionary.
• NTFS ADS are more file-like and are currently separate items altogether, using the Windows notation of file:stream.
• SunOS fsattrs are not supported (but, as a quick summary: they essentially attach a completely independent FS namespace to every file).

On Linux xattrs are small by Kernel limitation. On OSX resource forks can be as big as normal files. Ditto for ADS.

There have been a couple ideas how to manage this better:

1. Add "subitems" to items; a chunk ID list for items that are attached to this item, as

   • streams / resource forks
   • fsattrs

   This approach allows to encode even SunOS extremely advanced structure, and the added indirection avoids inflating the length of the items list unnecessarily.

2. Use in-band signalling in paths. Because no OS accepts NUL in paths, this is the obvious choice for the ISB (and could be thus argued to be out-of-band, but in-stream, in fact).

   This is pretty much the same thing as 1.), with the advantage that older Borg correctly syncs to this and finds all chunks, while having the disadvantage that these items would be in the main items list, enlarging the length of it.

Either way allows for the semantics required and enables support for large xattrs/fsattrs/ADS. They also allow the proper case selection when extracting.

[infectormp]

just for info, kernel 4.13 get large xattr support https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e50e5129f384ae282adebfb561189cdb19b81cee