Handle symlinks and excluded files in serialization

[mihaimaruseac]

Description
Following #190: during migration I did not yet add support for symlinks and for excluding files. This will be handled later when we find a need for it during end-to-end testing.

[spencerschrock]

I did not yet add support for symlinks

To clarify, I believe the code already supports symlinks as written (see #251), the TODO refers to preventing symlinks.

model-transparency/model_signing/serialization/serialize_by_file_shard.py

Lines 126 to 129 in 90e3ed3

	# TODO: github.com/sigstore/model-transparency/issues/196 - Add checks
	# to exclude symlinks if desired.
	serialize_by_file.check_file_or_directory(model_path)

This restriction was added in #24, and the only reason I can tell is from this block, which reads as a TODO to potentially allow symlinks:

To avoid bugs where we read the link rather than its target,
we don't allow symlinks for now.
NOTE: It seems that Python's read() always follows symlinks,
so it may be safe to allow them. (readlink() is the function
to read the link metadata).

It would be simple to add a toggle and check for them, but is it needed? Would a signer/verifier care about the presence of a symlink?

[mihaimaruseac]

There might be an issue when the model directory has a symlink inside it that points to a file outside. If we blindly read the symlink, we won't be able to verify the model after it gets uploaded, as that file won't be sent over.

If we add a toggle to error when a symlink is found, we can detect this scenario earlier.

I don't know if there are other cases where we'd care. If the model contains a symlink to a file within itself, nothing should change

[spencerschrock]

we won't be able to verify the model after it gets uploaded, as that file won't be sent over.

I assume this is heavily dependent on the upload process for a given model hub? My symlink knowledge is superficial at best, so I did a naive test with scp from macOS -> Linux which seemed to have copied the contents.:

mkdir /tmp/foo
echo "HELLO" > /tmp/foo/bar.txt
mkdir /tmp/symlink-test
ln -s /tmp/foo/bar.txt /tmp/symlink-test/baz.txt
scp -r /tmp/symlink-test <host>:~

And then on the host, I see a normal file with the expected contents:

$ cat symlink-test/baz.txt 
HELLO
$ ls -laht symlink-test/baz.txt 
-rw-r--r-- 1 <me> <group> 13 Jul 23 11:48 symlink-test/baz.txt

If we add a toggle to error when a symlink is found, we can detect this scenario earlier.

Sounds good. It's trivial to add the toggle, and it will be easy enough to remove if needed later.

[mihaimaruseac]

Yeah, scp always follows symbolic links, but other tools might not.