Move Binary Data outside of Fedora

[carolyncole]

Currently we store our binary data in the fedora repository. This lead to migration hassles and copying the data when we migrate. We would like to move binary files to a location on the file system, and then have fedora either point via metadata to them or serve up as a forward.

[carolyncole]

From what I can tell if we had a server to host the content from a directory we could use add_external_file_to_file_set to store our content externally and have fedora redirect:
https://github.com/samvera/hydra-works/blob/d8969788d64638714eac5743704a9bf21319d8e2/spec/hydra/works/services/add_external_file_to_file_set_spec.rb

Right now I think we are hosting the thumbnails locally through passenger. We could think about doing something similar.

@jrpatterson @informaticianme Any thoughts on the level of effort to serve a directory of files from the repo server (maybe a network share)? We are looking to remove the binary content from Fedora.

[awead]

There was an PR to enable external files in ActiveFedora, but it was never merged:
samvera/active_fedora#1234
Perhaps that could be of some use here.

[pketienne]

The relevant documentation for external content for Fedora can be found here:
https://wiki.duraspace.org/display/FEDORA471/External+Content

Here's a git issue from the Islandora community that seems to have a good discussion around external content with Fedora - specifically limitations of the current external content feature:
Islandora/documentation#564

[carolyncole]

2 patterns are to let

• fedora manage the content as external, and or just tell fedora where this is
• Put the file somewhere and then create the mount point into fedora (A predicate is a path) Fedora does not understand this is a file

For Samvera only the second pattern is possibly easier
For other things accessing you repository (bag export) would not work with option number two.

@little9 @bess will take a look and determine which pattern would be best for both PSU and the community.

[carolyncole]

We may include upgrading to to Fedora 4.7.5

[carolyncole]

• Research the two patterns
• Research upgrading Fedora
• [optional] Upgrade Fedora based on recommendation
• Pilot A fedora instance on Demo Server to document the process
• Go through the steps with a DCE Fedora Box - Single box
• Go through the steps with QA - Set of 4 servers [ Timing information]

[carolyncole]

https://docs.google.com/document/d/1jA43RCnZY8F-gBmXoDVXO4weGiJQvtUUZEBxFpQpJpQ/edit

Leaning towards option 2:

• Content would need to be on disk
  pair tree?
  according to id or hash

• Set up a http server for it (PR into Fedora 4 to allow you to give it a file path)

• Find any place that references a FileSet and Change it to reference an external File Set

• Versioning is necessary, but it does not need to occur at the fedora level

• Fixity may be a requirement. We may be able to leverage the fits fixity

[carolyncole]

https://docs.google.com/document/d/1jA43RCnZY8F-gBmXoDVXO4weGiJQvtUUZEBxFpQpJpQ/edit

[pketienne]

Added jira tickets for likely infrastructure work to enable sharing of filesystem files via httpd:

• https://servicedesk.css.psu.edu/browse/APPINT-2254
• https://servicedesk.css.psu.edu/browse/APPINT-2255