SpikeSortingRecordingSelection attributes

[magland]

I was noticing that LabTeam is not a primary key for SpikeSortingRecordingSelection (see below).

Does that mean that if one lab team does spike sorting with particular sort group, sort interval, and sorting parameters, then another lab team cannot also do that sorting? Seems not right... unless the lab team is associated with the session, in which case it should not be an attribute of this table.

@schema
class SpikeSortingRecordingSelection(dj.Manual):
    definition = """
    # Defines recordings to be sorted
    -> SortGroup
    -> SortInterval
    -> SpikeSortingPreprocessingParameters
    ---
    -> IntervalList
    -> LabTeam
    """

[khl02007]

@magland thanks, that's a good point. I think we initially had implemented the lab team system because some people in the lab did not want everyone else to have access to their nwb file, but we certainly do want to encourage sharing. @lfrank what do you think? I can move LabTeam to the primary key if you're OK with it.

[lfrank]

I think that’s a reasonable thing to do, particularly for the case where someone wants to redo the curation, so go ahead. Just FYI, one reason we didn’t do that at first was because it could lead to confusions where the same data is referenced twice in some downstream analysis, but given that we could use multiple sorters and have the same issue, I think we’ll just have to be careful of that. On Feb 25, 2022, at 12:57 PM, Kyu Hyun Lee ***@***.******@***.***>> wrote: This Message Is From an External Sender This message came from outside your organization. @magland<https://urldefense.com/v3/__https://github.com/magland__;!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0XhcGMw4A$> thanks, that's a good point. I think we initially had implemented the lab team system because some people in the lab did not want everyone else to have access to their nwb file, but we certainly do want to encourage sharing. @lfrank<https://urldefense.com/v3/__https://github.com/lfrank__;!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0XiNsQVgA$> what do you think? I can move LabTeam to the primary key if you're OK with it. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/LorenFrankLab/nwb_datajoint/issues/133*issuecomment-1051260961__;Iw!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0VAQzKFhQ$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABV4PSLAACRVX3HMC27IOA3U47UJ3ANCNFSM5PLDM3OQ__;!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0WT-27GqA$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0WrSRzHOQ$> or Android<https://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!LQC6Cpwp!4wzJmuYqJdUXeyadXojBiiDBkg_cEGWWbN_Nv2PrBQy436aHrltBF3Mgv0VJzZzNKw$>. You are receiving this because you were mentioned.

[CBroz1]

@khl02007 It's seems like it's currently the case that a second team running an otherwise identical key will cause the first run to be deleted:

spyglass/src/spyglass/spikesorting/v0/spikesorting_recording.py

Lines 319 to 321 in fa1114e

	recording_path = str(recording_dir / Path(recording_name))
	if os.path.exists(recording_path):
	shutil.rmtree(recording_path)

Do we want to fix that by adding team to the folder name? I'm thinking there should be a prompt before deleting another team's folder

Ignoring team, there are currently no duplicate keys, so no cause for concern about overwritten files. But does the team influence the outcome? Or is the process deterministic, such that the same parameters will yield the same result? I'm revisiting this table with #917 in mind

It team just indicates run ownership, I think we now navigate that better with cautious delete style ownership. I wouldn't propose edits to this table, but just trying to wrap my head around the need discussed here to both tackle recompute and consider for future pipeline development

[khl02007]

@CBroz1 I see, I didn't realize this issue, thanks for catching it. SpikeSortingRecording.make is supposed to be deterministic, and would give you the same results regardless of which team runs it. I think there is little point in running SpikeSortingRecording.make on the same set of parameters when someone else has done it. I think the labteam name was put in as a secondary key to protect against a nonmember of the team deleting the entry. Does this help you make decisions about how to change it?

[CBroz1]

That's helpful, thanks. I'll reopen pending a fix of the possibility of the folder overwrite

I think this rmtree should be replaced with an insert that points to the existing folder