Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Big changes in sftp_handle.py and minor in utils.py and configuration.json #247

Merged
merged 13 commits into from
Feb 16, 2024
Merged

Big changes in sftp_handle.py and minor in utils.py and configuration.json #247

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f198cee
Included big changes in md5 verification and error handling
Shettland Jan 23, 2024
4f01530
Included read_md5_checksum and compress_file methods
Shettland Jan 23, 2024
e703977
Changed fields to suit the ones in sftp_handle.py
Shettland Jan 23, 2024
730e3a4
Included big changes in md5 verification
Shettland Jan 23, 2024
7927dd2
Small changes in utils.py
Shettland Jan 29, 2024
98f96ab
Included big changes in md5 verifivation and error handling, linting2
Shettland Jan 29, 2024
e611847
Big changes in md5 verification, small hotfix for sftp_handle.py
Shettland Jan 30, 2024
05aba1f
Big changes in md5 verification, small hotfix for sftp_handle.py, lin…
Shettland Jan 31, 2024
cab8bbb
Included several changes to resolve conversations in #247 and fixed s…
Shettland Feb 15, 2024
860a48d
Included sequencing_institution and a sample_json_fields for sftp_han…
Shettland Feb 15, 2024
317ba9c
Removed commented code and unnecessary variable
Shettland Feb 16, 2024
be41a24
Changed info to warning for files not found in md5sum
Shettland Feb 16, 2024
03dfd71
Big changes in sftp_handle. Brought back a variable that was necessary
Shettland Feb 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 17 additions & 25 deletions relecov_tools/conf/configuration.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,36 +194,29 @@
"sftp_server": "sftprelecov.isciii.es",
"sftp_port": "22"
},
"metadata_processing": {
"header_flag": "CAMPO",
"excel_sheet": "METADATA_LAB"
},
"abort_if_md5_mismatch": "False",
"platform_storage_folder": "tmp/relecov",
"allowed_sample_extensions": [
"fastq.gz",
"fasta"
],
"allowed_R1_delimiters": [
"_R1_",
"_R1.",
".R1.",
"Forward",
"forward",
"_1_",
"_1.",
".1."
],
"allowed_R2_delimiters": [
"_R2_",
"_R2.",
".R2.",
"Reverse",
"reverse",
"_2_",
"_2.",
".2."
"platform_storage_folder": "/tmp/relecov",
"allowed_file_extensions": [
".fastq.gz",
".fastq",
".fq",
".fq.gz",
".fasta",
".fasta.gz"
],
"allowed_download_options": [
"download_only",
"download_clean",
"delete_only"
],
"skip_when_found": [
"#",
"Hash",
"Path"
]
},
"GISAID_configuration": {
Expand Down Expand Up @@ -369,7 +362,6 @@
"sequencing_sample_id": "sequencing_sample_id"
}
},
"md5_file_name": "md5_check_file.csv",
"ENA_fields": {
"ENA_configuration": {
"study_alias": "RELECOV",
Expand Down
24 changes: 12 additions & 12 deletions relecov_tools/gisaid_upload.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@ def __init__(
def complete_mand_fields(self, dataframe):
"""Complete mandatory empty fields with 'unknown'"""
dataframe.loc[dataframe["covv_gender"] == "", "covv_gender"] = "unknown"
dataframe.loc[
dataframe["covv_patient_age"] == "", "covv_patient_age"
] = "unknown"
dataframe.loc[dataframe["covv_patient_age"] == "", "covv_patient_age"] = (
"unknown"
)

authors = [authors_field for authors_field in dataframe["covv_authors"]]
if "" in authors or "unknown" in authors:
Expand All @@ -115,17 +115,17 @@ def complete_mand_fields(self, dataframe):
)
sys.exit(1)

dataframe.loc[
dataframe["covv_subm_lab_addr"] == "", "covv_subm_lab_addr"
] = "unknown"
dataframe.loc[dataframe["covv_subm_lab_addr"] == "", "covv_subm_lab_addr"] = (
"unknown"
)
dataframe.loc[dataframe["covv_subm_lab"] == "", "covv_subm_lab"] = "unknown"
dataframe.loc[
dataframe["covv_orig_lab_addr"] == "", "covv_orig_lab_addr"
] = "unknown"
dataframe.loc[dataframe["covv_orig_lab_addr"] == "", "covv_orig_lab_addr"] = (
"unknown"
)
dataframe.loc[dataframe["covv_orig_lab"] == "", "covv_orig_lab"] = "unknown"
dataframe.loc[
dataframe["covv_patient_status"] == "", "covv_patient_status"
] = "unknown"
dataframe.loc[dataframe["covv_patient_status"] == "", "covv_patient_status"] = (
"unknown"
)
dataframe.loc[dataframe["covv_type"] == "", "covv_type"] = "betacoronavirus"
dataframe.loc[dataframe["covv_passage"] == "", "covv_passage"] = "Original"

Expand Down
Loading
Loading