Skip to content

Commit

Permalink
fix: make usage of name and slug consistent (#3620)
Browse files Browse the repository at this point in the history
  • Loading branch information
m-alisafaee authored Oct 10, 2023
1 parent db9f93b commit b31ade0
Show file tree
Hide file tree
Showing 441 changed files with 2,515 additions and 1,952 deletions.
2 changes: 1 addition & 1 deletion AUTHORS.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
6 changes: 2 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion QA_PROTOCOL.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
6 changes: 2 additions & 4 deletions docs/_ext/cheatsheet.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2018-2023- Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion docs/changes.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
6 changes: 2 additions & 4 deletions docs/cheatsheet/conf.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
6 changes: 2 additions & 4 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion docs/gettingstarted.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
:orphan:

..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/introduction.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/license.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ License

.. code-block:: text
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/plugins.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/api.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/commands/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/core.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/database.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/gateways.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/dataset_provider.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/datasets.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/projects.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/provenance.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/workflow.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/plugins.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/service.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/service_errors.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).
Expand Down
6 changes: 2 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
5 changes: 2 additions & 3 deletions renku/command/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
22 changes: 11 additions & 11 deletions renku/command/checks/datasets.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,17 +77,17 @@ def check_missing_files(dataset_gateway: IDatasetGateway, **_):
path = project_context.path / file_.entity.path
file_exists = path.exists() or (file_.is_external and os.path.lexists(path))
if not file_exists:
missing[dataset.name].append(file_.entity.path)
missing[dataset.slug].append(file_.entity.path)

if not missing:
return True, False, None

problems = WARNING + "There are missing files in datasets."

for dataset_name, files in missing.items():
for dataset_slug, files in missing.items():
problems += (
"\n\t"
+ click.style(dataset_name, fg="yellow")
+ click.style(dataset_slug, fg="yellow")
+ ":\n\t "
+ "\n\t ".join(click.style(path, fg="red") for path in files)
)
Expand All @@ -114,9 +114,9 @@ def fix_or_report(dataset):
dataset.unfreeze()
dataset.derived_from = None
dataset.freeze()
communication.info(f"Fixing dataset '{dataset.name}'")
communication.info(f"Fixing dataset '{dataset.slug}'")
else:
invalid_datasets.append(dataset.name)
invalid_datasets.append(dataset.slug)

for dataset in dataset_gateway.get_provenance_tails():
while dataset.derived_from is not None and dataset.derived_from.url_id is not None:
Expand All @@ -137,7 +137,7 @@ def fix_or_report(dataset):
WARNING
+ "There are invalid dataset metadata in the project (use 'renku doctor --fix' to fix them):"
+ "\n\n\t"
+ "\n\t".join(click.style(name, fg="yellow") for name in invalid_datasets)
+ "\n\t".join(click.style(slug, fg="yellow") for slug in invalid_datasets)
+ "\n"
)

Expand Down Expand Up @@ -177,12 +177,12 @@ def check_dataset_files_outside_datadir(fix, dataset_gateway: IDatasetGateway, *
continue

if fix:
communication.info(f"Fixing dataset '{dataset.name}' files.")
communication.info(f"Fixing dataset '{dataset.slug}' files.")
dataset.unfreeze()
for file in detected_files:
dataset.unlink_file(file.entity.path)
dataset.freeze()
add_to_dataset(dataset.name, urls=[file.entity.path for file in detected_files], link=True)
add_to_dataset(dataset.slug, urls=[file.entity.path for file in detected_files], link=True)
else:
invalid_files.extend(detected_files)

Expand Down Expand Up @@ -220,7 +220,7 @@ def check_external_files(fix, dataset_gateway: IDatasetGateway, **_):
for file in dataset.files:
if file.is_external:
external_files.append(file.entity.path)
datasets[dataset.name].append(file)
datasets[dataset.slug].append(file)

if not external_files:
return True, False, None
Expand All @@ -240,7 +240,7 @@ def check_external_files(fix, dataset_gateway: IDatasetGateway, **_):
f"dataset with an external storage backend:\n\t{external_files_str}"
)

for name, files in datasets.items():
file_unlink(name=name, yes=True, dataset_files=files)
for slug, files in datasets.items():
file_unlink(slug=slug, yes=True, dataset_files=files)

return True, False, None
4 changes: 2 additions & 2 deletions renku/command/checks/validate_shacl.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,15 @@ def check_datasets_structure(dataset_gateway: IDatasetGateway, **_):
try:
conform, graph, t = _check_shacl_structure(data)
except (Exception, BaseException) as e:
problems.append(f"Couldn't validate dataset '{dataset.name}': {e}\n\n")
problems.append(f"Couldn't validate dataset '{dataset.slug}': {e}\n\n")
continue

if conform:
continue

ok = False

problems.append(f"{dataset.name}\n\t{_shacl_graph_to_string(graph)}\n")
problems.append(f"{dataset.slug}\n\t{_shacl_graph_to_string(graph)}\n")

if ok:
return True, False, None
Expand Down
5 changes: 2 additions & 3 deletions renku/command/clone.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2018-2023- Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
5 changes: 2 additions & 3 deletions renku/command/config.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
7 changes: 3 additions & 4 deletions renku/command/dataset.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -39,7 +38,7 @@


def search_datasets_command():
"""Command to get all the datasets whose name starts with the given string."""
"""Command to get all the datasets whose slug starts with the given string."""
return Command().command(search_datasets).require_migration().with_database()


Expand Down
5 changes: 2 additions & 3 deletions renku/command/doctor.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2020 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
9 changes: 6 additions & 3 deletions renku/command/format/dataset_files.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@ def get_lfs_tracking():
record.is_lfs = False

def naturalize(value) -> str:
return naturalsize(value).upper().replace("BYTES", " B")
try:
return naturalsize(value).upper().replace("BYTES", " B")
except ValueError:
return str(value)

def get_file_sizes():
if not any(r for r in records if r.size is None): # All records already have a size
Expand Down Expand Up @@ -181,8 +184,8 @@ def json(records, **_):
"dataset": ("title", "dataset"),
"full_path": ("full_path", None),
"path": ("path", None),
"short_name": ("dataset_name", "dataset name"),
"dataset_name": ("dataset_name", "dataset name"),
"short_name": ("dataset_slug", "dataset slug"),
"dataset_slug": ("dataset_slug", "dataset slug"),
"size": ("size", None),
"lfs": ("is_lfs", "lfs"),
"source": ("source", None),
Expand Down
Loading

0 comments on commit b31ade0

Please sign in to comment.