Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make usage of name and slug consistent #3620

Merged
merged 4 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion AUTHORS.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
6 changes: 2 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion QA_PROTOCOL.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
6 changes: 2 additions & 4 deletions docs/_ext/cheatsheet.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2018-2023- Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion docs/changes.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
6 changes: 2 additions & 4 deletions docs/cheatsheet/conf.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
6 changes: 2 additions & 4 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 1 addition & 1 deletion docs/gettingstarted.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
:orphan:

..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/introduction.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/license.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ License

.. code-block:: text

Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/plugins.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/api.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/commands/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/core.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/database.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/gateways.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/dataset_provider.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/datasets.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/projects.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/provenance.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/models/workflow.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/plugins.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/service.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/service_errors.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
..
Copyright 2017-2023 - Swiss Data Science Center (SDSC)
Copyright Swiss Data Science Center (SDSC).
A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
Eidgenössische Technische Hochschule Zürich (ETHZ).

Expand Down
6 changes: 2 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# -*- coding: utf-8 -*-
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
5 changes: 2 additions & 3 deletions renku/command/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
22 changes: 11 additions & 11 deletions renku/command/checks/datasets.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,17 +77,17 @@ def check_missing_files(dataset_gateway: IDatasetGateway, **_):
path = project_context.path / file_.entity.path
file_exists = path.exists() or (file_.is_external and os.path.lexists(path))
if not file_exists:
missing[dataset.name].append(file_.entity.path)
missing[dataset.slug].append(file_.entity.path)

if not missing:
return True, False, None

problems = WARNING + "There are missing files in datasets."

for dataset_name, files in missing.items():
for dataset_slug, files in missing.items():
problems += (
"\n\t"
+ click.style(dataset_name, fg="yellow")
+ click.style(dataset_slug, fg="yellow")
+ ":\n\t "
+ "\n\t ".join(click.style(path, fg="red") for path in files)
)
Expand All @@ -114,9 +114,9 @@ def fix_or_report(dataset):
dataset.unfreeze()
dataset.derived_from = None
dataset.freeze()
communication.info(f"Fixing dataset '{dataset.name}'")
communication.info(f"Fixing dataset '{dataset.slug}'")
else:
invalid_datasets.append(dataset.name)
invalid_datasets.append(dataset.slug)

for dataset in dataset_gateway.get_provenance_tails():
while dataset.derived_from is not None and dataset.derived_from.url_id is not None:
Expand All @@ -137,7 +137,7 @@ def fix_or_report(dataset):
WARNING
+ "There are invalid dataset metadata in the project (use 'renku doctor --fix' to fix them):"
+ "\n\n\t"
+ "\n\t".join(click.style(name, fg="yellow") for name in invalid_datasets)
+ "\n\t".join(click.style(slug, fg="yellow") for slug in invalid_datasets)
+ "\n"
)

Expand Down Expand Up @@ -177,12 +177,12 @@ def check_dataset_files_outside_datadir(fix, dataset_gateway: IDatasetGateway, *
continue

if fix:
communication.info(f"Fixing dataset '{dataset.name}' files.")
communication.info(f"Fixing dataset '{dataset.slug}' files.")
dataset.unfreeze()
for file in detected_files:
dataset.unlink_file(file.entity.path)
dataset.freeze()
add_to_dataset(dataset.name, urls=[file.entity.path for file in detected_files], link=True)
add_to_dataset(dataset.slug, urls=[file.entity.path for file in detected_files], link=True)
else:
invalid_files.extend(detected_files)

Expand Down Expand Up @@ -220,7 +220,7 @@ def check_external_files(fix, dataset_gateway: IDatasetGateway, **_):
for file in dataset.files:
if file.is_external:
external_files.append(file.entity.path)
datasets[dataset.name].append(file)
datasets[dataset.slug].append(file)

if not external_files:
return True, False, None
Expand All @@ -240,7 +240,7 @@ def check_external_files(fix, dataset_gateway: IDatasetGateway, **_):
f"dataset with an external storage backend:\n\t{external_files_str}"
)

for name, files in datasets.items():
file_unlink(name=name, yes=True, dataset_files=files)
for slug, files in datasets.items():
file_unlink(slug=slug, yes=True, dataset_files=files)

return True, False, None
4 changes: 2 additions & 2 deletions renku/command/checks/validate_shacl.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,15 @@ def check_datasets_structure(dataset_gateway: IDatasetGateway, **_):
try:
conform, graph, t = _check_shacl_structure(data)
except (Exception, BaseException) as e:
problems.append(f"Couldn't validate dataset '{dataset.name}': {e}\n\n")
problems.append(f"Couldn't validate dataset '{dataset.slug}': {e}\n\n")
continue

if conform:
continue

ok = False

problems.append(f"{dataset.name}\n\t{_shacl_graph_to_string(graph)}\n")
problems.append(f"{dataset.slug}\n\t{_shacl_graph_to_string(graph)}\n")

if ok:
return True, False, None
Expand Down
5 changes: 2 additions & 3 deletions renku/command/clone.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2018-2023- Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
5 changes: 2 additions & 3 deletions renku/command/config.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
7 changes: 3 additions & 4 deletions renku/command/dataset.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2017-2023 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -39,7 +38,7 @@


def search_datasets_command():
"""Command to get all the datasets whose name starts with the given string."""
"""Command to get all the datasets whose slug starts with the given string."""
return Command().command(search_datasets).require_migration().with_database()


Expand Down
5 changes: 2 additions & 3 deletions renku/command/doctor.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#
# Copyright 2020 - Swiss Data Science Center (SDSC)
# A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
# Copyright Swiss Data Science Center (SDSC). A partnership between
# École Polytechnique Fédérale de Lausanne (EPFL) and
# Eidgenössische Technische Hochschule Zürich (ETHZ).
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
9 changes: 6 additions & 3 deletions renku/command/format/dataset_files.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@ def get_lfs_tracking():
record.is_lfs = False

def naturalize(value) -> str:
return naturalsize(value).upper().replace("BYTES", " B")
try:
return naturalsize(value).upper().replace("BYTES", " B")
except ValueError:
return str(value)

def get_file_sizes():
if not any(r for r in records if r.size is None): # All records already have a size
Expand Down Expand Up @@ -181,8 +184,8 @@ def json(records, **_):
"dataset": ("title", "dataset"),
"full_path": ("full_path", None),
"path": ("path", None),
"short_name": ("dataset_name", "dataset name"),
"dataset_name": ("dataset_name", "dataset name"),
"short_name": ("dataset_slug", "dataset slug"),
"dataset_slug": ("dataset_slug", "dataset slug"),
"size": ("size", None),
"lfs": ("is_lfs", "lfs"),
"source": ("source", None),
Expand Down
Loading