Add timestamps to downloaded files #514

Ovler-Young · 2025-02-11T05:09:36Z

Set downloaded files' modified time from API

Pass BBCourseContent.modified timestamp through download hierarchy
Apply timestamp using os.utime() after file writes

This change preserves the content modification times reported by the Blackboard API instead of using local download time. Files will now show their actual last updated time from the learning platform.

Affected components:

content/document.py
content/attachment.py
content/base.py

Close #486

Fix tester problems

Ovler-Young · 2025-02-11T20:46:23Z

This seems to be not working

(cherry picked from commit 1841bb4)

Ovler-Young · 2025-02-11T23:15:15Z

It works now!

CHANGELOG.md

sanjacob · 2025-02-12T23:37:13Z

This is really good. If folders also have a modification time, it might be worth to set it too, unless this is already done by the OS upon changing one of the children. That leaves links and webdav attachments, which I guess aren't as important (and in the case of the latter I don't even think it would be correct), up for your consideration. Also, you are free to add your user tag on the CHANGELOG if you desire. Overall great PR though. Seems to pass tests too.

sanjacob · 2025-02-12T23:38:47Z

Unrelated, but if you are able to test #510 I would really appreciate it.

Ovler-Young · 2025-02-12T23:59:19Z

If folders also have a modification time, it might be worth to set it too, unless this is already done by the OS upon changing one of the children.

Folder time is not set by changing children. I'll need to check if similar change will work.

That leaves links and webdav attachments, which I guess aren't as important (and in the case of the latter I don't even think it would be correct), up for your consideration.

I'll set up links, as it has content: BBCourseContent. As for webdav attachments, I'll also try to add the timestamps.

PR 510

I checked some of my ultraDocumentBody but don't seem to have similar problem as original issue. I'll test if I can repeat this problem.

Ovler-Young · 2025-02-13T04:18:40Z

html files still have no correct timestamps. ~~Will fix tomorrow~~ Fixed

(cherry picked from commit 302e1e4)

… file timestamps accordingly, for folders

Ovler-Young · 2025-02-13T05:03:06Z

I've added the timestamp whereever have mkdir. I think that's it?

Please tell me if there still have anything missing.

Build could be found here https://github.com/Ovler-Young/BlackboardSync/actions/runs/13300762864/job/37141668562

sanjacob · 2025-02-13T07:59:45Z

blackboard_sync/content/course.py

@@ -1,6 +1,7 @@
 import logging
 from pathlib import Path
 from datetime import datetime
+import os


This import seems out of order here, better at the top

sanjacob · 2025-02-13T08:00:39Z

blackboard_sync/content/course.py

@@ -41,6 +43,11 @@ def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
        for child in self.children:
            child.write(path, executor)

+        if self.modified_time:
+            timestamp = self.modified_time.timestamp()
+            path.touch(exist_ok=True)


Surely the folder exists by this point. So there is no use in touching it

sanjacob · 2025-02-13T08:01:55Z

blackboard_sync/content/content.py

+        if (self.modified_time and
+                (self.handler is not None or self.body is not None)):
+            timestamp = self.modified_time.timestamp()
+            path.touch(exist_ok=True)


I think by this point path is guaranteed to exist (other than by destructive user action) so no need to touch

sanjacob

Thank you for adding the folder, link and webdav modification times. However, there are some things I would like you to change. Most important is to not add unnecessary code. For instance, in document.py we are doing the os.utime operation when this can already be handled down the line by attachment.write. There are also some imports which seem out of order and one instance of a name not being consistent with other classes. Thank you for your help, however, and feel free to make the changes whenever you can.

sanjacob · 2025-02-13T08:04:03Z

blackboard_sync/content/document.py

@@ -37,6 +41,11 @@ def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
        for attachment in self.attachments:
            attachment.write(path, executor)


I think it's better to do this in the write method, as it can just be handled by write_base

So just pass it to attachment.write

sanjacob · 2025-02-13T08:05:21Z

blackboard_sync/content/externallink.py

@@ -31,7 +32,7 @@ def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
            body = self.create_unix_body(self.url)
            path = path.with_suffix(".desktop")

-        super().write_base(path, executor, body)
+        super().write_base(path, executor, body, self.modified_time)


This is exactly how it should be

sanjacob · 2025-02-13T08:05:37Z

blackboard_sync/content/folder.py

@@ -1,4 +1,5 @@
 from pathlib import Path
+import os


Again, this import seems better at the top

sanjacob · 2025-02-13T08:06:11Z

blackboard_sync/content/folder.py

@@ -11,9 +12,10 @@
 class Folder:
    """Content of type `x-bb-folder`."""

-    def __init__(self, _: BBCourseContent, api_path: BBContentPath,
+    def __init__(self, coursecontent: BBCourseContent, api_path: BBContentPath,


Rename to content, as in other instances

sanjacob · 2025-02-13T08:06:54Z

blackboard_sync/content/folder.py

@@ -28,6 +30,10 @@ def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
        for child in self.children:
            child.write(path, executor)

+        if self.modified_time:


This is fine - since folder doesn't inherit from BStream or FStream

sanjacob · 2025-02-13T08:07:17Z

blackboard_sync/content/webdav.py

@@ -113,11 +113,12 @@ def __init__(self, link: Link, job: DownloadJob) -> None:
        self.valid = validate_webdav_response(self.stream, link.href,
                                              job.session.instance_url)

-    def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
+    def write(self, path: Path,
+              executor: ThreadPoolExecutor, modified_time=None) -> None:


Lacking type datetime | None

sanjacob · 2025-02-13T08:08:13Z

blackboard_sync/content/attachment.py

@@ -1,5 +1,6 @@
 import uuid
 import mimetypes
+from datetime import datetime


This import would be better with the other imports of the form 'from x import y', I usually order by length

Ovler-Young · 2025-02-17T03:08:18Z

Okay! I'll make chages, perhap during the Spring Break!

sanjacob · 2025-02-17T09:46:45Z

Sounds good!

Ovler-Young added 3 commits February 11, 2025 05:00

Add timestamps to downloaded files

bf3f595

Fix modified time assignment for attachments in Document class

dccd72f

Fix tester problems

Update test to include modified_time parameter for Attachment mock calls

3a43517

Ovler-Young marked this pull request as ready for review February 11, 2025 05:29

Add modified time handling in file write operations

d13cb6d

(cherry picked from commit 1841bb4)

sanjacob reviewed Feb 12, 2025

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

Add username

c432d89

Ovler-Young added 2 commits February 13, 2025 01:46

Pass modified time to write methods in folders, webdav, and links

1516409

fix lint

047c213

Ovler-Young force-pushed the modified-time branch from 1280fc8 to 047c213 Compare February 13, 2025 02:11

Fix modified_time assignment in Folder initialization

59dcf03

Refactor ContentBody initialization to set modified_time from content

db1431f

(cherry picked from commit 302e1e4)

Ovler-Young force-pushed the modified-time branch 2 times, most recently from c1437ac to ddc21aa Compare February 13, 2025 04:55

Set modified_time in Content, Course, and Document classes and update…

d04ae42

… file timestamps accordingly, for folders

Ovler-Young force-pushed the modified-time branch from ddc21aa to d04ae42 Compare February 13, 2025 05:00

sanjacob reviewed Feb 13, 2025

View reviewed changes

sanjacob requested changes Feb 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add timestamps to downloaded files #514

Add timestamps to downloaded files #514

Ovler-Young commented Feb 11, 2025

Ovler-Young commented Feb 11, 2025

Ovler-Young commented Feb 11, 2025

sanjacob commented Feb 12, 2025

sanjacob commented Feb 12, 2025

Ovler-Young commented Feb 12, 2025

Ovler-Young commented Feb 13, 2025 •

edited

Loading

Ovler-Young commented Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025 •

edited

Loading

sanjacob Feb 13, 2025

sanjacob left a comment

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

sanjacob Feb 13, 2025

Ovler-Young commented Feb 17, 2025

sanjacob commented Feb 17, 2025

		@@ -37,6 +41,11 @@ def write(self, path: Path, executor: ThreadPoolExecutor) -> None:
		for attachment in self.attachments:
		attachment.write(path, executor)

Add timestamps to downloaded files #514

Are you sure you want to change the base?

Add timestamps to downloaded files #514

Conversation

Ovler-Young commented Feb 11, 2025

Ovler-Young commented Feb 11, 2025

Ovler-Young commented Feb 11, 2025

sanjacob commented Feb 12, 2025

sanjacob commented Feb 12, 2025

Ovler-Young commented Feb 12, 2025

Ovler-Young commented Feb 13, 2025 • edited Loading

Ovler-Young commented Feb 13, 2025

Choose a reason for hiding this comment

sanjacob Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sanjacob left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ovler-Young commented Feb 17, 2025

sanjacob commented Feb 17, 2025

Ovler-Young commented Feb 13, 2025 •

edited

Loading

sanjacob Feb 13, 2025 •

edited

Loading