Support language dataset for DmTorchDataset #1592
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…penvinotoolkit#1582) I ran into an issue where plugins loaded into the `Environment` are not retained after merging. Turns out a generator is being passed to [Environment.merge](https://github.com/openvinotoolkit/datumaro/blob/30b1add52d6fe458dba5e32e1f63ffeea999ad7b/src/datumaro/components/environment.py#L271) instead of the expected Sequence. This meant that custom plugins were never registered into the merged environment. Reproducible example: ```python from datumaro.components.dataset import Dataset from datumaro.components.environment import Environment from datumaro.components.exporter import Exporter from datumaro.components.hl_ops import HLOps from datumaro.components.media import Image class MyPlugin(Exporter): pass environment = Environment() environment.exporters.register('my_plugin', MyPlugin) dataset1 = Dataset(media_type=Image, env=environment) datasets = [dataset1, dataset1.clone()] merged = HLOps.merge(*datasets) assert 'my_plugin' in dataset1.env.exporters # Passes assert 'my_plugin' in merged.env.exporters # Fails ``` --------- Co-authored-by: Sooah Lee <[email protected]>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.2 to 3.26.4. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's changelog</a>.</em></p> <blockquote> <h1>CodeQL Action Changelog</h1> <p>See the <a href="https://github.com/github/codeql-action/releases">releases page</a> for the relevant changes to the CodeQL CLI and language packs.</p> <p>Note that the only difference between <code>v2</code> and <code>v3</code> of the CodeQL Action is the node version they support, with <code>v3</code> running on node 20 while we continue to release <code>v2</code> to support running on node 16. For example <code>3.22.11</code> was the first <code>v3</code> release and is functionally identical to <code>2.22.11</code>. This approach ensures an easy way to track exactly which features are included in different versions, indicated by the minor and patch version numbers.</p> <h2>[UNRELEASED]</h2> <p>No user facing changes.</p> <h2>3.26.4 - 21 Aug 2024</h2> <ul> <li><em>Deprecation:</em> The <code>add-snippets</code> input on the <code>analyze</code> Action is deprecated and will be removed in the first release in August 2025. <a href="https://redirect.github.com/github/codeql-action/pull/2436">#2436</a></li> <li>Fix an issue where the disk usage system call used for telemetry would fail on MacOS ARM machines with System Integrity Protection disabled, and then surface a warning. The system call is now disabled for these machines. <a href="https://redirect.github.com/github/codeql-action/pull/2434">#2434</a></li> </ul> <h2>3.26.3 - 19 Aug 2024</h2> <ul> <li>Fix an issue where the CodeQL Action could not write diagnostic messages on Windows. This issue did not impact analysis quality. <a href="https://redirect.github.com/github/codeql-action/pull/2430">#2430</a></li> </ul> <h2>3.26.2 - 14 Aug 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.18.2. <a href="https://redirect.github.com/github/codeql-action/pull/2417">#2417</a></li> </ul> <h2>3.26.1 - 13 Aug 2024</h2> <p>No user facing changes.</p> <h2>3.26.0 - 06 Aug 2024</h2> <ul> <li><em>Deprecation:</em> Swift analysis on Ubuntu runner images is no longer supported. Please migrate to a macOS runner if this affects you. <a href="https://redirect.github.com/github/codeql-action/pull/2403">#2403</a></li> <li>Bump the minimum CodeQL bundle version to 2.13.5. <a href="https://redirect.github.com/github/codeql-action/pull/2408">#2408</a></li> </ul> <h2>3.25.15 - 26 Jul 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.18.1. <a href="https://redirect.github.com/github/codeql-action/pull/2385">#2385</a></li> </ul> <h2>3.25.14 - 25 Jul 2024</h2> <ul> <li>Experimental: add a new <code>start-proxy</code> action which starts the same HTTP proxy as used by <a href="https://github.com/github/dependabot-action"><code>github/dependabot-action</code></a>. Do not use this in production as it is part of an internal experiment and subject to change at any time. <a href="https://redirect.github.com/github/codeql-action/pull/2376">#2376</a></li> </ul> <h2>3.25.13 - 19 Jul 2024</h2> <ul> <li>Add <code>codeql-version</code> to outputs. <a href="https://redirect.github.com/github/codeql-action/pull/2368">#2368</a></li> <li>Add a deprecation warning for customers using CodeQL version 2.13.4 and earlier. These versions of CodeQL were discontinued on 9 July 2024 alongside GitHub Enterprise Server 3.9, and will be unsupported by CodeQL Action versions 3.26.0 and later and versions 2.26.0 and later. <a href="https://redirect.github.com/github/codeql-action/pull/2375">#2375</a> <ul> <li>If you are using one of these versions, please update to CodeQL CLI version 2.13.5 or later. For instance, if you have specified a custom version of the CLI using the 'tools' input to the 'init' Action, you can remove this input to use the default version.</li> <li>Alternatively, if you want to continue using a version of the CodeQL CLI between 2.12.6 and 2.13.4, you can replace <code>github/codeql-action/*@V3</code> by <code>github/codeql-action/*@v3.25.13</code> and <code>github/codeql-action/*@v2</code> by <code>github/codeql-action/*@v2.25.13</code> in your code scanning workflow to ensure you continue using this version of the CodeQL Action.</li> </ul> </li> </ul> <h2>3.25.12 - 12 Jul 2024</h2> <ul> <li>Improve the reliability and performance of analyzing code when analyzing a compiled language with the <code>autobuild</code> <a href="https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes">build mode</a> on GitHub Enterprise Server. This feature is already available to GitHub.com users. <a href="https://redirect.github.com/github/codeql-action/pull/2353">#2353</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/github/codeql-action/commit/f0f3afee809481da311ca3a6ff1ff51d81dbeb24"><code>f0f3afe</code></a> Merge main into releases/v3 (<a href="https://redirect.github.com/github/codeql-action/issues/2437">#2437</a>)</li> <li><a href="https://github.com/github/codeql-action/commit/e3543591a523b40efe5604333b75986a211c9d50"><code>e354359</code></a> Update changelog for v3.26.4</li> <li><a href="https://github.com/github/codeql-action/commit/ae01f807ca7dae87a61d9c5e36c06676ade69b1b"><code>ae01f80</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2436">#2436</a> from rvermeulen/rvermeulen/deprecate-add-snippets</li> <li><a href="https://github.com/github/codeql-action/commit/72bc3f7f619fdbada5d8d589b47c52c32f9ad097"><code>72bc3f7</code></a> Address incorrect changelog location</li> <li><a href="https://github.com/github/codeql-action/commit/7388c476ae17d48d06bd893197f2865e72233827"><code>7388c47</code></a> Merge branch 'main' into rvermeulen/deprecate-add-snippets</li> <li><a href="https://github.com/github/codeql-action/commit/d7c48ef5a8182e33f305da9abd0f642eb687244d"><code>d7c48ef</code></a> Add link to PR deprecating <code>add-snippets</code> to CHANGELOG.md</li> <li><a href="https://github.com/github/codeql-action/commit/ec21b8f8a46388ea7527dbba9f40d839d52aabaf"><code>ec21b8f</code></a> Update changelog with deprecation.</li> <li><a href="https://github.com/github/codeql-action/commit/4067cdab784c667cf1b7fa95169f3a0e0a381d63"><code>4067cda</code></a> Add deprecation message to <code>add-snippets</code> input.</li> <li><a href="https://github.com/github/codeql-action/commit/202b3b97bf2ab4efa04bf0fde6e14313688b956b"><code>202b3b9</code></a> Stop checking disk usage for MacOS ARM with SIP disabled (<a href="https://redirect.github.com/github/codeql-action/issues/2434">#2434</a>)</li> <li><a href="https://github.com/github/codeql-action/commit/512e3066dd919110d2039970efdc5443f96d2eb5"><code>512e306</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2404">#2404</a> from github/marcogario/proxy_64</li> <li>Additional commits viewable in <a href="https://github.com/github/codeql-action/compare/429e1977040da7a23b6822b13c129cd1ba93dbb2...f0f3afee809481da311ca3a6ff1ff51d81dbeb24">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
sooahleex
changed the title
wonjuleee
approved these changes
Aug 26, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## releases/1.9.0 #1592 +/- ##
=================================================
Coverage ? 81.06%
=================================================
Files ? 278
Lines ? 32517
Branches ? 6607
=================================================
Hits ? 26360
Misses ? 4701
Partials ? 1456
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
DmTorchDatasetHow to test
Checklist
License
Feel free to contact the maintainers if that's a concern.