Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow modules to be loaded from local directory #916

Closed
1 of 6 tasks
tvanhens opened this issue Oct 29, 2019 · 5 comments
Closed
1 of 6 tasks

Allow modules to be loaded from local directory #916

tvanhens opened this issue Oct 29, 2019 · 5 comments
Labels
closed-for-staleness effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2

Comments

@tvanhens
Copy link

🚀 Feature Request

Affected Languages

  • TypeScript or Javascript
  • Python
  • Java
  • .NET (C#, F#, ...)

General Information

  • JSII Version: n/a

  • Platform: n/a

  • I may be able to implement this feature request

  • This feature might incur a breaking change

Description

While building a JSii wrapper for a Clojure CDK API, I've come to realize that it is beneficial to rely on npm & node_modules for JSii libraries. The bundling of the Java JSii runtime with the maven CDK modules opens the door to dependency conflicts that can be avoided by relying directly on node_modules. Additionally, maven modules are hard to load dynamically into a JVM which makes it hard to dynamically work with CDK constructs.

Right now the JSii protocol only allows for archived modules to be loaded (from the classpath in the Java case). If the JSii protocol was expanded to allow modules to be loaded locally from a node_modules folder it would make it easier to enable more dynamic workflows in REPL-friendly languages.

Proposed Solution

I can see two plausible solutions - both non-breaking:

  1. Amend the load request to allow an optional boolean local argument. When this argument is supplied as true then modules are loaded from the local directory's node_modules folder. This would keep the surface area of the protocol largely unchanged and allows for a non-breaking change by falling back to the current behavior when the argument is not present.

  2. Add a new require request that would allow a local module to be loaded. This increases the surface area of the API but prevents decreasing the complexity of the load API request. The commonality that would be shared between require and load could be factored into shared private functions to prevent drift between the two APIs.

I'm happy to put together a PR for either of these solutions (or an alternative solution) if this change is desirable.
@tvanhens tvanhens added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Oct 29, 2019
@RomainMuller
Copy link
Contributor

Hey,

I'd like to understand a bit more on your Clojure implementation here. Are you building on top of the Java libraries, or building a bindings generator for Clojure separately? (Note that while I know what Clojure is, I haven't quite used it for anything significant at this point).

Besides this, jsii may not rely on node forever. Having to have a compatible node runtime when developing in another language is a burden that we might eventually be able to do away with. This would imply switching to something maybe like WebAssembly in the future, where it won't be possible anymore to actually "load npm". We're actively trying to avoid customer-side dependency on npm precisely to avoid getting locked into it by features we cannot port out to a different execution engine.

@tvanhens
Copy link
Author

Ah fair enough, hadn't thought about the underlying runtime changing that significantly. We are building on top of the jsii runtime for Java but not the generated classes. Instead, since Clojure can create namespaces and bindings at runtime, we generate the API wrappers in memory. Since these bindings don't have to be packaged and are generated from the underlying artifacts it is possible to pull new jsii artifacts in at runtime while using the REPL:

See https://github.com/StediInc/cdk-clj/blob/master/src/stedi/cdk/import.clj for an example

This uses the .jsii file contained in each jsii module to dynamically construct a Clojure-friendly API. I suppose another option for us would be to pull in the packages dynamically from GitHub releases, the drawback here is that we wouldn't be able to leverage the standardized coordinates of a package management system to retrieve non-cdk-core artifacts in a known way.

@RomainMuller
Copy link
Contributor

@tvanhens - supposedly, all those packages (including 3P) are published to npmjs.com and can hence be retrieved using the standard npmjs.com API. The libraries are packaged as .tgz archives and the .jsii file is known to be located at package/.jsii (from memory, but wherever it's at in one such package is where you should look it up in any others - at least for now, and if that changes I reckon we'd put a field in the package.json field to actually indicate the desired location within the package).

@RomainMuller
Copy link
Contributor

And by the way - if you're interested to contribute your Clojure handling directly into "mainland jsii", we'd welcome that ;)

In some future (talking months, not weeks here), I want to make the language support "pluggable" and this might give you more of the flexibility you require.

@SomayaB SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Nov 18, 2019
@RomainMuller RomainMuller added the effort/large Large work item – several weeks of effort label Jan 24, 2020
@RomainMuller RomainMuller removed their assignment Jun 24, 2021
mergify bot pushed a commit that referenced this issue Aug 1, 2022
@dependabot
chore(deps): Update attrs requirement from ~=21.2 to >=21.2,<23.0 in …
5275da6 
…/packages/@jsii/python-runtime (#3692)

Updates the requirements on [attrs](https://github.com/python-attrs/attrs) to permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/attrs/releases">attrs's releases</a>.</em></p>
<blockquote>
<h2>22.1.0</h2>
<h1>Highlights</h1>
<p>The main features of this release are:</p>
<ul>
<li>The departure of Python 2.7 (enjoy your retirement!),</li>
<li>and the arrival of Python 3.11.</li>
</ul>
<p>We had loftier goals feature-wise, but didn't want to block others embracing Python 3.11.</p>
<p>❤️ <strong>Huge</strong> thanks to my <a href="https://github.com/sponsors/hynek">GitHub sponsors</a>, <a href="https://tidelift.com/subscription/pkg/pypi-attrs">Tidelift subscribers</a>, and <a href="https://ko-fi.com/the_hynek">Ko-fi buyers</a>! ❤️</p>
<p>None of my projects would exist in their current form without you!</p>
<h1>Full Changelog</h1>
<h2>Backwards-incompatible Changes</h2>
<ul>
<li>
<p>Python 2.7 is not supported anymore.</p>
<p>Dealing with Python 2.7 tooling has become too difficult for a volunteer-run project.</p>
<p>We have supported Python 2 more than 2 years after it was officially discontinued and feel that we have paid our dues. All version up to 21.4.0 from December 2021 remain fully functional, of course. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/936">#936</a></p>
</li>
<li>
<p>The deprecated <code>cmp</code> attribute of <code>attrs.Attribute</code> has been removed. This does not affect the <em>cmp</em> argument to <code>attr.s</code> that can be used as a shortcut to set <em>eq</em> and <em>order</em> at the same time. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/939">#939</a></p>
</li>
</ul>
<h2>Changes</h2>
<ul>
<li>Instantiation of frozen slotted classes is now faster. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/898">#898</a></li>
<li>If an <code>eq</code> key is defined, it is also used before hashing the attribute. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/909">#909</a></li>
<li>Added <code>attrs.validators.min_len()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/916">#916</a></li>
<li><code>attrs.validators.deep_iterable()</code>'s <em>member_validator</em> argument now also accepts a list of validators and wraps them in an <code>attrs.validators.and_()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/925">#925</a></li>
<li>Added missing type stub re-imports for <code>attrs.converters</code> and <code>attrs.filters</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/931">#931</a></li>
<li>Added missing stub for <code>attr(s).cmp_using()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/949">#949</a></li>
<li><code>attrs.validators._in()</code>'s <code>ValueError</code> is not missing the attribute, expected options, and the value it got anymore. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/951">#951</a></li>
<li>Python 3.11 is now officially supported. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/969">#969</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/attrs/blob/main/CHANGELOG.rst">attrs's changelog</a>.</em></p>
<blockquote>
<h2>22.1.0 (2022-07-28)</h2>
<p>Backwards-incompatible Changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>
<p>Python 2.7 is not supported anymore.</p>
<p>Dealing with Python 2.7 tooling has become too difficult for a volunteer-run project.</p>
<p>We have supported Python 2 more than 2 years after it was officially discontinued and feel that we have paid our dues.
All version up to 21.4.0 from December 2021 remain fully functional, of course.
<code>[#936](python-attrs/attrs#936) &lt;https://github.com/python-attrs/attrs/issues/936&gt;</code>_</p>
</li>
<li>
<p>The deprecated <code>cmp</code> attribute of <code>attrs.Attribute</code> has been removed.
This does not affect the <em>cmp</em> argument to <code>attr.s</code> that can be used as a shortcut to set <em>eq</em> and <em>order</em> at the same time.
<code>[#939](python-attrs/attrs#939) &lt;https://github.com/python-attrs/attrs/issues/939&gt;</code>_</p>
</li>
</ul>
<p>Changes
^^^^^^^</p>
<ul>
<li>Instantiation of frozen slotted classes is now faster.
<code>[#898](python-attrs/attrs#898) &lt;https://github.com/python-attrs/attrs/issues/898&gt;</code>_</li>
<li>If an <code>eq</code> key is defined, it is also used before hashing the attribute.
<code>[#909](python-attrs/attrs#909) &lt;https://github.com/python-attrs/attrs/issues/909&gt;</code>_</li>
<li>Added <code>attrs.validators.min_len()</code>.
<code>[#916](python-attrs/attrs#916) &lt;https://github.com/python-attrs/attrs/issues/916&gt;</code>_</li>
<li><code>attrs.validators.deep_iterable()</code>'s <em>member_validator</em> argument now also accepts a list of validators and wraps them in an <code>attrs.validators.and_()</code>.
<code>[#925](python-attrs/attrs#925) &lt;https://github.com/python-attrs/attrs/issues/925&gt;</code>_</li>
<li>Added missing type stub re-imports for <code>attrs.converters</code> and <code>attrs.filters</code>.
<code>[#931](python-attrs/attrs#931) &lt;https://github.com/python-attrs/attrs/issues/931&gt;</code>_</li>
<li>Added missing stub for <code>attr(s).cmp_using()</code>.
<code>[#949](python-attrs/attrs#949) &lt;https://github.com/python-attrs/attrs/issues/949&gt;</code>_</li>
<li><code>attrs.validators._in()</code>'s <code>ValueError</code> is not missing the attribute, expected options, and the value it got anymore.
<code>[#951](python-attrs/attrs#951) &lt;https://github.com/python-attrs/attrs/issues/951&gt;</code>_</li>
<li>Python 3.11 is now officially supported.
<code>[#969](python-attrs/attrs#969) &lt;https://github.com/python-attrs/attrs/issues/969&gt;</code>_</li>
</ul>
<hr />
<h2>21.4.0 (2021-12-29)</h2>
<p>Changes
^^^^^^^</p>
<ul>
<li>Fixed the test suite on PyPy3.8 where <code>cloudpickle</code> does not work.
<code>[#892](python-attrs/attrs#892) &lt;https://github.com/python-attrs/attrs/issues/892&gt;</code>_</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/python-attrs/attrs/commit/b3dfebe2e10b44437c4f97d788fb5220d790efd0"><code>b3dfebe</code></a> Prepare 22.1.0</li>
<li><a href="https://github.com/python-attrs/attrs/commit/c89abcd6e6a826b00898f93e851c96b78c80891f"><code>c89abcd</code></a> It totally is correct</li>
<li><a href="https://github.com/python-attrs/attrs/commit/9f118b7dd5328ee5abd7f3880971ab9554047c3a"><code>9f118b7</code></a> Tune first steps</li>
<li><a href="https://github.com/python-attrs/attrs/commit/a3d7f20e54ba29d5ec73563c167d877351228cf2"><code>a3d7f20</code></a> Polish contributing guide</li>
<li><a href="https://github.com/python-attrs/attrs/commit/899497f165b3c8291b66c9c3a01b64881b69cf39"><code>899497f</code></a> Clarify (c) ownership</li>
<li><a href="https://github.com/python-attrs/attrs/commit/696fd786901a6b82b828dd71e3fd64bfae1014ca"><code>696fd78</code></a> Hyphenate compound adjectives</li>
<li><a href="https://github.com/python-attrs/attrs/commit/65c06831c1eee4d9895511c7db5caffba9d93c0e"><code>65c0683</code></a> Use NG APIs in glossary</li>
<li><a href="https://github.com/python-attrs/attrs/commit/5d84d9a5686e8210b1616447ac05021b6b211157"><code>5d84d9a</code></a> Move mypy config to pyproject.toml</li>
<li><a href="https://github.com/python-attrs/attrs/commit/1590917bf7c49e35bb429d117650f660aa45e0e4"><code>1590917</code></a> Update readme.rst (<a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/986">#986</a>)</li>
<li><a href="https://github.com/python-attrs/attrs/commit/a67c84f51e4e3df875961d287583abaef673eb48"><code>a67c84f</code></a> Add more prominent callout about slots to API docs</li>
<li>Additional commits viewable in <a href="https://github.com/python-attrs/attrs/compare/21.2.0...22.1.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
@github-actions
Copy link
Contributor

This issue has not received any attention in 2 years. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added closing-soon This issue will automatically close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Jun 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
closed-for-staleness effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@RomainMuller @tvanhens @SomayaB