Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Follow redirect location as new referrers for nested module imports #2031

Merged
merged 7 commits into from
Apr 2, 2019

Conversation

kevinkassimo
Copy link
Contributor

@kevinkassimo kevinkassimo commented Apr 1, 2019

Initial attempt to resolve #1742 and #2021. Both examples in the 2 issues seem to be working now:

$ ./target/debug/deno --info https://import-meta.now.sh/redirect.js
Downloading https://import-meta.now.sh/redirect.js
Downloading https://import-meta.now.sh/sub/foo/redirect2.js
Downloading https://import-meta.now.sh/a.js
Downloading https://import-meta.now.sh/up/b.js
Downloading https://import-meta.now.sh/up/up/c.js
Downloading https://import-meta.now.sh/up/up/up/d.js
local: /Users/kun/Library/Caches/deno/deps/https/import-meta.now.sh/sub/final1.js
type: JavaScript
deps:
https://import-meta.now.sh/sub/final1.js
  └─┬ https://import-meta.now.sh/up/up/final2.js
    ├── https://import-meta.now.sh/a.js
    ├── https://import-meta.now.sh/up/b.js
    ├── https://import-meta.now.sh/up/up/c.js
    └── https://import-meta.now.sh/up/up/up/d.js

It is achieved by extending *.mime files into *.headers.json files, which is now of JSON format and might optionally contain a field redirect_to that points to the location of the final redirect.

For example, for https://import-meta.now.sh/redirect.js, $DENO_DIR/deps/https/import-meta.now.sh/redirect.js is never created. Instead, $DENO_DIR/deps/https/import-meta.now.sh/redirect.js.headers.json is created with the following content:

{"redirect_to":"https://import-meta.now.sh/sub/final1.js"}

which points to the actual cached file location (https://import-meta.now.sh/sub/final1.js)

TODO:

  • Ensure this also works for TypeScript (might need help from @kitsonk)
  • Find and fix potentially introduced bugs
  • Add tests

(Struggling to catch up with recent codebase changes so there is high potential of new bugs introduced)

cli/deno_dir.rs Outdated
/// download will be written to "filename". This happens no matter the value of
/// use_cache.
fn get_source_code_async(
self: &Self,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer if we could keep these functions out of DenoDir and rather provide explicit parameters for whatever it needs. (During a recent refactoring in this file I discovered these methods to be rather tangled up without a particularly good reason.)
"filename" should already give the location of the cache. You should be able to use that for resolving the "mime file" ?

cli/deno_dir.rs Outdated Show resolved Hide resolved
// May not exist. DON'T unwrap.
let _ = std::fs::remove_file(&metadata_filename);
let p = PathBuf::from(filename);
// TODO(kevinkassimo): consider introduce serde::Deserialize to make things simpler.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need to do this soon. There's another place in the source map handler where I'm manually parsing JSON like this.

cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
@kevinkassimo kevinkassimo marked this pull request as ready for review April 1, 2019 23:24
@kevinkassimo kevinkassimo changed the title [WIP] Follow redirect location as new referrers for nested module imports Follow redirect location as new referrers for nested module imports Apr 1, 2019
cli/deno_dir.rs Outdated Show resolved Hide resolved
@ry
Copy link
Member

ry commented Apr 2, 2019

@kevinkassimo Is this ready or are you still working on it? I'm ready to land.

cli/deno_dir.rs Outdated Show resolved Hide resolved
@kevinkassimo
Copy link
Contributor Author

@ry should be ready

cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
cli/deno_dir.rs Outdated Show resolved Hide resolved
Copy link
Member

@ry ry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for this important and difficult fix.

We need to refactor DenoDir soon. I will start an issue and let's scope out what a new implementation would need.

@ry ry merged commit 534b8d3 into denoland:master Apr 2, 2019
@kevinkassimo kevinkassimo deleted the import/redirect branch December 27, 2019 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Relative import from redirected url
2 participants