Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSConnect: speed up pin_search by allowing using guid behind the scenes #79

Open
machow opened this issue Apr 11, 2022 · 2 comments
Open

Comments

@machow
Copy link
Collaborator

machow commented Apr 11, 2022

Currently, python pins with RSConnect needs to look up the user and content guid for actions like

fs.info("michael.chow/mtcars")

however, looking up the user guid requires iterating through all users. This is an expensive operation that R pins does as little as possible by using the guids whenver possible (through a local cache file).

Let's support something like

# similar protocol format to sqlalchemy uris
# may need to use "rsc://guid+<user_guid>
fs.info("rsc+guid://<user_guid>")                  # GET v1/users/{guid}
fs.info("rsc+guid://<user_guid>/<content_name>")   # GET v1/content?name=<content_name>?user_guid=<user_guid>

fs.info("rsc+guid://content/<content_guid>")       # GET v1/content/<content_guid>

Note that we must meet these criteria...

  • Files are cached correctly (e.g. could use the guids for everything in the cache)
  • Users do not need to specify guid in pin_* methods
  • We could allow users to specify guid in pin_* methods if it's not a lot of work. Otherwise, could start by having pins use it as an internal fast path.
@machow machow added this to pins Apr 11, 2022
@machow machow added the be:rsc label Apr 11, 2022
@machow machow changed the title RSConnect: allow using guid behind the scenes RSConnect: speed up pin_search by allowing using guid behind the scenes Apr 11, 2022
@machow machow added the .ux label Apr 11, 2022
@machow
Copy link
Collaborator Author

machow commented Apr 20, 2022

For now, let's inform people how many pins match the search, and that the search could take a while to return resutls.

@hadley
Copy link
Member

hadley commented Nov 2, 2022

Ideally I think we'd fix this upstream on the connect side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants