Include size in pkgselect responses #1744
Conversation
Updating tests to account for the addition of size and physical key in the lambda response. Switching from `to_json` to `to_dict` in file_list_to_folder to avoid double encoding the folder view.
Codecov Report
@@ Coverage Diff @@
## master #1744 +/- ##
==========================================
+ Coverage 89.44% 89.46% +0.02%
==========================================
Files 62 62
Lines 7190 7215 +25
==========================================
+ Hits 6431 6455 +24
- Misses 759 760 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| folder.reset_index(inplace=True) | ||
| folder.rename(columns={0: 'logical_key'}, inplace=True) | ||
| prefixes = folder[folder.logical_key.str.contains('/')].to_dict(orient='records') | ||
| objects = folder[~folder.logical_key.str.contains('/')].to_dict(orient='records') |
There was a problem hiding this comment.
What happens in keys with successive // to both the regex in the groupby and here?
There was a problem hiding this comment.
// is illegal in logical keys isn't it?
|
@kevinemoore can you return meta for the package / prefix as well? |
There was a problem hiding this comment.
-
let's not return
physical_keys for prefixes, bc they are not relevant (they actually point to random objects under the prefix) and waste bandwidth -
when trying to query an object the api throws
local variable 'result' referenced before assignment -
should we return the whole meta object here or just
user_meta? @akarve
Moving metadata for prefixes and top-level package metadata under `contents` in the response so that meta will show up in the same place for the detail and folder views.
|
| objects = folder[~folder.str.endswith('/')].sort_values().tolist() | ||
| except AttributeError: | ||
| groups = df.groupby(df.logical_key.str.extract('([^/]+/?).*')[0], dropna=True) | ||
| folder = groups.agg( |
There was a problem hiding this comment.
Does this automatically skip undefined values?
Co-authored-by: Aneesh Karve <[email protected]>
…nto pkgselect-size
Description
Add size to the pkgselect response. The implementation still does most of the work in s3select and pandas (C), but could be slow for large responses (e.g., a folder with many objects).
TODO