Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move package API requests to one file, consolidate naming and internal API #2154

Merged
merged 16 commits into from
Apr 9, 2021

Conversation

fiskus
Copy link
Member

@fiskus fiskus commented Apr 7, 2021

I need this refactoring because I want to be able to validate params for every package API request in one place

@codecov
Copy link

codecov bot commented Apr 7, 2021

Codecov Report

Merging #2154 (e94fead) into master (836e812) will increase coverage by 0.02%.
The diff coverage is 35.38%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2154      +/-   ##
==========================================
+ Coverage   47.36%   47.38%   +0.02%     
==========================================
  Files         439      440       +1     
  Lines       21068    21095      +27     
  Branches     2432     2432              
==========================================
+ Hits         9978     9996      +18     
- Misses      10180    10189       +9     
  Partials      910      910              
Flag Coverage Δ
api-python 89.87% <ø> (ø)
catalog 16.27% <35.38%> (+0.11%) ⬆️
lambda 93.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
catalog/app/containers/Bucket/PackageCopyDialog.js 0.00% <0.00%> (ø)
...talog/app/containers/Bucket/PackageCreateDialog.js 0.00% <0.00%> (ø)
...g/app/containers/Bucket/PackageDirectoryDialog.tsx 0.00% <0.00%> (ø)
...alog/app/containers/Bucket/PackageUpdateDialog.tsx 0.00% <0.00%> (ø)
catalog/app/containers/Bucket/requests/package.ts 38.88% <38.88%> (ø)
.../containers/Bucket/PackageDialog/PackageDialog.tsx 26.00% <100.00%> (-0.59%) ⬇️
catalog/app/containers/Bucket/requests/index.ts 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 836e812...e94fead. Read the comment docs.

@fiskus fiskus requested a review from nl0 April 7, 2021 08:40
Copy link
Member

@nl0 nl0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the general direction is right, but i think we should

  1. adjust naming
  2. remove unnecessary level of abstraction by calling req directly

catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
}): Promise<O>
}

const uploadManifest = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think this function should exist

  1. its name doesnt reflect all its use cases, it looks like a leaked abstraction;
  2. it doesnt have enough logic, i think it's better to just call req directly (you can parameterize body and response types to add type checking there)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a place where I can validate the request body with Schema

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can validate it the same way by using type arg for req

catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
catalog/app/containers/Bucket/requests/packageUpload.ts Outdated Show resolved Hide resolved
@fiskus
Copy link
Member Author

fiskus commented Apr 7, 2021

remove unnecessary level of abstraction by calling req directly

I can't do this, because my goal was to consolidate requests in one function where I can validate the request body

@fiskus fiskus requested a review from nl0 April 7, 2021 15:35
@nl0
Copy link
Member

nl0 commented Apr 7, 2021

I can't do this, because my goal was to consolidate requests in one function where I can validate the request body

but requests are different, so are the bodies. you can type-check request body by simply using type arg when calling req directly.

@nl0
Copy link
Member

nl0 commented Apr 7, 2021

ok, i see that by "validate" you probably mean some runtime validation, not type-checking, but i still dont quite get it

@nl0 nl0 mentioned this pull request Apr 7, 2021
@fiskus
Copy link
Member Author

fiskus commented Apr 7, 2021

function uploadManifest(req, s3, endpoint, workflow, body) {
  const schema = await s3.fetchFile(workflow.objectSchemaurl)
  const error = validateBody(schema)
  if (error.length) throw error
  return req(endpoint, body)
}

function useCreatePackage(body) {
  const req = useAPIConnector()
  const s3 = useS3()
  return uploadManifest(req, s3, "/packages", workflow, body)
}

function useCopyPackage(body) {
  const req = useAPIConnector()
  const s3 = useS3()
  return uploadManifest(req, s3,"/packages/promote",  workflow, body)
}

@fiskus
Copy link
Member Author

fiskus commented Apr 7, 2021

I don't have working code for the next PR, but I wanted something like this ^

@nl0
Copy link
Member

nl0 commented Apr 7, 2021

ok now i get it, but

  1. uploadManifest doesnt seem like an appropriate name for this function
  2. probably you dont need to type-check request body if you plan to validate it in runtime against a schema

@fiskus
Copy link
Member Author

fiskus commented Apr 7, 2021

Do you have in mind what name could be appropriate?

@nl0
Copy link
Member

nl0 commented Apr 7, 2021

Do you have in mind what name could be appropriate?

dunno, smth like requestWithValidation maybe, but it's your call since you have better idea of its potential use cases and the logic you want to add there

@fiskus
Copy link
Member Author

fiskus commented Apr 7, 2021

I can be confident about an object's shape with type-checking, but Schema gives more possibilities like, for example, checking number values ({ min: 0, max: 256} for filesize).
Also, Schema can be different for different workflows

@nl0
Copy link
Member

nl0 commented Apr 7, 2021

I can be confident about an object's shape with type-checking, but Schema gives more possibilities like, for example, checking number values ({ min: 0, max: 256} for filesize).
Also, Schema can be different for different workflows

true, but you must somehow guarantee that the schema is compatible with the type

@fiskus fiskus merged commit b97f574 into master Apr 9, 2021
@fiskus fiskus deleted the package-api-requests branch April 9, 2021 08:18
nl0 added a commit that referenced this pull request May 4, 2021
* master: (54 commits)
  Use stable nginx version for catalog image (#2182)
  Ability to add S3 folders / files to package (#2171)
  lambda for adding S3 data to existing package (#2180)
  use github tarball for faster installation (#2181)
  Bump py from 1.7.0 to 1.10.0 in /lambdas/es/indexer (#2176)
  Bump py from 1.8.0 to 1.10.0 in /lambdas/s3select (#2177)
  Bump py from 1.8.0 to 1.10.0 in /lambdas/thumbnail (#2178)
  Allow unicode characters for package routes by allowing any character (#2179)
  Additional NotFoundPage scoped to Bucket (#2175)
  Docs: fix catalog config path (#2168)
  rework pkgpush auth (#2170)
  Use AWS credentials for directory package and copy package submit (#2172)
  Document package push limitations in catalog [ci skip] (#2161)
  Preview warnings accordion (#2167)
  tweak warning text (#2169)
  Copy tweaks (#2164)
  Don't crash pkgselect for empty manifests (#2147)
  add codecov config (#2155)
  Simplify warning messages for package name (#2134)
  Move package API requests to one file, consolidate naming and internal API (#2154)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants