Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catalog: GraphQL for packages #2552

Merged
merged 46 commits into from
Dec 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a0c40d9
utils/useQuery: helper wrapper for urql useQuery
nl0 Nov 1, 2021
df316b0
dont try to load catalog settings in local mode
nl0 Oct 25, 2021
d5b1205
WIP graphql schema changes for packages and related stuff
nl0 Oct 25, 2021
12c667a
shared: update gql schema
nl0 Nov 23, 2021
daea162
graphqlrc: add JsonRecord scalar
nl0 Oct 27, 2021
320f8fb
shared: graphql schema: JsonDict -> JsonRecord
nl0 Nov 23, 2021
a9e9ab8
update generated gql stuff
nl0 Oct 27, 2021
a9e62f3
Bucket/PackageList: get data from graphql
nl0 Oct 27, 2021
1b06c3a
shared: gql schema: add package list order
nl0 Nov 23, 2021
9432e1b
gql generated files: regenerate to include order
nl0 Oct 28, 2021
ea27baa
PackageList: tsify, use gql
nl0 Oct 28, 2021
a985596
Bucket/requests: rm requests migrated to gql
nl0 Oct 28, 2021
4675ac3
cat: adjust gql schema
nl0 Nov 23, 2021
9a2b7be
PackageUpdateDialog: hash is optional
nl0 Oct 30, 2021
9f04060
adjust PackageList
nl0 Oct 30, 2021
ff5a080
PackageRevisions: gql, ts
nl0 Oct 30, 2021
3027bc3
rm obsolete requests
nl0 Oct 30, 2021
c2b9fec
PackageRevisions: useQuery
nl0 Nov 1, 2021
a17a4f7
Bucket/PackageList: useQuery
nl0 Nov 1, 2021
24c96f9
update gql schema
nl0 Nov 1, 2021
aefc3b1
update generated gql stuff
nl0 Nov 1, 2021
cccbfa0
Bucket/PackageRevisions: adjust for gql changes
nl0 Nov 1, 2021
b73d51c
Bucket/PackageTree: ts, gql, refactor
nl0 Nov 1, 2021
cee1a39
Bucket/requests: rm unused code
nl0 Nov 1, 2021
3b9d708
Bucket/errrors: rm unused code
nl0 Nov 1, 2021
c7e4840
Bucket/PackageTree: use gql for logical key resolution
nl0 Nov 2, 2021
396baeb
Bucket/requests: rm unused code
nl0 Nov 2, 2021
b743b74
PackageTree: fix some gql stuff
nl0 Nov 3, 2021
bee1268
PackageTree: fix gql stuff
nl0 Nov 3, 2021
01d99f9
fix requests
nl0 Nov 22, 2021
1ac0042
utils/GraphQL: keys for new types
nl0 Nov 22, 2021
438010b
update gql schema
nl0 Nov 22, 2021
fdf66fc
catalog: further work on packages gql stuff
nl0 Nov 22, 2021
02da532
show package folder size
nl0 Nov 26, 2021
a8fff9d
support relative urls for s3proxy and api gateway
nl0 Nov 28, 2021
65043c3
AWS/Credentials: handle missing Expiration
nl0 Dec 2, 2021
f26bde2
utils/useQuery: more descriptive type var names
nl0 Dec 6, 2021
c820588
address CR feedback
nl0 Dec 6, 2021
b83315c
rm PackageTree.js
nl0 Dec 7, 2021
72a98b6
gql schema: nullable totalEntries and totalBytes
nl0 Dec 14, 2021
5776624
regen gql model
nl0 Dec 14, 2021
8ebf590
PackageRevisions: support nullable totalEntries
nl0 Dec 14, 2021
d287367
support revisions with non-unique hashes
nl0 Dec 20, 2021
374e745
gql schema: Floats for sizes
nl0 Dec 22, 2021
4c170e9
regen gql stuff
nl0 Dec 22, 2021
4c01178
changelog entry
nl0 Dec 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions catalog/.graphqlrc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ extensions:
scalars:
Datetime: Date
Json: utils/types#Json
JsonRecord: utils/types#JsonRecord
Original file line number Diff line number Diff line change
@@ -1,54 +1,71 @@
import * as dateFns from 'date-fns'
import * as R from 'ramda'
import * as React from 'react'
import { useHistory, Link, Redirect } from 'react-router-dom'
import * as RRDom from 'react-router-dom'
import * as M from '@material-ui/core'
import { fade } from '@material-ui/core/styles'
import type { ResultOf } from '@graphql-typed-document-node/core'

import Skeleton from 'components/Skeleton'
import Sparkline from 'components/Sparkline'
import * as AWS from 'utils/AWS'
import * as APIConnector from 'utils/APIConnector'
// import AsyncResult from 'utils/AsyncResult'
// import * as BucketConfig from 'utils/BucketConfig'
import * as Config from 'utils/Config'
import * as Data from 'utils/Data'
// import * as LinkedData from 'utils/LinkedData'
import * as Model from 'model'
import * as BucketPreferences from 'utils/BucketPreferences'
import MetaTitle from 'utils/MetaTitle'
import * as NamedRoutes from 'utils/NamedRoutes'
import StyledLink from 'utils/StyledLink'
import * as SVG from 'utils/SVG'
import * as BucketPreferences from 'utils/BucketPreferences'
import StyledLink from 'utils/StyledLink'
import * as Format from 'utils/format'
import parseSearch from 'utils/parseSearch'
import mkStorage from 'utils/storage'
import { readableQuantity } from 'utils/string'
import useDebouncedInput from 'utils/useDebouncedInput'
import usePrevious from 'utils/usePrevious'
import useQuery from 'utils/useQuery'

import { usePackageCreateDialog } from '../PackageCreateDialog'
import Pagination from '../Pagination'
import { displayError } from '../errors'

import { usePackageCreateDialog } from './PackageCreateDialog'
import Pagination from './Pagination'
import { displayError } from './errors'
import * as requests from './requests'
import PACKAGE_COUNT_QUERY from './gql/PackageCount.generated'
import PACKAGE_LIST_QUERY from './gql/PackageList.generated'

const EXAMPLE_PACKAGE_URL = 'https://docs.quiltdata.com/walkthrough/editing-a-package'

const PER_PAGE = 30

const SORT_OPTIONS = [
{ key: 'modified', label: 'Updated' },
{ key: 'name', label: 'Name' },
]
{
key: 'modified',
value: Model.GQLTypes.PackageListOrder.MODIFIED,
label: 'Updated',
},
{
key: 'name',
value: Model.GQLTypes.PackageListOrder.NAME,
label: 'Name',
},
] as const

type SortMode = typeof SORT_OPTIONS[number]['key']

const DEFAULT_SORT = SORT_OPTIONS[0]

// Possible values are 'modified', 'name'
const storage = mkStorage({ sortPackagesBy: 'SORT_PACKAGES_BY' })

const getSort = (key) => (key && SORT_OPTIONS.find((o) => o.key === key)) || DEFAULT_SORT
const getSort = (key: unknown) => {
if (!key) return DEFAULT_SORT
return SORT_OPTIONS.find((o) => o.key === key) || DEFAULT_SORT
}

type CountsProps = NonNullable<
NonNullable<
ResultOf<typeof PACKAGE_LIST_QUERY>['packages']
>['page'][number]['accessCounts']
>

function Counts({ counts, total }) {
const [cursor, setCursor] = React.useState(null)
function Counts({ counts, total }: CountsProps) {
const [cursor, setCursor] = React.useState<number | null>(null)
const t = M.useTheme()
const xs = M.useMediaQuery(t.breakpoints.down('xs'))
const sm = M.useMediaQuery(t.breakpoints.down('sm'))
Expand Down Expand Up @@ -172,35 +189,45 @@ const usePackageStyles = M.makeStyles((t) => ({
},
}))

function Package({ name, modified, revisions, bucket, views }) {
type PackageProps = NonNullable<
ResultOf<typeof PACKAGE_LIST_QUERY>['packages']
>['page'][number]

function Package({ name, modified, revisions, bucket, accessCounts }: PackageProps) {
const { urls } = NamedRoutes.use()
const classes = usePackageStyles()
const t = M.useTheme()
const xs = M.useMediaQuery(t.breakpoints.down('xs'))
return (
<M.Paper className={classes.root}>
<div className={classes.handleContainer}>
<Link className={classes.handle} to={urls.bucketPackageDetail(bucket, name)}>
<RRDom.Link
className={classes.handle}
to={urls.bucketPackageDetail(bucket, name)}
>
<span className={classes.handleClickArea} />
<span className={classes.handleText}>{name}</span>
</Link>
</RRDom.Link>
</div>
<M.Box pl={2} pb={2} pt={1}>
<span className={classes.revisions}>
{revisions}{' '}
{revisions.total}{' '}
{xs ? (
'Rev.'
) : (
<Format.Plural value={revisions} one="Revision" other="Revisions" />
<Format.Plural value={revisions.total} one="Revision" other="Revisions" />
)}
</span>
<M.Box mr={2} component="span" />
<span className={classes.updated} title={modified ? modified.toString() : null}>
<span
className={classes.updated}
title={modified ? modified.toString() : undefined}
>
{xs ? 'Upd. ' : 'Updated '}
{modified ? <Format.Relative value={modified} /> : '[unknown: see console]'}
</span>
</M.Box>
{!!views && <Counts {...views} />}
{!!accessCounts && <Counts {...accessCounts} />}
</M.Paper>
)
}
Expand All @@ -227,7 +254,13 @@ const useSortDropdownStyles = M.makeStyles((t) => ({
},
}))

function SortDropdown({ value, options, makeSortUrl }) {
interface SortDropdownProps {
value: SortMode
options: typeof SORT_OPTIONS
makeSortUrl: (mode: SortMode) => string
}

function SortDropdown({ value, options, makeSortUrl }: SortDropdownProps) {
const t = M.useTheme()
const xs = M.useMediaQuery(t.breakpoints.down('xs'))
const classes = useSortDropdownStyles()
Expand Down Expand Up @@ -271,7 +304,7 @@ function SortDropdown({ value, options, makeSortUrl }) {
{options.map((o) => (
<M.MenuItem
onClick={() => handleClick(o.key)}
component={Link}
component={RRDom.Link}
to={makeSortUrl(o.key)}
key={o.key}
selected={o.key === value}
Expand Down Expand Up @@ -317,49 +350,48 @@ export default function PackageList({
params: { bucket },
},
location,
}) {
const history = useHistory()
const s3 = AWS.S3.use()
const req = APIConnector.use()
// const sign = AWS.Signer.useS3Signer()
// const { analyticsBucket, apiGatewayEndpoint: endpoint } = Config.useConfig()
const { analyticsBucket } = Config.useConfig()
}: RRDom.RouteComponentProps<{ bucket: string }>) {
const history = RRDom.useHistory()
const { urls } = NamedRoutes.use()
// const bucketCfg = BucketConfig.useCurrentBucketConfig()
const classes = useStyles()

const scrollRef = React.useRef(null)
const scrollRef = React.useRef<HTMLDivElement | null>(null)

const { sort, filter, p } = parseSearch(location.search)
const page = p && parseInt(p, 10)
const { sort, filter, p } = parseSearch(location.search, true)
const page = p ? parseInt(p, 10) : undefined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see multiple undefined in this module. Can we use null here? I think that it's more straightforward to use null, and leave undefined only when it's really undefined

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use null here?

here -- yes, but not everywhere (e.g. not as a title attribute for a span, since it only accepts string | undefined)

I think that it's more straightforward to use null, and leave undefined only when it's really undefined

tbh i don't feel much difference in most cases 🤷 (with a notable exception of JSON.strignify)

const computedPage = page || 1
const computedSort = getSort(sort)
const computedFilter = filter || ''
const filtering = useDebouncedInput(computedFilter, 500)
const today = React.useMemo(() => new Date(), [])

const [counter, setCounter] = React.useState(0)
const totalCountQuery = useQuery({
query: PACKAGE_COUNT_QUERY,
variables: { bucket, filter: null },
requestPolicy: 'cache-and-network',
})

const totalCountData = Data.use(requests.countPackages, { req, bucket, counter })
const filteredCountData = Data.use(requests.countPackages, {
req,
bucket,
filter: computedFilter,
counter,
const filteredCountQuery = useQuery({
query: PACKAGE_COUNT_QUERY,
variables: { bucket, filter: filter || null },
requestPolicy: 'cache-and-network',
})
const packagesData = Data.use(requests.listPackages, {
s3,
req,
analyticsBucket,
bucket,
filter,
sort: computedSort.key,
perPage: PER_PAGE,
page,
today,
counter,

const packagesQuery = useQuery({
query: PACKAGE_LIST_QUERY,
variables: {
bucket,
filter: filter || null,
order: computedSort.value,
page: computedPage,
perPage: PER_PAGE,
},
requestPolicy: 'cache-and-network',
})

const refreshData = React.useCallback(() => {
packagesQuery.run({ requestPolicy: 'cache-and-network' })
}, [packagesQuery])

const makeSortUrl = React.useCallback(
(s) =>
urls.bucketPackageList(bucket, {
Expand Down Expand Up @@ -407,11 +439,9 @@ export default function PackageList({

const onExited = React.useCallback(
(res) => {
if (res && res.pushed) {
setCounter(R.inc)
}
if (res?.pushed) refreshData()
},
[setCounter],
[refreshData],
)

const preferences = BucketPreferences.use()
Expand All @@ -424,8 +454,8 @@ export default function PackageList({

{createDialog.element}

{totalCountData.case({
_: () => (
{totalCountQuery.case({
fetching: () => (
<M.Box pb={{ xs: 0, sm: 5 }} mx={{ xs: -2, sm: 0 }}>
<M.Box mt={{ xs: 0, sm: 3 }} display="flex" justifyContent="space-between">
<M.Box
Expand Down Expand Up @@ -459,8 +489,9 @@ export default function PackageList({
))}
</M.Box>
),
Err: displayError(),
Ok: (totalCount) => {
error: displayError(),
data: (totalCountData) => {
const totalCount = totalCountData.packages?.total
if (!totalCount) {
return (
<M.Box pt={5} textAlign="center">
Expand Down Expand Up @@ -542,10 +573,11 @@ export default function PackageList({
</M.Box>
</M.Box>

{filteredCountData.case({
_: () => R.range(0, 10).map((i) => <PackageSkel key={i} />),
Err: displayError(),
Ok: (filteredCount) => {
{filteredCountQuery.case({
fetching: () => R.range(0, 10).map((i) => <PackageSkel key={i} />),
error: displayError(),
data: (filteredCountData) => {
const filteredCount = filteredCountData.packages?.total
if (!filteredCount) {
return (
<M.Box
Expand All @@ -564,21 +596,22 @@ export default function PackageList({
const pages = Math.ceil(filteredCount / PER_PAGE)

if (computedPage > pages) {
return <Redirect to={makePageUrl(pages)} />
return <RRDom.Redirect to={makePageUrl(pages)} />
}

return (
<>
{packagesData.case({
_: () => {
{packagesQuery.case({
fetching: () => {
const items =
computedPage < pages ? PER_PAGE : filteredCount % PER_PAGE
return R.range(0, items).map((i) => <PackageSkel key={i} />)
},
Err: displayError(),
Ok: R.map((pkg) => (
<Package key={pkg.name} {...pkg} bucket={bucket} />
)),
error: displayError(),
data: (packagesData) =>
(packagesData.packages?.page || []).map((pkg) => (
<Package key={pkg.name} {...pkg} />
)),
})}
{pages > 1 && (
<Pagination {...{ pages, page: computedPage, makePageUrl }} />
Expand Down
Loading