Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE #2785

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions app/backfill-synonym-slugs.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import { tables } from '@architect/functions'

Check warning on line 1 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L1

Added line #L1 was not covered by tests
import type { DynamoDBDocument } from '@aws-sdk/lib-dynamodb'
import { paginateScan } from '@aws-sdk/lib-dynamodb'
import { slug } from 'github-slugger'

Check warning on line 4 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L3-L4

Added lines #L3 - L4 were not covered by tests

import type { Synonym } from '~/routes/synonyms/synonyms.lib'

function* chunks<T>(arr: T[], n: number): Generator<T[], void> {
for (let i = 0; i < arr.length; i += n) {
yield arr.slice(i, i + n)

Check warning on line 10 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L8-L10

Added lines #L8 - L10 were not covered by tests
}
}

export async function backfillSynonyms() {
console.log('Starting SYNONYM SLUG backfill...')
const db = await tables()
const client = db._doc as unknown as DynamoDBDocument
const TableName = db.name('synonyms')
const pages = paginateScan(

Check warning on line 19 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L14-L19

Added lines #L14 - L19 were not covered by tests
{ client },
{
TableName,
}
)

for await (const page of pages) {

Check warning on line 26 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L26

Added line #L26 was not covered by tests
const chunked = [...chunks(page.Items || [], 25)]
const synonymsToUpdate = [] as Synonym[]
for (const chunk of chunked) {
for (const record of chunk) {
const synonym = record as unknown as Synonym

Check warning on line 31 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L28-L31

Added lines #L28 - L31 were not covered by tests
if (!synonym.slug) {
synonymsToUpdate.push(synonym)

Check warning on line 33 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L33

Added line #L33 was not covered by tests
}
}
if (synonymsToUpdate.length > 0) {
await client.batchWrite({

Check warning on line 37 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L37

Added line #L37 was not covered by tests
RequestItems: {
[TableName]: synonymsToUpdate.map((synonym) => ({

Check warning on line 39 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L39

Added line #L39 was not covered by tests
PutRequest: {
Item: {
synonymId: synonym.synonymId,
eventId: synonym.eventId,
slug: slug(synonym.eventId),
},
},
})),
},
})
console.log(`updated ${synonymsToUpdate.length} records`)

Check warning on line 50 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L50

Added line #L50 was not covered by tests
}
}
}
console.log('... End SYNONYM SLUG backfill')

Check warning on line 54 in app/backfill-synonym-slugs.ts

View check run for this annotation

Codecov / codecov/patch

app/backfill-synonym-slugs.ts#L54

Added line #L54 was not covered by tests
}
Loading