Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(spec): tag POC #435

Merged
merged 14 commits into from
Apr 27, 2022
Merged

feat(spec): tag POC #435

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7671b0a
feat(all): tag POC
bodinsamuel Apr 26, 2022
9dbc271
produce 2 files
bodinsamuel Apr 26, 2022
432b8bb
🤓
bodinsamuel Apr 27, 2022
d16f2db
alias for lite
bodinsamuel Apr 27, 2022
8ad4f6f
try to clean up script
bodinsamuel Apr 27, 2022
521b914
Merge branch 'main' into feat/tagging-poc
Apr 27, 2022
dd4f6e0
cache
bodinsamuel Apr 27, 2022
b8a8342
Merge branch 'main' into feat/tagging-poc
Apr 27, 2022
1cd50b7
Merge branch 'main' into feat/tagging-poc
Apr 27, 2022
4ae5f9e
review
bodinsamuel Apr 27, 2022
d2cefcc
modify text directly
bodinsamuel Apr 27, 2022
fd0ace9
use env
bodinsamuel Apr 27, 2022
6899303
Merge branch 'main' into feat/tagging-poc
Apr 27, 2022
de0c17a
Merge branch 'main' into feat/tagging-poc
Apr 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 58 additions & 26 deletions scripts/buildSpecs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,51 @@ const ALGOLIASEARCH_LITE_OPERATIONS = [
'post',
];

async function propagateTagsToOperations(
bundledPath: string,
client: string
): Promise<boolean> {
async function propagateTagsToOperations({
bodinsamuel marked this conversation as resolved.
Show resolved Hide resolved
bundledPath,
clientName,
}: {
bundledPath: string;
clientName: string;
}): Promise<void> {
if (!(await exists(bundledPath))) {
throw new Error(`Bundled file not found ${bundledPath}.`);
}

const pathToDoc = bundledPath.replace('.yml', '.doc.yml');

const bundledSpec = yaml.load(
await fsp.readFile(bundledPath, 'utf8')
) as Spec;
const bundledDocSpec = yaml.load(
await fsp.readFile(bundledPath, 'utf8')
) as Spec;
const tagsDefinitions = bundledSpec.tags;

for (const [pathKey, pathMethods] of Object.entries(bundledSpec.paths)) {
for (const [method, specMethod] of Object.entries(pathMethods)) {
// In the main bundle we need to only have the clientName before open-api generator will use this to determine the name of the client
specMethod.tags = [clientName];

for (const pathMethods of Object.values(bundledSpec.paths)) {
for (const specMethod of Object.values(pathMethods)) {
specMethod.tags = [client];
if (!bundledDocSpec.paths[pathKey][method].tags) {
continue;
}

// Checks that specified tags are well defined at root level
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I there a reason we want to ensure that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because if you don't define tags at the root level then nothing will appear in the doc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will still default to the base tag so I guess it's fine, what I mean is this feature is for the doc and shouldn't prevent dev but it's just my opinion

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once it's done it's done, if you add a tag that means you expect to change the doc. So I'm not too worried about this impacting dev flow.

for (const tag of bundledDocSpec.paths[pathKey][method].tags) {
if (tag === clientName) {
return;
}

const tagExists = tagsDefinitions
? tagsDefinitions.find((t) => t.name === tag)
: null;
if (!tagExists) {
throw new Error(
`Tag "${tag}" in "client[${clientName}] -> operation[${specMethod.operationId}]" is not defined`
);
}
}
}
}

Expand All @@ -37,8 +67,12 @@ async function propagateTagsToOperations(
noRefs: true,
})
);

return true;
await fsp.writeFile(
pathToDoc,
yaml.dump(bundledDocSpec, {
noRefs: true,
})
);
bodinsamuel marked this conversation as resolved.
Show resolved Hide resolved
}

async function lintCommon(verbose: boolean, useCache: boolean): Promise<void> {
Expand Down Expand Up @@ -84,11 +118,11 @@ async function buildLiteSpec(
outputFormat: string,
verbose: boolean
): Promise<void> {
const searchSpec = yaml.load(
const parsed = yaml.load(
await fsp.readFile(toAbsolutePath(bundledPath), 'utf8')
) as Spec;

searchSpec.paths = Object.entries(searchSpec.paths).reduce(
parsed.paths = Object.entries(parsed.paths).reduce(
(acc, [path, operations]) => {
for (const [method, operation] of Object.entries(operations)) {
if (
Expand All @@ -105,15 +139,12 @@ async function buildLiteSpec(
);

const liteBundledPath = `specs/bundled/${spec}.${outputFormat}`;
await fsp.writeFile(toAbsolutePath(liteBundledPath), yaml.dump(searchSpec));
await fsp.writeFile(toAbsolutePath(liteBundledPath), yaml.dump(parsed));

if (
!(await propagateTagsToOperations(toAbsolutePath(liteBundledPath), spec))
) {
throw new Error(
`Unable to propage tags to operations for \`${spec}\` spec.`
);
}
await propagateTagsToOperations({
bundledPath: toAbsolutePath(liteBundledPath),
clientName: spec,
});

await run(`yarn specs:fix bundled/${spec}.${outputFormat}`, {
verbose,
Expand All @@ -134,7 +165,10 @@ async function buildSpec(
createSpinner(`'${client}' spec`, verbose).start().info();

if (useCache) {
const generatedFiles = [`bundled/${client}.yml`];
const generatedFiles = [
`bundled/${client}.yml`,
`bundled/${client}.doc.yml`,
];

if (shouldBundleLiteSpec) {
generatedFiles.push(`bundled/${spec}.yml`);
Expand Down Expand Up @@ -165,12 +199,10 @@ async function buildSpec(
{ verbose }
);

if (!(await propagateTagsToOperations(toAbsolutePath(bundledPath), client))) {
spinner.fail();
throw new Error(
`Unable to propage tags to operations for \`${client}\` spec.`
);
}
await propagateTagsToOperations({
bundledPath: toAbsolutePath(bundledPath),
clientName: spec,
});

spinner.text = `linting ${client} spec`;
await run(`yarn specs:fix ${client}`, { verbose });
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/advanced/getLogs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
get:
tags:
- Advanced
operationId: getLogs
description: Return the lastest log entries.
summary: Return the lastest log entries.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/advanced/getTask.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
get:
tags:
- Indices
Comment on lines +2 to +3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to avoid any manual changes added in the specs, and default to the name of the folder the spec is stored in?

This way, we only have to handle where to put the file, not what tag it should match

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So with today's structure, the tag for getTask would be advanced, but we can also create a new folder for indices, and we default to it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope because of deleteIndex and saveObject living in the same rest path

Copy link
Member

@shortcuts shortcuts Apr 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry I did not fully explained why, but looking at the rest path, search settings objects all have the same so I guess we can do whatever we want here with the folders, wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could but it would need more care in spec.yml, experience tells me deducting something from the filesystem will blow up on your face at some point.
And deleteIndex and saveObject are not splittable because of the limitation of $ref object, so my point still stand.

Copy link
Member

@shortcuts shortcuts Apr 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are both index operations so it wouldn't choc me to see both of them under an index folder/tag.

My point is only that we don't have extra logic but I get yours too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one is index operation, the other record operation. it's definitely debatable but as a user I would find it weird :p

operationId: getTask
description: Check the current status of a given task.
summary: Check the current status of a given task.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/dictionaries/batchDictionaryEntries.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
post:
tags:
- Dictionnaries
operationId: batchDictionaryEntries
description: Send a batch of dictionary entries.
summary: Send a batch of dictionary entries.
Expand Down
58 changes: 31 additions & 27 deletions specs/search/paths/dictionaries/dictionarySettings.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,35 @@
get:
tags:
- Dictionnaries
operationId: getDictionarySettings
description: Retrieve dictionaries settings. The API stores languages whose standard entries are disabled. Fetch settings does not return false values.
summary: Retrieve dictionaries settings.
responses:
'200':
description: OK
content:
application/json:
schema:
title: getDictionarySettingsResponse
additionalProperties: false
type: object
required:
- disableStandardEntries
properties:
disableStandardEntries:
$ref: 'common/parameters.yml#/standardEntries'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

put:
tags:
- Dictionnaries
operationId: setDictionarySettings
description: Set dictionary settings.
summary: Set dictionary settings.
Expand Down Expand Up @@ -27,30 +58,3 @@ put:
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

get:
operationId: getDictionarySettings
description: Retrieve dictionaries settings.
summary: Retrieve dictionaries settings. The API stores languages whose standard entries are disabled. Fetch settings does not return false values.
responses:
'200':
description: OK
content:
application/json:
schema:
title: getDictionarySettingsResponse
additionalProperties: false
type: object
required:
- disableStandardEntries
properties:
disableStandardEntries:
$ref: 'common/parameters.yml#/standardEntries'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'
2 changes: 2 additions & 0 deletions specs/search/paths/dictionaries/getDictionaryLanguages.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
get:
tags:
- Dictionnaries
operationId: getDictionaryLanguages
description: List dictionaries supported per language.
summary: List dictionaries supported per language.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/dictionaries/searchDictionaryEntries.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
post:
tags:
- Dictionnaries
operationId: searchDictionaryEntries
description: Search the dictionary entries.
summary: Search the dictionary entries.
Expand Down
50 changes: 28 additions & 22 deletions specs/search/paths/keys/key.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,30 @@
get:
tags:
- Api Keys
operationId: getApiKey
summary: Get an API key.
description: Get the permissions of an API key.
parameters:
- $ref: 'common/parameters.yml#/KeyString'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: 'common/schemas.yml#/key'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

put:
tags:
- Api Keys
operationId: updateApiKey
summary: Update an API key.
description: Replace every permission of an existing API key.
Expand Down Expand Up @@ -36,29 +62,9 @@ put:
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

get:
operationId: getApiKey
summary: Get an API key.
description: Get the permissions of an API key.
parameters:
- $ref: 'common/parameters.yml#/KeyString'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: 'common/schemas.yml#/key'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

delete:
tags:
- Api Keys
operationId: deleteApiKey
summary: Delete an API key.
description: Delete an existing API Key.
Expand Down
56 changes: 30 additions & 26 deletions specs/search/paths/keys/keys.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,6 @@
post:
operationId: addApiKey
summary: Create a new API key.
description: Add a new API Key with specific permissions/restrictions.
requestBody:
required: true
content:
application/json:
schema:
$ref: 'common/schemas.yml#/apiKey'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: 'common/schemas.yml#/addApiKeyResponse'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

get:
tags:
- Api Keys
operationId: listApiKeys
summary: Get the full list of API Keys.
description: List API keys, along with their associated rights.
Expand Down Expand Up @@ -53,3 +29,31 @@ get:
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'

post:
tags:
- Api Keys
operationId: addApiKey
summary: Create a new API key.
description: Add a new API Key with specific permissions/restrictions.
requestBody:
required: true
content:
application/json:
schema:
$ref: 'common/schemas.yml#/apiKey'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: 'common/schemas.yml#/addApiKeyResponse'
'400':
$ref: '../../../common/responses/BadRequest.yml'
'402':
$ref: '../../../common/responses/FeatureNotEnabled.yml'
'403':
$ref: '../../../common/responses/MethodNotAllowed.yml'
'404':
$ref: '../../../common/responses/IndexNotFound.yml'
2 changes: 2 additions & 0 deletions specs/search/paths/keys/restoreApiKey.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
post:
tags:
- Api Keys
operationId: restoreApiKey
summary: Restore an API key.
description: Restore a deleted API key, along with its associated rights.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/manage_indices/listIndices.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
get:
tags:
- Indices
operationId: listIndices
summary: List existing indexes.
description: List existing indexes from an application.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/manage_indices/operationIndex.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
post:
tags:
- Indices
operationId: operationIndex
summary: Copy/move index.
description: Peforms a copy or a move operation on a index.
Expand Down
2 changes: 2 additions & 0 deletions specs/search/paths/multiclusters/batchAssignUserIds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
post:
tags:
- Clusters
operationId: batchAssignUserIds
summary: Batch assign userIDs
description: >
Expand Down
Loading