[Search] Regenerate with 2024-03-01-Preview spec #28576

dgetu · 2024-02-14T17:27:25Z

Packages impacted by this PR

@azure/search-documents

Issues associated with this PR

Resolves #28299, #28300

Describe the problem that is addressed by this PR

It might be helpful to review this commit-by-commit through 41c3338

What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?

Are there test cases added in this PR? (If not, why?)

Provide a list of related PRs (if any)

Command used to generate this PR:***(Applicable only to SDK release request PRs)*

Checklists

Added impacted package name to the issue description
Added a changelog (if necessary)

See #28772 for more info on the last commit and title

maorleger

Unfortunately wasn't able to complete reviewing, but there's a lot I like about it 😄

There are some concerns I have regarding the types and breaking changes - happy to discuss!

sdk/search/search-documents/.eslintrc.json

maorleger · 2024-02-20T20:30:36Z

sdk/search/search-documents/.vscode/settings.json

@@ -1,3 +1,3 @@
 {
-  "cSpell.words": ["hnsw", "openai"]
+  "cSpell.words": ["hnsw", "openai", "Vectorizer"]


Nit: can we move this to https://github.com/maorleger/azure-sdk-for-js/blob/main/.vscode/cspell.json and keep all the cspell in the same place?

Sure, these actually belong in .vscode/cspell.json, so I'll move those over.

sdk/search/search-documents/CHANGELOG.md

sdk/search/search-documents/README.md

maorleger · 2024-02-20T20:34:16Z

sdk/search/search-documents/review/search-documents.api.md

@@ -27,12 +28,12 @@ export interface AnalyzedTokenInfo {

 // @public
 export interface AnalyzeRequest {
-    analyzerName?: string;
-    charFilters?: string[];
+    analyzerName?: LexicalAnalyzerName;


Ignore me if these are all aliases to string, but if they are unions or some other narrower type this would be a breaking change?

I won't comment on every change here yet but we should not have breaking changes in a minor release so please do validate that 👍

Some of these changes are just named string aliases, some of them are string literal unions where they're output-only parameters. But only purportedly. I mentioned this elsewhere, but it's probably not worth the maintenance burden to manually track down which extensible enums are output-only, and I'd have to think about whether or not it's worth making a Swagger parser to automate this.

maorleger · 2024-02-20T20:35:24Z

sdk/search/search-documents/review/search-documents.api.md

@@ -232,77 +237,67 @@ export interface BaseVectorSearchVectorizer {
 }

 // @public
-export type BlobIndexerDataToExtract = string;
+export type BlobIndexerDataToExtract = "storageMetadata" | "allMetadata" | "contentAndMetadata";


Like here, if these are input params then this would be a breaking change. If they are output only then I think it's fine

I intended for only output parameters to have custom string literal unions, but I think I might be creating too much of a maintenance burden here. Without proper tooling to help determine what types are output-only, and to catch when an output only type is consumed in a later revision, it's probably too much effort to maintain this convenience.

We're hoping to iron out our extensible enum plans over the coming weeks. Until then, I'm thinking I'll revert these where I can. But there's a not insignificant chance that I've already introduced a string literal union where it doesn't belong in the last GA release. Lots of bugs to squash there.

I've added a script to generate diffs against the upstream stable and main (preview) branches for the API Extractor output. Any unified diff GUI/TUI should be able to render those files (under the review dir), but any hunks that don't modify/remove any lines are omitted since they're ostensibly just new features. There are some spurious errors in there though (the difftool removes a line in one hunk and adds an identical line in another).

In the same commit, there's a script to generate string literal unions from TS files with enum declarations. Again, spurious errors, but nothing too difficult to manually check/correct.

Hopefully, these changes will make it a bit easier to assess where the API surface diverges and speeds up feature development moving forward. I'll need to make a runbook though; this is getting a bit too complex not to have documentation.

maorleger · 2024-02-20T20:36:45Z

sdk/search/search-documents/samples-dev/bufferedSenderAutoFlushTimer.ts

@@ -76,7 +76,7 @@ export async function main() {
      console.log(response);
    });

-    bufferedClient.uploadDocuments([
+    await bufferedClient.uploadDocuments([


Thanks, i'm starting to like the idea of linting our samples-dev files as well

azure-sdk · 2024-03-09T04:47:06Z

API change check

APIView has identified API level changes in this PR and created following API reviews.

@azure/search-documents

maorleger · 2024-03-12T20:07:12Z

sdk/search/search-documents/review/search-documents.api.md

@@ -44,31 +45,10 @@ export interface AnalyzeResult {
 export type AnalyzeTextOptions = OperationOptions & AnalyzeRequest;

 // @public
-export interface AnswerResult {


Did these get moved or no longer needed? I guess why are they missing and is that expected?

They were renamed to QueryAnswer and QueryCaption.

maorleger · 2024-03-12T20:11:09Z

sdk/search/search-documents/review/search-documents.api.md

@@ -2189,6 +2294,7 @@ export class SearchClient<TModel extends object> implements IndexDocumentsClient
    readonly indexName: string;
    mergeDocuments(documents: TModel[], options?: MergeDocumentsOptions): Promise<IndexDocumentsResult>;
    mergeOrUploadDocuments(documents: TModel[], options?: MergeOrUploadDocumentsOptions): Promise<IndexDocumentsResult>;
+    readonly pipeline: Pipeline;


did we mean to expose the pipeline here?

Yes. The service is moving fast, so we're exposing the pipeline to allow customers to send raw requests with our HTTP pipeline. It's called out in the changelog.

maorleger · 2024-03-12T20:12:37Z

sdk/search/search-documents/scripts/generateTestVarNames.pl

@@ -0,0 +1,29 @@
+#!/usr/bin/env -S perl -i
+
+use Algorithm::Combinatorics qw(variations_with_repetition);


what is this for, and why not use JavaScript? 😈

It's a small script to save me some time while hand-editing the typeDefinitions.ts test files. It just renames every const variable declaration with sequential strings. There's more that I intended to automate in that space, but I think I either lost some of that progress in a Codespace rebuild or otherwise shelved that for other priorities.

As for why it's Perl, I find that text file manipulation is a bit more idiomatic. perl -i modifies files in-place, so it's common to find Perl one-liners in the form perl -i -pe 's/foo/bar/msg' foo.txt (replaces all occurrences of foo with bar).

FWIW we used to have a small .js build script for EventGrid that did some massaging of types. I think in general it'd be good to stick to JS/TS when possible so you can hand it off to other maintainers more easily.

…ove unused internal exports

github-actions bot added the Search label Feb 14, 2024

dgetu linked an issue Feb 16, 2024 that may be closed by this pull request

Investigate what it would take to expose pipeline (DPG equivalent) in the search SDK #28300

Closed

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch 7 times, most recently from fe1e6de to 658ac19 Compare February 17, 2024 20:48

dgetu marked this pull request as ready for review February 18, 2024 03:11

dgetu requested review from bterlson, xirzec, jeremymeng and a team as code owners February 18, 2024 03:11

maorleger reviewed Feb 20, 2024

View reviewed changes

dgetu mentioned this pull request Mar 5, 2024

[Search] Regenerate with 2024-03-01-Preview spec #28772

Merged

2 tasks

dgetu requested review from mikeharder, ckairen, deyaaeldeen and mpodwysocki as code owners March 9, 2024 04:36

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from c9cc71a to 5a1f38d Compare March 9, 2024 04:52

maorleger approved these changes Mar 12, 2024

View reviewed changes

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch 2 times, most recently from d292cc6 to 0cf5c76 Compare March 14, 2024 08:27

Lint

d43b84b

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from 0cf5c76 to 0199a1d Compare March 17, 2024 00:40

dgetu requested a review from witemple-msft as a code owner March 17, 2024 00:40

dgetu added 2 commits March 17, 2024 01:39

Clear up environment variable names

b1d096b

Publish samples

cc06ea3

dgetu and others added 7 commits March 17, 2024 01:39

Bump Autorest emitter version

6de94be

Change package version to 12.1.0-beta.1

4e93716

Apply GA API surface to preview build

0e33c81

Fix tests

0ea0cda

Expose generated client pipelines for raw requests

471bf7e

Add sample for sticky query sessions

7264a96

[Search] Move local cspell to global cspell dir

0527a46

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from 0199a1d to f65827e Compare March 17, 2024 01:50

dgetu changed the title ~~[Search] Apply v12 API changes to preview~~ [Search] Regenerate with 2024-03-01-Preview spec Mar 17, 2024

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from ef34ff6 to f55a190 Compare March 17, 2024 05:55

dgetu added 2 commits March 17, 2024 06:07

Move some string literal unions to generated file

3639879

Match extensible enum and string literal union surface to stable, rem…

f27fc79

…ove unused internal exports

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from f55a190 to b45bd2a Compare March 17, 2024 06:10

Regenerate with 2024-03-01-Preview spec

b2c22d9

dgetu force-pushed the search/catch-beta-up-to-ga-12 branch from b45bd2a to b2c22d9 Compare March 17, 2024 06:39

dgetu enabled auto-merge (squash) March 17, 2024 06:54

dgetu merged commit d738efb into Azure:main Mar 17, 2024
14 checks passed

dgetu linked an issue Mar 25, 2024 that may be closed by this pull request

Add full support for search Oct preview release #28301

Closed

dgetu mentioned this pull request Mar 25, 2024

Add full support for search Oct preview release #28301

Closed

dgetu linked an issue Apr 30, 2024 that may be closed by this pull request

Add Search embedding sample for reduced embedding dimensions #29070

Closed

dgetu mentioned this pull request Apr 30, 2024

Add Search embedding sample for reduced embedding dimensions #29070

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Search] Regenerate with 2024-03-01-Preview spec #28576

[Search] Regenerate with 2024-03-01-Preview spec #28576

dgetu commented Feb 14, 2024 •

edited

Loading

maorleger left a comment

maorleger Feb 20, 2024

dgetu Feb 29, 2024

maorleger Feb 20, 2024

dgetu Mar 1, 2024

maorleger Feb 20, 2024

dgetu Mar 1, 2024

dgetu Mar 13, 2024

maorleger Feb 20, 2024

azure-sdk commented Mar 9, 2024

maorleger Mar 12, 2024

dgetu Mar 12, 2024

maorleger Mar 12, 2024

dgetu Mar 12, 2024

maorleger Mar 12, 2024

dgetu Mar 12, 2024

xirzec Mar 14, 2024

		@@ -0,0 +1,29 @@
		#!/usr/bin/env -S perl -i

		use Algorithm::Combinatorics qw(variations_with_repetition);

[Search] Regenerate with 2024-03-01-Preview spec #28576

[Search] Regenerate with 2024-03-01-Preview spec #28576

Conversation

dgetu commented Feb 14, 2024 • edited Loading

Packages impacted by this PR

Issues associated with this PR

Describe the problem that is addressed by this PR

What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?

Are there test cases added in this PR? (If not, why?)

Provide a list of related PRs (if any)

Command used to generate this PR:**(Applicable only to SDK release request PRs)

Checklists

maorleger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azure-sdk commented Mar 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgetu commented Feb 14, 2024 •

edited

Loading

Command used to generate this PR:***(Applicable only to SDK release request PRs)*