Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Pending service change] [FormRecognizer] Tests: enable ignored tests once service-side issues are fixed #12319

Closed
4 of 6 tasks
kinelski opened this issue May 27, 2020 · 4 comments · Fixed by #38339
Closed
4 of 6 tasks

[Pending service change] [FormRecognizer] Tests: enable ignored tests once service-side issues are fixed #12319

kinelski opened this issue May 27, 2020 · 4 comments · Fixed by #38339
Assignees
@kinelski
Labels
Client This issue points to a problem in the data-plane of the library. Cognitive - Form Recognizer Cognitive Services test-reliability Issue that causes tests to be unreliable
Milestone
Backlog

Comments

@kinelski
Copy link
Member

kinelski commented May 27, 2020 •
edited
Loading

Service-side issues are preventing us from enabling some of the Form Recognizer tests. This work item keeps track of those so we can re-enable them once the corresponding issues are fixed.

Service issues:
1. Service returns "Unsupported media type" error when multipage form is sent via GitHub URI.
2. (SOLVED) Custom form recognition without labels does not return information about blank page.
3. (SOLVED) Custom form recognition with labels returns wrong error when an invalid URL is passed to it (expected: 3003, actual: 3014).
4. (SOLVED) Service returns 500 response when passing non-existing model ID to StartCopyModel.
5. (FIXED ON SDK SIDE) Service can return text angle between 180 and 360 in custom form recognition without labels.

Tests to enable:

  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageForm (true) (blocked by 5).
  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageForm (false) (blocked by 1, 5).
  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageFormWithBlankPage (true) (blocked by 2).
  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageFormWithBlankPage (false) (blocked by 1, 2).
  • StartRecognizeCustomFormsFromUriThrowsForNonExistingContent (true) (blocked by 3).
  • CopyModelError; disabled per service team's request (blocked by 4).
@kinelski kinelski added Test Debt Cognitive Services Client This issue points to a problem in the data-plane of the library. labels May 27, 2020
@kinelski kinelski added this to the Backlog milestone May 27, 2020
@kinelski kinelski mentioned this issue May 27, 2020
Merged
@kinelski kinelski changed the title [FormRecognizer] Tests: enabled ignored tests once service-side issues are fixed [FormRecognizer] Tests: enable ignored tests once service-side issues are fixed May 27, 2020
@kinelski kinelski mentioned this issue May 27, 2020
Closed
@AlexGhiondea AlexGhiondea modified the milestones: Backlog, [2020] July Jun 3, 2020
@annelo-msft annelo-msft changed the title [FormRecognizer] Tests: enable ignored tests once service-side issues are fixed [Pending service change] [FormRecognizer] Tests: enable ignored tests once service-side issues are fixed Jun 4, 2020
This was referenced Jun 9, 2020
Merged
Merged
@maririos maririos mentioned this issue Jun 17, 2020
Merged
@kinelski kinelski mentioned this issue Jun 22, 2020
Merged
This was referenced Jun 29, 2020
Closed
Merged
@kinelski kinelski mentioned this issue Aug 17, 2020
Merged
@maririos maririos modified the milestones: [2020] November, Backlog Oct 8, 2020
@maririos
Copy link
Member

maririos commented Dec 4, 2020

Validate and follow up again during MQ

@maririos maririos modified the milestones: Backlog, MQ-2020 Dec 4, 2020
@maririos maririos removed this from the MQ-2020 milestone Dec 29, 2020
@maririos
Copy link
Member

Still not fixed. Will ping the service about it.
Removing MQ milestone

@jsquire jsquire added test-reliability Issue that causes tests to be unreliable and removed Test Debt labels Jul 16, 2021
@kinelski kinelski added this to the [2022] June milestone May 12, 2022
@jsquire jsquire modified the milestones: [2022] June, [2022] July Jun 14, 2022
@kinelski kinelski modified the milestones: [2022] July, Backlog Jun 21, 2022
@kinelski kinelski modified the milestones: Backlog, 2022-09 Aug 11, 2022
@kinelski kinelski modified the milestones: 2022-09, Backlog Sep 14, 2022
@pallavit pallavit closed this as not planned Won't fix, can't repro, duplicate, stale Jul 10, 2023
@kinelski
Copy link
Member Author

Reopening this as we still need to re-enable these tests.

@kinelski kinelski reopened this Jul 10, 2023
@kinelski
Copy link
Member Author

kinelski commented Aug 22, 2023 •
edited
Loading

It's been 3 years since this issue has been opened and the bug was never fixed on the service side. For this reason, we'll update the tests on our side to reflect the current service behavior:

  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageForm will expect an exception.
  • StartRecognizeCustomFormsWithoutLabelsCanParseMultipageFormWithBlankPage will be deleted since it would be just validating the same exception.

kinelski added a commit to kinelski/azure-sdk-for-net that referenced this issue Aug 23, 2023
@kinelski
Fixing Azure#12319
4f031fa
@kinelski kinelski mentioned this issue Aug 23, 2023
Merged
@github-actions github-actions bot locked and limited conversation to collaborators Nov 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kinelski kinelski

Labels
Client This issue points to a problem in the data-plane of the library. Cognitive - Form Recognizer Cognitive Services test-reliability Issue that causes tests to be unreliable
Projects
None yet
Milestone
Backlog
Development

Successfully merging a pull request may close this issue.

[FormRecognizer] Fixing a couple of minor issues kinelski/azure-sdk-for-net
5 participants
@jsquire @kinelski @AlexGhiondea @maririos @pallavit