Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Accuracy] The answer doesn't come from the knowledge base sent to LLM #3082

Open
raych1 opened this issue Nov 1, 2023 · 0 comments
Open
Assignees
Labels
AI-Bot question Further information is requested

Comments

@raych1
Copy link

raych1 commented Nov 1, 2023

Bug Report

  • What is your prompt?
<Citation 1>
Source:
[5. Generate or Refresh SDK code from a TypeSpec project](https://github.com/Azure/azure-rest-api-specs/blob/main/documentation/typespec-rest-api-dev-process.md#5-generate-or-refresh-sdk-code-from-a-typespec-project)
Content:
### 5. Generate or Refresh SDK code from a TypeSpec project

The section describe the process for data-plane SDKs. Management-plane SDKs still follow separate existing `autorest` process.  
This assumes you have cloned language SDKs into your local folder at same level of `azure-rest-api-spec`.
"```
\-
azure-rest-api-specs/
azure-rest-api-specs-pr/
azure-sdk-for-java/
azure-sdk-for-js/
azure-sdk-for-net/
azure-sdk-for-python/
``````  "
You can then run `./eng/script/TypeSpec-Generate-Sdk.ps1` script to generate the necessary SDK folder and project structure if it does not already exist, and then regenerate the SDK source code.  
Scenarios:  
1. Test generation of SDK project and code with local TypeSpec changes in your `azure-rest-api-specs` repo. Please note this cannot be used to submit a SDK PR as it does not contain a valid commit id for the TypeSpec file.  
```cli
cd specifications/contoso/contoso.widgetmanager
..\..\..\eng\script\TypeSpec-Generate-Sdk.ps1 -SdkRepoRootDirectory C:\Github\fork\azure-sdk-for-java\
"```  
2. Generate/refresh SDK code for PR submission. The following command will update tsp-location.yaml with commit id and repo info so the build can be generated and verified by the CI and release pipeline.  
```"
cd specifications/contoso/contoso.widgetmanager
..\..\..\eng\script\TypeSpec-Generate-Sdk.ps1 -SdkRepoRootDirectory C:\Github\fork\azure-sdk-for-java\ [commit id] Azure/azure-rest-api-specs
"```

<Citation 2>
Source:
[Create a PR against main](https://eng.ms/docs/products/azure-developer-experience/design/api-specs-pr/api-tutorial/api-tutorial-3-publish#create-a-pr-against-main)
Content:
# Create a PR against main

> [!NOTE]
> You must have a [GitHub account linked with your Microsoft account](xref:access) and permissions to any of the
> repositories listed in this tutorial.  
When you're ready for a final review, it's time to create a pull request (PR) from your release branch you created
originally to the `main` branch on [Azure/azure-rest-api-specs](https://github.com/Azure/azure-rest-api-specs).  
* :warning: Do not merge `main` or other release branches unless you know there are changes required.
* :white_check_mark: Rebase on `main` if changes are required in a separate topic branch.

<Citation 3>
Source:
[Prepare for API review](https://eng.ms/docs/products/azure-developer-experience/design/api-review#prepare-for-api-review)
Content:
## Prepare for API review

When you have your REST API definition ready for review, [create a pull request with your Azure REST API spec update](xref:api-specs-pr).  
You should prepare and be ready to present the following information during your review:  
|Title | Importance | Brief description| Example and support documentation |
|--|--|--|--|
API spec PR link | Must Have | PR with your API spec definition against [azure/azure-rest-api-specs-pr](https://github.com/azure/azure-rest-api-specs-pr) or [azure/azure-rest-api-specs](https://github.com/azure/azure-rest-api-specs) | In order to access the PR, it may be necessary to join the [Azure GitHub organization](https://repos.opensource.microsoft.com/orgs/Azure). <p><p><ol><li>Go to https://repos.opensource.microsoft.com/orgs/Azure</li><li>Click **Join Organization**.</li></ol> |
API spec PR CI checks | Important |Make sure all CI is passing before requesting a review. | [ci-fix](https://aka.ms/ci-fix) |
Core concepts | Must have | A glossary of nouns and verbs  | [Example](https://github.com/Azure/azure-sdk-pr/blob/main/onboarding/Core_Concepts.pdf)
API reference docs | Good to have | Your API documentation consists of two parts, the API spec document (TypeSpec/Swagger), and the generated developer docs. These docs are often the first experience a developer will have with your service. Make a good first impression!!| **Recommended for Swagger**: [VS Code extension](https://marketplace.visualstudio.com/items?itemName=42Crunch.vscode-openapi) load this extension and then open your Swagger file in VS Code and select **Preview**. <p><p><ol><li>Go to your PR and look for the comment added by the `openapi-pipeline-app bot` and title `Swagger Generation Artifacts`. In the comment posted by the bot, you will find a link to your generated documentation under `ApiDocPreview`.</li><li>Expand `Detail` and look for `Click here to preview`.</li><li>Review your docs! Ensure that `Summary` and `Description` fields are accurate and complete.</li></ol>|
Hero scenarios | Good to have | Top scenarios on how service is consumed. | [Guidelines on how to identify hero scenarios](https://github.com/Azure/azure-sdk-pr/blob/main/training/azure-sdk-apis/principles/approachable/README.md#hero-scenarios)<p><p> [Examples](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/appconfiguration/Azure.Data.AppConfiguration#examples)
Review [API style guidelines](https://aka.ms/azapi/style) | Good to do | Review to learn about the Spectral linting tool that can be used inside of VS Code. Running this tool will help you identify and address many of the common issues and greatly improve your Open API (Swagger) document. | [API style guidelines](https://aka.ms/azapi/style)|

Your role is teams assistant to provide support for inquiries related to the Azure SDK domain.
Answer the following question as precise as possible using the "Citation" section above.
Do not mention **Citation** like 'in **Citation 1**' in the answer.
Do not try to make up the answer. Just say "Sorry, I do not know." if you do not know the answer.
Include useful hyperlinks from the text in the answer when possible.
Add the source of the answer at the end of the response. Use this format: `**Source**: [title](link)`, and make sure to add a new line after the answer before including the source.
Respond with "Sorry, I do not know." and do not include Source in the answer if the question cannot be answered from the context above.
When the question is about greeting, such as "hi", "hello", "who are you", etc. Respond with "Hello! I am an AI assistant to support your inquiries related to the Azure SDK domain." and do not include **Source** in the answer.


Q: The stage fails with at uthentication:
 

"devbranch:main
Syncing [throttling] specs
swaggerPath:/__w/1/s/specification
Cloning into 'Azure-azure-rest-api-specs-pr'...
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/Azure/azure-rest-api-specs-pr.git/'
Remove-Item: /__w/_temp/f1c32e25-a165-4144-957c-9e3c2f631c64.ps1:633
Line |
633 |      Remove-Item -Recurse -Force -Path $workDir
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot find path '/__w/1/s/Azure-azure-rest-api-specs-pr' because it
     | does not exist.
##[error]PowerShell exited with code '1'."

Link: https://dev.azure.com/msazure/One/_build/results?buildId=81011194&view=logs&j=8e569197-c49a-5e50-a519-c3be4c797011&t=249c85b6-de13-56d7-1bd5-0efb82990695&l=32
 
Has the PAT expired?
A:
  • What is the response of LLM?
PROMPT SUCCEEDED: duration=4.6133619 prompt=2259 completion=159 response=It looks like the authentication failed due to the removal of password authentication on August 13, 2021. It is recommended to use other modes of authentication such as SSH or personal access tokens (PAT). You can find more information about this in the [documentation](https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls).

**Source**: [Create a PR against main](https://eng.ms/docs/products/azure-developer-experience/design/api-specs-pr/api-tutorial/api-tutorial-3-publish#create-a-pr-against-main)
  • What is the link to your post in Teams Channel?
    Testing Post
@raych1 raych1 added question Further information is requested AI-Bot labels Nov 1, 2023
@raych1 raych1 self-assigned this Nov 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AI-Bot question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant