-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v0.4.0: CDK 移行 #62
v0.4.0: CDK 移行 #62
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
クリティカルなところの動作確認/コメントを終えました。
デプロイオプションのチェックが終わってないので、mergeはもうしばしお待ち下さい。
}) | ||
); | ||
|
||
const dataSource = new kendra.CfnDataSource(this, 'S3DataSource', { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
元の webclawler connector のまま & index作成時に同期しておいてもらえると嬉しいです。
jp-rag-sample/kendra/kendra-docs-index.yaml
Lines 106 to 127 in 4b8ee29
KendraDocsDS: | |
Type: 'AWS::Kendra::DataSource' | |
Properties: | |
DataSourceConfiguration: | |
WebCrawlerConfiguration: | |
UrlInclusionPatterns: | |
- '.*https://docs.aws.amazon.com/ja_jp/lex/.*' | |
- '.*https://docs.aws.amazon.com/ja_jp/kendra/.*' | |
- '.*https://docs.aws.amazon.com/ja_jp/sagemaker/.*' | |
Urls: | |
SiteMapsConfiguration: | |
SiteMaps: | |
- 'https://docs.aws.amazon.com/ja_jp/lex/latest/dg/sitemap.xml' | |
- 'https://docs.aws.amazon.com/ja_jp/kendra/latest/dg/sitemap.xml' | |
- 'https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/sitemap.xml' | |
IndexId: !GetAtt DocsKendraIndex.Id | |
Name: 'KendraDocsDS' | |
RoleArn: !GetAtt KendraDSRole.Arn | |
Type: 'WEBCRAWLER' | |
LanguageCode: ja | |
DependsOn: | |
DocsKendraIndex |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
反映しました。変更点としては元は一回デプロイするだけだったのに対して新バージョンではデプロイするたびに onUpdate が走るため初回のみ自動 Sync を行う onCreate に変更しました。
packages/web/src/utils/service.ts
Outdated
return data; | ||
} | ||
|
||
export function getKendraQuery( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kendra の access control 対応をする場合は、QueryInput に jwt を挿入する必要あり。
await Auth.currentAuthenticatedUser()
をする場合は、関数の定義と呼び出し元で対応必要。
export async function getKendraQuery(
/**
* Kendra Query API への request Bodyを作成
*/
queryText: string,
attributeFilter: AttributeFilter,
sortingConfiguration: SortingConfiguration | undefined
): Promise<QueryCommandInput> {
return {
IndexId: indexId,
PageNumber: 1,
PageSize: 10,
QueryText: queryText,
AttributeFilter: attributeFilter,
SortingConfiguration: sortingConfiguration,
UserContext: {
Token: (await Auth.currentAuthenticatedUser()).signInUserSession.accessToken.jwtToken
}
};
}
export async function overwriteQuery(
/**
* Kendra Query API への request Bodyへフィルタリング情報を付与
*/
prevQuery: QueryCommandInput,
newAttributeFilter: AttributeFilter,
newSortingConfiguration: SortingConfiguration | undefined
): Promise<QueryCommandInput> {
return {
...prevQuery,
AttributeFilter: newAttributeFilter,
SortingConfiguration: newSortingConfiguration,
UserContext: {
Token: (await Auth.currentAuthenticatedUser()).signInUserSession.accessToken.jwtToken
}
};
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cognito user poolに group 名: "legal" をつくり、グループメンバーに対象ユーザを追加して
#24
こちらサンプルデータをkendraに読み込ませて検証可能。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
検索クエリ「GDPR」
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ありがとうございます。動作確認取れましたので反映しました。
aiAgent[currentQueryId].suggestedQuery; | ||
|
||
// 並列で複数のクエリを実行 | ||
const kendraResults = await Promise.all( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
jwt 対応に伴う変更
const kendraResults = await Promise.all(
suggestedQueries.map(async (query) => {
const kendraQueryInput = await getKendraQuery(
query,
getAttributeFilter(filterOptions),
getCurrentSortOrder(filterOptions)
);
return kendraQuery(kendraQueryInput);
})
);
// F-1. currentConversation.userQuery があるとき (top barから検索済みであること)を確認 | ||
if (currentConversation?.userQuery !== undefined) { | ||
// F-2. currentConversation.userQuery の言語設定、ソート順、フィルタを変更しクエリを再実行 | ||
const q = overwriteQuery( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
jwt 対応に伴う対応
const q = await overwriteQuery(
currentConversation.userQuery,
getAttributeFilter(filterOptions),
getCurrentSortOrder(filterOptions)
);
filterOptions[1], // ソート順序 | ||
]); | ||
// K-3. 現在設定中のfilterは見ずに、言語設定とソート順序だけを反映させてKendraへQuery | ||
const q = getKendraQuery( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
jwt 対応に伴う変更
const q = await getKendraQuery(
queryText,
getAttributeFilter(filterOptions),
getCurrentSortOrder(filterOptions)
);
| 合計 | 833 | | ||
ご利用いただく際の、構成と料金試算例が以下になります。従量課金制となっており、実際の料金はご利用内容により変動いたします。 | ||
|
||
| サービス | 項目 | 数量 | 単価 | 料金 (USD) | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
詳細な見積もりありがとうございます!!
#### (1) Filterを設定して再検索が可能 | ||
- **フィルター検索**: ドキュメントの種類、作成日時などでフィルタリングが可能 | ||
- **Incremental Learning (英語のみ)**: 検索結果の良し悪しフィードバックを次回以降に反映 | ||
- **セキュリティ**: WAF、MFA、IP制限、メールドメイン制限、SAML連携などを実装 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
最高です!ありがとうございます!
|
||
// Model Params | ||
|
||
const CLAUDE_MESSAGE_DEFAULT_PARAMS: ClaudeMessageParams = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
これはragのサンプルなので、デフォルトパラメータを
"temperature": 0,
"top_k": 1,
"max_tokens": 500
にすることはできますか?
- 出力のゆらぎを抑えたい
- top_p と tempereture を併用することは推奨されていない。
の2点が理由です。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ありがとうございます、修正しました
@@ -0,0 +1,18 @@ | |||
# 移行ガイド v0.3.0 → v0.4.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
v0.3.0 ユーザーが v0.4.0 へ移行するため、git pull
した後 or 前に amplify で 作ったものを削除する手順も書いていただくことは可能ですか?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
修正しました
google workspace, entraIDでもログインが可能なことを検証しました。 PRありがとうございます!とてもよくなりました!! マージしましょう! |
Motivation
#47
Proposed Changes