Skip to content

Commit

Permalink
Reinvent2023 keynote1 (#4488)
Browse files Browse the repository at this point in the history 
* adding notebooks for V7 LMI (#270)

Co-authored-by: “Vivek <“[email protected]”>

* feat: jumpstart llm load test benchmarking (#277)

* feat: initial jumpstart llm benchmarking commit

* fix: throughput robustness and pricing

* feat: jumpstart benchmarking, deploy endpoint and model specs

* feat: jumpstart benchmarking generalize concurrency probe

* fix: jumpstart latency benchmarking logging changes

* fix: adjust throughput computations

* fix: variety of cleanup to jumpstart llm benchmarking

* chore: clean up notebooks

* chore: black

* chore: cleanup jumpstart inference benchmarking

* fix: adjust error logging in concurrency probe

* chore: concurrency probe finalization

* chore: clean up notebooks

* chore: add tranformers requirement install

* chore: black

* chore: grammar

* chore: pip install change

* chore: adjust load metrics for missing token statistics

* [minor updates]

* pinned version for image generation notebook (#282)

* pinned version for image generation notebook

* re-triggered test

* pinned version

* dummy commit

* code formatting fix (#287)

* Badges vivek (#292)

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

---------

Co-authored-by: “Vivek <“[email protected]”>

* fix the badge issue (#289)

Co-authored-by: Harsha Reddy <[email protected]>

---------

Co-authored-by: Vivek Gangasani <[email protected]>
Co-authored-by: “Vivek <“[email protected]”>
Co-authored-by: ulrichkr <[email protected]>
Co-authored-by: vivekmadan2 <[email protected]>
Co-authored-by: xieyongliang <[email protected]>
Co-authored-by: Harsha Reddy <[email protected]>
  • Loading branch information
@akrishna1995 @vivekgangasani
@ulrichkr @vivekmadan2 @xieyongliang @sthodimi
7 people authored Nov 27, 2023
1 parent 4abe8c5 commit a09ee0b
Show file tree
Hide file tree
Showing 20 changed files with 3,837 additions and 48 deletions.
1,182 changes: 1,182 additions & 0 deletions inference/generativeai/llm-workshop/deploy-V7-lmi/llama2_70b-lmi-trtllm.ipynb
View file

Large diffs are not rendered by default.

1,184 changes: 1,184 additions & 0 deletions inference/generativeai/llm-workshop/deploy-V7-lmi/llama2_70b_lmi_v7.ipynb
View file

Large diffs are not rendered by default.

32 changes: 16 additions & 16 deletions ...olnet-models-on-sagemaker/stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"\n",
"This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.\n",
"\n",
"![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"---"
]
Expand Down Expand Up @@ -782,35 +782,35 @@
"This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
"\n",
"\n",
"![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
"![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
"\n",
"![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n"
"![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n"
]
}
],
Expand Down
Loading

0 comments on commit a09ee0b

Please sign in to comment.