Should we remove defunct OpenAI models?

[krassowski]

Description

Some OpenAI models included are no longer supported and do not work. Should these be removed?

Reproduce

1. Select one of the models from the list and try to chat
2. See

Traceback (most recent call last):
  File "/jupyter-ai/jupyter_ai/chat_handlers/base.py", line 113, in on_message
    await self.process_message(message)
  File "/jupyter_ai/chat_handlers/default.py", line 54, in process_message
    response = await self.llm_chain.apredict(input=message.body, stop=["\nHuman:"])
  File "/lib/python3.10/site-packages/langchain/chains/llm.py", line 310, in apredict
    return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
  File "/lib/python3.10/site-packages/langchain/chains/base.py", line 413, in acall
    return await self.ainvoke(
  File "/lib/python3.10/site-packages/langchain/chains/base.py", line 209, in ainvoke
    raise e
  File "/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
    await self._acall(inputs, run_manager=run_manager)
  File "/lib/python3.10/site-packages/langchain/chains/llm.py", line 275, in _acall
    response = await self.agenerate([inputs], run_manager=run_manager)
  File "/lib/python3.10/site-packages/langchain/chains/llm.py", line 142, in agenerate
    return await self.llm.agenerate_prompt(
  File "/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 535, in agenerate_prompt
    return await self.agenerate(
  File "/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 893, in agenerate
    output = await self._agenerate_helper(
  File "/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 757, in _agenerate_helper
    raise e
  File "/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 741, in _agenerate_helper
    await self._agenerate(
  File "/lib/python3.10/site-packages/langchain_community/llms/openai.py", line 523, in _agenerate
    response = await acompletion_with_retry(
  File "/lib/python3.10/site-packages/langchain_community/llms/openai.py", line 133, in acompletion_with_retry
    return await llm.async_client.create(**kwargs)
  File "/lib/python3.10/site-packages/openai/resources/completions.py", line 1133, in create
    return await self._post(
  File "/lib/python3.10/site-packages/openai/_base_client.py", line 1536, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "/lib/python3.10/site-packages/openai/_base_client.py", line 1315, in request
    return await self._request(
  File "/lib/python3.10/site-packages/openai/_base_client.py", line 1392, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'message': 'The model `text-davinci-002` has been deprecated, learn more here: https://platform.openai.com/docs/deprecations', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}

Expected behavior

Context

On January 4th, 2024 Open Ai shut down a number of models (in bold these that I know are still used in this extension)

• text-ada-001
• text-babbage-001
• text-curie-001
• text-davinci-001
• text-davinci-002
• text-davinci-003
• davinci-instruct-beta
• curie-instruct-beta
• code-search-ada-code-001
• code-search-ada-text-001
• code-search-babbage-code-001
• code-search-babbage-text-001
• text-search-ada-doc-001
• text-search-ada-query-001
• text-search-babbage-doc-001
• text-search-babbage-query-001
• text-search-curie-doc-001
• text-search-curie-query-001
• text-search-davinci-doc-001
• text-search-davinci-query-001
• text-similarity-ada-001
• text-similarity-babbage-001
• text-similarity-curie-001
• text-similarity-davinci-001
• text-davinci-edit-001
• code-davinci-edit-001
• text-davinci-insert-001
• text-davinci-insert-002

References:

• https://openai.com/blog/gpt-4-api-general-availability
• https://platform.openai.com/docs/deprecations/deprecation-history

jupyter-ai/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py

Lines 539 to 544 in 6de2047

	models = [
	"text-davinci-003",
	"text-davinci-002",
	"text-curie-001",
	"text-babbage-001",
	"text-ada-001",

[diveshjain-phy]

I cannot see the recent gpt-4o models; can the list be updated?