Request: Summarize request errors after exhausting all retry attempts

[RyanMarten]

Currently, we are printing the errors like this.

It would be nice to just use a set to condense this. Instead say X error occurred Y times.

The other thing that would be nice is making those error messages easier to read. Right now they are escaped json. Can we unescape the json.

See how the original warning message has nice highlighting around numbers, so it is easier to see the 429 HTTP response code.

[RyanMarten]

Let's also do this summary when we fail require_all_responses.
Let's log an error and then raise after.
Right now we are raising and including a big block of text. @kartik4949
Instead of doing a sample of errors, we can do a better summary

ValueError: Some requests failed and require_all_responses is True. Sample of the first 10 errors encountered:
 ['Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}", "API error: {'message': 'Internal Server Error', 'code': 500}"]
['Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}", "API error: {'message': 'Internal Server Error', 'code': 500}"]
['Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}", "API error: {'message': 'Internal Server Error', 'code': 500}"]
['Response is empty', "API error: {'message': 'Internal Server Error', 'code': 500}", "API error: {'message': 'Internal Server Error', 'code': 500}", "API error: {'message': 'Internal Server Error', 'code': 500}", 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}']
['Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}", 'Response is empty']
["API error: {'message': 'Internal Server Error', 'code': 500}", 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}", 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}']
['Response is empty', "API error: {'message': 'Provider returned error', 'code': 429, 'metadata': {'raw': 'error code: 1015', 'provider_name': 'Kluster'}}", "API error: {'message': 'Provider returned error', 'code': 429, 'metadata': {'raw': 'error code: 1015', 'provider_name': 'Kluster'}}", 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}"]
['Response is empty', 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}']
['Response is empty', "API error: {'message': 'Internal Server Error', 'code': 500}", 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"statusCode":429,"error":"Too Many Requests","message":"Rate limit on total number of requests per minute exceeded. Please, try again later or upgrade your account. https://platform.kluster.ai/billing"}\', \'provider_name\': \'Kluster\'}}', 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}']
['Response is empty', "API error: {'message': 'Internal Server Error', 'code': 500}", 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', 'Response is empty', 'API error: {\'message\': \'Provider returned error\', \'code\': 429, \'metadata\': {\'raw\': \'{"error":{"message":"Number of concurrent connections for the account exceeded. https://platform.kluster.ai/billing","errorCode":13000,"type":"invalid_request_error"}}\', \'provider_name\': \'Kluster\'}}', "API error: {'message': 'Internal Server Error', 'code': 500}"]

[RyanMarten]

This is what it looks like when require_all_responses fails

[alpayariyak]

+1