Client network socket disconnected before secure TLS connection was established to https://www.googleapis.com/oauth2/v4/token

[robwells124]

Bug Description

I am using the cloud-sql-nodejs-connector library in my NextJs project, however I get intermittent errors from the connector causing the connection to fail. This function run in the Instrumentation process and sets up the properties I need to connect to a package which contains shared services between my projects (database/ redis / e.t.c). This works in most cases however I've been getting this issue :

request to https://www.googleapis.com/oauth2/v4/token failed, reason: Client network socket disconnected before secure TLS connection was established

I'll produce the stacktrace for it below since the error is happening in the getCloudSQLInstance's method performRefresh() but i'm trying to discern why this is happening but i'm not having much luck, any ideas why I'd be getting this error?

I am using version 1.3.3 of this package

Example code (or command)

export async function register() {
   if (process.env.NEXT_RUNTIME === "nodejs") {
       const { env } = await import("@/env.mjs")
       const { AuthTypes, Connector, IpAddressTypes } = await import("@google-cloud/cloud-sql-connector")

       const { GoogleAuth } = await import("google-auth-library")

       const auth = new GoogleAuth({ scopes: ["https://www.googleapis.com/auth/sqlservice.admin"] })

       const JSONCredentials = JSON.parse(env.GCP_CREDENTIALS_STRING.replaceAll("\n", "\\n"))

       const connector = new Connector({ auth: auth.fromJSON(JSONCredentials) })

       const clientOpts = await connector.getOptions({
           instanceConnectionName: env.SQL_INSTANCE_CONNECTION_NAME,
           ipType: IpAddressTypes.PUBLIC,
           authType: AuthTypes.IAM
       })

       global.serverCoreConfig = {
           algoliaOptions: { apiKey: env.ALGOLIA_ADMIN_API_KEY, appId: env.ALGOLIA_APP_ID, indexName: env.ALGOLIA_PRIMARY_INDEX },
           gcsOptions: {
               storage: { credentials: JSONCredentials },
               bucketName: env.STORAGE_BUCKET_NAME
           },
           redisOptions: { url: env.KV_URL, keyPrefix: env.REDIS_KEY_PREFIX },
           sequelizeOptions: {
               username: env.SQL_USER,
               database: env.SQL_DATABASE,
               dialectOptions: {
                   ...clientOpts
               },
               logging: false
           },
           cloudinaryOptions: {
               api_key: env.CLOUDINARY_API_KEY,
               api_secret: env.CLOUDINARY_API_SECRET
           }
       }
   }

   if (process.env.NEXT_RUNTIME === "edge") {
   }
}

Stacktrace

[Error]: An error occurred while loading instrumentation hook: request to https://www.googleapis.com/oauth2/v4/token failed, reason: Client network socket disconnected before secure TLS connection was established
    at w._request (/var/task/.next/server/chunks/1913.js:1:37383)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async T.n (/var/task/.next/server/chunks/1913.js:10:83688)
    at async T.i (/var/task/.next/server/chunks/1913.js:10:82200)
    at async s.refreshTokenNoCache (/var/task/.next/server/chunks/1913.js:10:43391)
    at async s.getRequestMetadataAsync (/var/task/.next/server/chunks/1913.js:10:51791)
    at async s.requestAsync (/var/task/.next/server/chunks/1913.js:10:53000)
    at async I.getInstanceMetadata (/var/task/.next/server/chunks/1380.js:8:39661)
    at async m.performRefresh (/var/task/.next/server/chunks/1380.js:8:37555)
    at async m.getCloudSQLInstance (/var/task/.next/server/chunks/1380.js:8:35796) {
  config: {
    method: 'POST',
    url: 'https://www.googleapis.com/oauth2/v4/token',
    data: {
      grant_type: '<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.',
      assertion: '<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.'
    },
    headers: {
      'Content-Type': 'application/x-www-form-urlencoded',
      'User-Agent': 'google-api-nodejs-client/9.11.0',
      'x-goog-api-client': 'gl-node/18.20.4',
      Accept: 'application/json'
    },
    responseType: 'json',
    retryConfig: {
      httpMethodsToRetry: [Array],
      currentRetryAttempt: 2,
      retry: 3,
      noResponseRetries: 2,
      statusCodesToRetry: [Array]
    },
    paramsSerializer: [Function: paramsSerializer],
    body: '<<REDACTED> - See `errorRedactor` option in `gaxios` for configuration>.',
    validateStatus: [Function: validateStatus],
    errorRedactor: [Function: e]
  },
  response: undefined,
  error: h [FetchError]: request to https://www.googleapis.com/oauth2/v4/token failed, reason: Client network socket disconnected before secure TLS connection was established
      at ClientRequest.<anonymous> (/var/task/.next/server/chunks/1913.js:13:62304)
      at ClientRequest.emit (node:events:529:35)
      at TLSSocket.socketErrorListener (node:_http_client:501:9)
      at TLSSocket.emit (node:events:517:28)
      at emitErrorNT (node:internal/streams/destroy:151:8)
      at emitErrorCloseNT (node:internal/streams/destroy:116:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
    type: 'system',
    errno: 'ECONNRESET',
    code: 'ECONNRESET'
  },
  code: 'ECONNRESET',
  [Symbol(gaxios-gaxios-error)]: '6.3.0'
}

Thanks for any help!

[jackwotherspoon]

Hi @robwells124, thanks for raising an issue on the Cloud SQL Node Connector 😄

It looks like there is an authentication issue with the GoogleAuth not being able to fetch an access token.

Do you happen to have a Proxy you are attempting to connect through or certain firewall rules that may be blocking the token from GoogleAuth to be fetched?

There are a couple similar issues floating around out there that may be worth looking at:

• request to https://www.googleapis.com/oauth2/v4/token failed, reason: read ECONNRESET microsoft/google-play-vsts-extension#176
• Error retrieving access token: Error: read ECONNRESET google/clasp#203

Let me know if either of the links provide value, if not I'll happily try and reproduce the issue and find the solution for you 😄

[hessjcg]

This appears to be an intermittent problem with google oauth2 rest api. I'm going to close this issue here. If this reoccurs, please feel free to open the issue again.