Ability for Logger to ship logs to CloudWatch Logs

[ferdingler]

Is your feature request related to a problem? Please describe.
As a developer, I would like for Amplify to support log shipping to CloudWatch logs via the use of Logger. So I can send certain logs (like errors or stack traces) to my AWS account for future analysis.

Describe the solution you'd like
This feature would leverage the Cognito integration such that log shipping is done in a secure fashion with temporary credentials. I would like for the Logger to take care of sending logs in batches in the background and support a local cache for lost connectivity.

Describe alternatives you've considered
I have considered using existing npm packages like https://www.npmjs.com/package/winston-cloudwatch.

[blairtaylor]

We would find this very useful as well.

[jkeys-ecg-nmsu]

I asked an AWS trainer about this, and he said it would probably be more flexible to ship logs to S3 so they can be more easily loaded into various other services (Athena, Redshift, EMR).

That being said, I am +1 for shipping application logs to one service or the other as an option for the Logger class.

FWIW I tried prototyping it myself just by extending Logger (don't own the code so I can't currently share it), and the ContinuationToken requirement to ship logs in a specific order combined with the fact that Logger is usually not a global object but is typically instantiated per-file for the sake of making the application logs easier to read, and it's not a trivial piece of code to write.

[dbhagen]

The route I'm taking on this is to add a lambda function that the logger can send content to do that will perform the CloudWatch logging for me.

amplify function add

And then I'm looking at https://www.npmjs.com/package/lambda-log to handle log shipping for me.
If I limit access to the function to only authed endpoints, I should protect myself from spam.

I'll report back when I have it working. Obviously, it would be great if the built-in Logger could have an option for a mirroring destination. I'm patching that into the Amplify-JS library I'm running in my app. If that turns well, I'll see if I can get it in as a PR.

[ferdingler]

The route I'm taking on this is to add a lambda function that the logger can send content to do that will perform the CloudWatch logging for me.

amplify function add

And then I'm looking at https://www.npmjs.com/package/lambda-log to handle log shipping for me.
If I limit access to the function to only authed endpoints, I should protect myself from spam.

I'll report back when I have it working. Obviously, it would be great if the built-in Logger could have an option for a mirroring destination. I'm patching that into the Amplify-JS library I'm running in my app. If that turns well, I'll see if I can get it in as a PR.

Sounds good! @dbhagen , a couple of comments though:

Be aware of the Lambda concurrency limits; Log shipping may be an operation that happens fairly frequently and depending how many clients (frontend apps) you have, you may be hitting your lambda function a lot: https://docs.aws.amazon.com/lambda/latest/dg/limits.html .

I also believe that the biggest challenge in doing Log shipping is not so much on the backend, but in the client library to do error handling, buffering to send logs in batches, local cache to deal with lost of connectivity, retry backoffs, etc. In your approach, you still need to handle all of that in the client.

[Giddyrino]

it boggles my mind there isn't some sort of built in support to access generated application logs in the amplify console.

it's like they didn't expect users would want to look at the logs if their deployment failed.

[karrettgelley]

Any update on this? My current workaround is to create an error model on my schema and create records whenever I hit a catch block. For instance, I added

type Error @model {
  error: String
  lineNumber: Int
  fileName: String

Then when I hit a catch block, I hit up my appsync api:

try {

} catch(e) {
  API.graphql(graphqlOperation(createError, {
    input: {error: e, lineNumber: 5, fileName: 'App.tsx'}
  }));
}

But it's hacky and error prone.

[blairtaylor]

@karrettgelley We are doing something very similar. It seems like a step backwards compared to other logging frameworks.

[CharanRoot]

do we have any update this feature ?

[julioxavierr]

+1

This is a must have

[iCodeForBananas]

We were also wanting to use Logger as a way to report on crash metrics in our React app.

Right now we have implemented a class resembling winston-cloudwatch that uses AWS.CloudWatchLogs.PutLogEvents with some custom code to manage batching messages and sequence tokens. 

I'd love to see our analytics and logging come from Amplify so that everything is in one place.

[simone1999]

Hi,
I have the same issue.
But can't amplify analytics be used for this?
https://docs.amplify.aws/lib/analytics/getting-started/q/platform/js

[hp4k1h5]

i followed the Logger guide and am developing an application locally. i don't know where/if my logs are written to. the guide does not indicate where these logs get written to, nor how to configure output location.

for reference, i am only importing the aws-amplify package and am not using the cli. does anyone know how to configure this even for local testing, or where these logs are getting written?

[xavierraffin]

@hp4k1h5 the logs are just console.log visible in Chrome inspector for example.
This is why this ticket ask to be able to set a Amazon Cloudwatch destination.

[hp4k1h5]

@xavierraffin thanks for this information, but i don't see any logs in the dev console at all. im not getting any errors either. so i believe i have instantiated the Logger correctly. all of my console.logs are there and the app is working as expected and not throwing any errors of any kind.

im also wondering what the point of having a wrapper around console is, if Logger just dumps messages to the dev console? i agree with the ask to have Logger ship to CloudWatch since that is what i would ultimately like as well.

[SourceCode]

The libraries winston and winston-cloudwatch may be useful for the common challenges around logging at scale.

https://github.com/winstonjs/winston

https://github.com/lazywithclass/winston-cloudwatch

[iCodeForBananas]

The libraries winston and winston-cloudwatch may be useful for the common challenges around logging at scale.

https://github.com/winstonjs/winston

https://github.com/lazywithclass/winston-cloudwatch

You shouldn't need to use a 3rd party library, the AWS CloudWatchLogs SDK has all the methods you need to add logging.

describeLogStreams -> get the sequence token
putEventLogs -> to push your event

[eddiekeller]

Circling back to this issue - I've begun investigating this and want to clarify and narrow down the use cases here. Are you looking to primarily send up logs to CloudWatch from the "user" level, or are you also wanting to send the background Amplify logging that is happening?

[hp4k1h5]

@eddiekeller ; i've moved on at least temporarlily from this issue.

at the time, i was using amplify as an import to a Vue application, and wanted a default way to write error level log information (re login/account issues) to cloudwatch.

i never figured out if i was not integrated enough or not finding the right api's.
i was using amplify in addition to SAM (serverless application model) . Without writing custom wrappers around the Log functions, i thought it was not possible to make a logger for our app's logging portal using aws amplify services.

our use case seemed clear but for whichever combo of reasons, i and others are left feeling like we have wasted investment in potentially useful development. i.e. >> i have a vue login portal and the Logger function integrates with the same aws token to deliver login success/failure metrics.

[iCodeForBananas]

@eddiekeller
Our use case is primarily for error monitoring either in the client or from an API response. Detecting issues in real time or before customers report them to us.

[eddiekeller]

Thanks for the responses. Small follow up to help me clarify this a bit more. It totally makes sense that you want to be able to send up errors (and possibly other logs) from the client side and from your API responses. My next thought is - what about the error logging that we have in Amplify itself? For instance, this logger.error() in our Auth.ts file - https://github.com/aws-amplify/amplify-js/blob/main/packages/auth/src/Auth.ts#L1367 . When there is an issue with userpool not being configured properly, our current logger just writes it to the console. Is the ask for these "background" Amplify logs to also get sent to your Cloudwatch? Or only logs that you specify in your code? Just want to make sure that we tackle this properly for everyone.

[iCodeForBananas]

If there are automatic includes like the error mentioned I feel like that would be a win and our teams would gladly accept those "background" Amplify logs as well.

[eddiekeller]

Ok, got it. So it sounds like the background ones would be useful but, in the immediate sense, are more like a "nice to have" and the core ask right now is in your own usage of the logger. Does that sound correct?

For the initial implementation at least, I'm thinking it might be best to restrict it to that since having it be automatic for the background Amplify logging will require a lot more work to run through everything...plus it would require some extra thought regarding cloudwatch pricing and log storage.

Like I said, I'm investigating this now and having some discussions with the team on this issue. Don't hesitate to comment with additional info.

[iCodeForBananas]

That sounds spot on!

We found immediate benefit in simple one off logging, but I could see value in automatic background logs. We just don't have that implemented so I don't have stats to back that up.

[eddiekeller]

I have out a PR for it now that I believe should address the asks here. #8309

[eddiekeller]

PR has been merged in. New ticket tracking future improvements has been made here #8354

[chemzo]

@eddiekeller as the PR has been reverted, should this issue be re-opened?

[alexanderMontague]

Not sure if anyone here can help, but this functionality was merged back in and I am attempting to use it. It seems to fail on the initial log post though details here - #8309 (comment)

[Milan-Shah]

@eddiekeller Seems like some of our customers are still having an issue, can you please take a look?

[eddiekeller]

@alexanderMontague Thanks for bringing that issue to my attention. It looks like another commenter found a temporary way around the bug. Did you try that? In the meantime, I'll work on throwing together a fix. Should be relatively simple based on what the other commenter found.

[johnsonfamily1234]

Hey all - I have this up and working in us-east-1 and it works fine. Now trying to implement in ap-southeast-2 with a very similar configuration I'm getting a "AWSCloudWatch - error getting log group - Error: Region is missing" error. I've tried everything I can think of but nothing seems to work. Anyone have any hypotheses I might explore? Thanks!

[github-actions]

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels or Discussions for those types of questions.