Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save log once when request fully processed #55

Merged
merged 2 commits into from
Jul 10, 2017

Conversation

frankie567
Copy link
Contributor

Hi,

Ok this is maybe completely dummy from me, but couldn't we save us some writings to DB by saving APIRequestLog only once ; at the end of the request processing (either from handle_exception or finalize_response)?

Here is a proposal for this. Tests seem to pass.

Regards 🙂

François Voron added 2 commits July 4, 2017 14:10
Actually, finalize_response is always called, even when
an exception is throwed.
So, we don't need to save status_code and fire save()
in handle_exception as it will always be done in
finalize_response.
# tracking db constraints should not create a failure on the API
return
# create log
self.request.log = APIRequestLog(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, this change would only create the object in memory and not persist to the db.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, yes, that's the point. But then, the object is carried over through request processing and we do save in DB at the end.

With these changes we have only one call to the DB instead of:

I think that this kind of optimizations is not pointless, as we surely want to limit the overhead brought by the tracking.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I see the other save locations as part of the whole picture.

@avelis
Copy link
Collaborator

avelis commented Jul 7, 2017

@frankie567 First off. I want to express appreciation for finding this library and contributing to it.

Some questions:

  1. In the event the thread was terminated prematurely would we potentially lose the saved data in memory?

The try catch was added (I do not have the time to find the PR/issues on this) because the save would could fail on initialize and caused users regular API views to stop functioning. The finalize response then tried to operate on an object that didn't exist. This led to the has attribute check. This info leads too:.

  1. With your understanding of the call sequence in this mixin can we ensure that the mixin will continue to not interrupt user execution flow if a save fails?

@avelis
Copy link
Collaborator

avelis commented Jul 7, 2017

@frankie567 My questions above are a little nuanced but wanted to make sure we didn't reintroduce issues that were once closed is all.

@frankie567
Copy link
Contributor Author

@avelis Don't worry, I perfectly understand your point. I'll try to bring as more details as possible.

1. In case of thread interruption
If the thread is exited before the finalize_response method is called, well yes, we'll indeed lose the record of this request. In the current version, we would have had a record in the DB in a "dangling" state without status_code and response. In my opinion, not a big deal if we lose this kind of records.

2. Prevent mixin from ruining the API
We still do have the hasattr tests both in handle_exception (#L85) and finalize_response (#L100). So we are quite assured that we do have a log in our request before doing anything.

What could happen indeed is a problem during the final save(). To be fully safe, we could wrap the call in a try: except: pass.

However, in the current version, if something fails during the initial object creation, I'm not sure the API continues gently. As we see on line #57, the try is caught with a return statement, meaning that the method will stop here and the parent initial method not called ; which will probably break the rest of the request pipeline.
If we finally decide to not merge this PR, maybe we should make a fix on this, replacing returnby pass.

To address these two last points, we surely could make a test, mocking model create()/save() calls to raise an exception and check that the status code of the response is still 200.

That's it! I hope I answered all your questions :)

Regards!

@avelis avelis merged commit e75bd8a into aschn:master Jul 10, 2017
@avelis
Copy link
Collaborator

avelis commented Jul 10, 2017

@frankie567 Thank you for your thought out responses to my questions. I merged in the PR. I appreciate the contribution to this library! I am sure there will be a performance improvement for sure.

@avelis
Copy link
Collaborator

avelis commented Jul 10, 2017

@frankie567 To address your other shared thoughts. If you still see in an improvement to not returning prematurely and potentially moving to use a pass statement. I am all for it!

@frankie567
Copy link
Contributor Author

You're welcome! Thanks for the merge :-)

Well, now that this PR is merged, this except: return is no longer an issue.

Still, there is still the case where the log save() could ruin everything. I believe it may happen when the library is first deployed, but the migration is not yet applied: API calls in-between would fail.

If you're okay with that, I will pop another PR with a test assuming DB failure and a proposal to prevent errors when it happens.

Best regards !

@avelis
Copy link
Collaborator

avelis commented Jul 10, 2017

@frankie567 I am absolutely ok with another PR. Look forward to it.

frankie567 pushed a commit to frankie567/drf-tracking that referenced this pull request Jul 11, 2017
This is a follow-up of the discussion in PR aschn#55.

If something is going wrong during log saving, we don't want
the API to fail and throw out an error to the end user.

To do this, silently swallow any exception throwed by save().

A test is provided to check that the response status_code of
the API is still 200 even in the event of a DB failure.
avelis pushed a commit that referenced this pull request Jul 12, 2017
This is a follow-up of the discussion in PR #55.

If something is going wrong during log saving, we don't want
the API to fail and throw out an error to the end user.

To do this, silently swallow any exception throwed by save().

A test is provided to check that the response status_code of
the API is still 200 even in the event of a DB failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants