Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for upcoming invoices w/ invoice items #320

Conversation

lskillen
Copy link
Contributor

Description

Adds support for Stripe's API for previewing upcoming invoices, and also adds support for subscription-based invoice items - This was a rather tricky one to implement given the ephemeral nature of the upcoming invoices. They aren't persisted to the database and their internals differ from the expected "norm" (i.e. they don't have a stripe ID).

An example of usage (as we have it in our application):

invoice = djstripe_models.Invoice.upcoming(
    customer=customer, subscription=sub, subscription_plan=plan)

An additional plan property accessor has been added to invoices - As detailed in the code comments, this was necessary for us to provide a consistent view for the invoice history. Instead of using the plan from the invoice subscription, it is now possible to use the plan for the from available subscription invoice item, and this access is encapsulated in the property.

The maximum length for stripe fields was also raised to 255, in accordance with the recommendations in the Stripe API Upgrades documentation. The main reason though was to make room for the longer composite key IDs used for subscription-based invoice items (necessary in order to ensure these remain unique per invoice). As per the plan change above, this also helps to provide a consistent view for the invoice history.

One odd thing to note is the use of the QuerySetMock for UpcomingInvoice. The reasoning behind this was to ensure that invoice.invoiceitems() for upcoming invoices still returned a queryset-like object that supports non-mutation operations such as iteration. I made a note in the documentation not to expect it to do anything else, so it's a bit like Stripe's own docs for upcoming invoices, they are not mutable. The main downside is the added dependency, so apologies for that!

Finally, I guess that the changes for _get_or_create_from_stripe_object might be contentious, which were required to handle the slightly different structure for the invoice items within the upcoming invoice, and to support things like not re-fetching the data from Stripe and not saving the created object. Both of the latter could be useful for some additional optimisations later.

I'll be happy to discuss any part of the PR during review!

Test Output

Ran: python runtests.py --skip-utc

Welcome to the dj-stripe test suite.

Step 1: Running unit tests.

nosetests . --verbosity=1
Creating test database for alias 'default'...
................................................................................................................................................................................................................S...S.........................................
----------------------------------------------------------------------
Ran 254 tests in 5.540s

OK (SKIP=2)
Destroying test database for alias 'default'...

Step 2: Generating coverage results.

Name                                             Stmts   Miss Branch BrPart  Cover   Missing
--------------------------------------------------------------------------------------------
djstripe/context_managers.py                         8      0      0      0   100%
djstripe/contrib/rest_framework/permissions.py       9      0      0      0   100%
djstripe/contrib/rest_framework/serializers.py      11      0      0      0   100%
djstripe/contrib/rest_framework/urls.py              5      0      0      0   100%
djstripe/contrib/rest_framework/views.py            36      0      2      0   100%
djstripe/decorators.py                              19      0      4      0   100%
djstripe/event_handlers.py                          44      0     16      0   100%
djstripe/exceptions.py                               7      0      0      0   100%
djstripe/fields.py                                  76      0     18      0   100%
djstripe/forms.py                                    7      0      0      0   100%
djstripe/managers.py                                37      0      0      0   100%
djstripe/middleware.py                              36      0     18      0   100%
djstripe/mixins.py                                  26      0      2      0   100%
djstripe/models.py                                 371      0    116      0   100%
djstripe/settings.py                                43      0     12      0   100%
djstripe/signals.py                                  3      0      0      0   100%
djstripe/stripe_objects.py                         485      0     75      0   100%
djstripe/sync.py                                    29      0      4      0   100%
djstripe/templatetags/djstripe_tags.py              20      0      4      0   100%
djstripe/urls.py                                     6      0      0      0   100%
djstripe/utils.py                                   35      0     14      0   100%
djstripe/views.py                                  133      0     22      0   100%
djstripe/webhooks.py                                25      0      8      0   100%
--------------------------------------------------------------------------------------------
TOTAL                                             1471      0    315      0   100%

Step 3: Checking for pep8 errors.

pep8 errors:
----------------------------------------------------------------------
None

Tests completed successfully with no errors. Congrats!

@lskillen lskillen changed the title Add support for upcoming invoices Add support for upcoming invoices w/ invoice items Jun 26, 2016
@lskillen lskillen mentioned this pull request Jun 26, 2016
@codecov-io
Copy link

codecov-io commented Jun 26, 2016

Current coverage is 100%

Merging #320 into #162-api-updates-through_2015-07-28 will not change coverage

@@           #162-api-updates-through_2015-07-28   #320   diff @@
=================================================================
  Files                                       23     23          
  Lines                                     1394   1475    +81   
  Methods                                      0      0          
  Messages                                     0      0          
  Branches                                   156    171    +15   
=================================================================
+ Hits                                      1394   1475    +81   
  Misses                                       0      0          
  Partials                                     0      0          

Powered by Codecov. Last updated by e60d17b...ce07a63

if not self.pk:
# InvoiceItems need a saved invoice because they're associated via
# a RelatedManager. It might be better to make this pattern more
# generic with some sort-of _attach_objects_post_save_hook().
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this. Would you mind adding that post save hook? (Another PR is fine if that would cause any conflicts in the PR chain)

lskillen added a commit to cloudsmith-io/dj-stripe that referenced this pull request Jun 28, 2016
Adds a new _attach_objects_post_save_hook, for attaching Stripe objects
to an instance only after it is saved.
"""
instance = cls(**cls._stripe_object_to_record(data))
instance._attach_objects_hook(cls, data)
instance.save()

if save:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap these two lines in newlines please

@lskillen
Copy link
Contributor Author

lskillen commented Jun 28, 2016

No problem, changes made - The double-space thing is going to be interesting to fix, in general. Some brain rewiring will be required as I've been doing that for years and years since I was forced to. ;-) You're right though, it technically isn't correct by today's standards. I even had spaces in this paragraph before I went back and deleted them.

return instance

@classmethod
def _get_or_create_from_stripe_object(cls, data, field_name="id"):
def _get_or_create_from_stripe_object(cls, data, field_name="id",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can stay on one line

Copy link
Contributor Author

@lskillen lskillen Jun 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That pushes it beyond 80 characters - Is that acceptable? We usually keep ours below it in line with PEP-8 (and it is likely a common theme throughout any submits I've made), but this is your project and I'll follow whatever guidelines you lay out. Could also bring it down a line instead?

    def _get_or_create_from_stripe_object(
             cls, data, field_name="id", refetch=True, save=True):
        field = data.get(field_name)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, seen your further comments later that E501 is ignored - Noted!

@lskillen
Copy link
Contributor Author

lskillen commented Jun 28, 2016

@kavdev I'm sure you're still reviewing (thanks btw, I do appreciate it), but I've added some comments as responses to yours. These are easy to miss sometimes so just pinging you here to raise awareness. Cheers!

if not line["id"].startswith(invoice.stripe_id):
line["id"] = "{invoice_id}-{sub_id}".format(
invoice_id=invoice.stripe_id,
sub_id=line["id"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use subscription_id here

@lskillen
Copy link
Contributor Author

For future reference: Unless I've already made the change and pushed it I'll add a thumbs up to a comment just so you know I've acknowledged it. This will either mean I've made the change locally already and I am waiting to push for some reason (such as giving you a chance to make further comments) or will implement the change as soon as I can then push. Hope that somewhat helps reviews. ;)

save = False

line.setdefault("customer", invoice.customer.stripe_id)
line.setdefault("date", int(invoice.date.strftime("%s")))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to convert a datetime to a unix timestamp? If so, please use calendar.timegm(invoice.date.timetuple()) instead.

Copy link
Contributor Author

@lskillen lskillen Jun 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that's right - Looks like we can borrow from Django instead though:

from django.utils import dateformat
line.setdefault("date", dateformat.format(invoice.date, 'U'))

I looked at the implementation and it's pretty similar to your suggestion with the main difference that it uses utctimetuple if the datetime is timezone aware:

    def U(self):
        "Seconds since the Unix epoch (January 1 1970 00:00:00 GMT)"
        if isinstance(self.data, datetime.datetime) and is_aware(self.data):
            return int(calendar.timegm(self.data.utctimetuple()))
        else:
            return int(time.mktime(self.data.timetuple()))

Thoughts?

Copy link
Member

@kavdev kavdev Jun 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lskillen: Amazing find! The calendar version is used throughout dj-stripe. I'll refactor after all of these PRs are merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lskillen
Copy link
Contributor Author

lskillen commented Jun 30, 2016

@kavdev Odd, those two hyperlinks don't take me to the comment you were talking about, that's why I missed it (sorry, wasn't being purposefully ignorant!). Found it by stepping back through the comments though.

I have two concerns with making it Customer.upcoming_invoice() exclusively:

  1. It doesn't match the Stripe API for parity (I had thought this was desirable).
  2. It would force client code to use the Invoice (technically UpcomingInvoice) class - Do you think there might be situations in which people are utilising a derived class for their own purposes?

My suggestions for the above:

  1. Leave Invoice.upcoming() and have Customer.upcoming_invoice() call it.
  2. Add a target_cls parameter to Customer.upcoming_invoice() so it can be documented there, and have it default to UpcomingInvoice. Although I see what you mean about the code within stripe_objects returning Stripe objects only, so possibly could re-arrange it like that.

I'll defer to you - Thoughts?

@kavdev
Copy link
Member

kavdev commented Jun 30, 2016

@lskillen: Having the call as a shortcut in Customer is fine with me -- you got me on API parity :)

As for the target_cls parameter: If we decide to go that route, all methods that return a dj-stripe object will have to be modified for consistancy. I like the idea, but I think that should be a separate feature request. It might even be better to use settings to override the default class with a derived class. That would ensure total consistency and more clarity.

@lskillen
Copy link
Contributor Author

lskillen commented Jun 30, 2016

@kavdev OK, sounds good to me - Tried to make things more consistent with regards to the Stripe object data versus dj-stripe objects divide. In order to keep the API parity I have created a new upcoming class method on djstripe.models.Invoice which performs the desired conversion of the Stripe object data received from StripeInvoice.upcoming(). The new upcoming_invoice() instance method on djstripe.models.Customer is a now a lightweight wrapper for djstripe.models.Invoice.upcoming() but injects the current customer instance as an keyword argument. Seems to work OK!

stripe_invoice = StripeInvoice.upcoming(**kwargs)

if stripe_invoice:
return cls._create_from_stripe_object(stripe_invoice, save=False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the class check in 567-571 necessary? You could just use UpcomingInvoice on this line instead of cls. If that was put in for extendibility, I'd rather go with the class replacement settings idea from a previous conversation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it was (originally), but I'll remove this and the extensible support can come later.

@kavdev
Copy link
Member

kavdev commented Jun 30, 2016

@lskillen: Almost there!

lskillen added a commit to cloudsmith-io/dj-stripe that referenced this pull request Jun 30, 2016
@lskillen
Copy link
Contributor Author

lskillen commented Jun 30, 2016

OK, fixes added - If it's all good I'll squash the secondary commits.

:rtype: ``djstripe.models.UpcomingInvoice``
"""
try:
data = cls._api().upcoming(
Copy link
Member

@kavdev kavdev Jun 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use upcoming_stripe_invoice here as well (instead of data) and that'll be the last fix

@kavdev
Copy link
Member

kavdev commented Jun 30, 2016

@lskillen: at over 95 comments, I believe you do, in fact, hold the record haha

lskillen added 2 commits June 30, 2016 21:51
Also adds migrations (including missing migrations).
Adds a new _attach_objects_post_save_hook, for attaching Stripe objects
to an instance only after it is saved.
@lskillen lskillen force-pushed the feature-upcoming-invoices branch from 1af6d21 to ce07a63 Compare June 30, 2016 20:51
@lskillen
Copy link
Contributor Author

@kavdev: You're a harsh task master, Alex! ;-) All sorted and squashed.

@kavdev
Copy link
Member

kavdev commented Jun 30, 2016

Perfect, merging!

@kavdev kavdev merged commit cd13e5c into dj-stripe:#162-api-updates-through_2015-07-28 Jun 30, 2016
@kavdev
Copy link
Member

kavdev commented Jul 1, 2016

@lskillen: What do you think about renaming UpcomingInvoice to InvoicePreview? After reading the docs more carefully, that's really what you're getting: an immutable preview of what an Invoice would state given the optional changes you specify.

@kavdev
Copy link
Member

kavdev commented Jul 1, 2016

(don't worry - I would be making the changes)

@lskillen
Copy link
Contributor Author

lskillen commented Jul 1, 2016

@kavdev That'd be OK too (it still fits the function), although the "upcoming invoice" nomenclature appears more often than the association with preview, such as:

Retrieve an upcoming invoice
In the case of upcoming invoices, the customer of the upcoming invoice is required.
At any time, you can preview the upcoming invoice for a customer.
Note that when you are viewing an upcoming invoice, you are simply viewing a preview.
The value passed in should be the same as the subscription_proration_date returned on the upcoming invoice resource.

Part of this might be due to backwards compatibility though - I think the preview features were added over time rather than being there from the start. Might just be easier leaving it if we assume that developers are looking up the API documentation and searching for "upcoming invoice" (I know I did several times!)

@kavdev
Copy link
Member

kavdev commented Jul 1, 2016

@lskillen: This line makes things pretty clear:

Note that when you are viewing an upcoming invoice, you are simply viewing a preview.

I'll keep it as UpcomingInvoice and emphasize that point with an Important admonition in the docs.

@lskillen lskillen deleted the feature-upcoming-invoices branch July 4, 2016 21:55
kavdev added a commit that referenced this pull request Jul 9, 2016
@kavdev
Copy link
Member

kavdev commented Jul 9, 2016

@lskillen see above commit

@lskillen
Copy link
Contributor Author

LGTM! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants