Generating OFX Files with Transaction Lists

[philliphall]

Probably a dumb question that I've reached a point to just ask...

What's the appropriate syntax for getting STMTTRN instances into BANKTRANLIST? Or is generation of statement transactions even supported?

My use case - I'm building a script that downloads transaction data via Plaid and translates them into OFX files (version 1 most likely) for compatibility purposes. I thought I'd just write my files on my own, but boy your module sure is awesome for validating fields and values to spec! I feel like I am so close to a working POC... just struggling to figure out how STMTTRN is intended to be wrapped into BANKTRANLIST. (Then I'm sure I'll run into lots of detailed compatibility things to work through!)

[csingley]

Or is generation of statement transactions even supported?

Have you seen this? There's also a detailed walkthrough of developing an entire messageset here, which includes this note about the semantics of different aggregate types:

The requirement that each *SYNCRQ / *SYNCRS may contain a variable number of transaction wrappers means that we can’t define these wrappers with SubAggregate, which maps every child element to a single class attribute.

Contained aggregates that are allowed to appear more than once are instead defined with a validator of type ListAggregate, and accessed via the Python list API. Unique children are defined in the usual manner, and accessed as instance attributes.

This is the relationship I was trying to express in the definition of BANKTRANLIST as a list type (i.e. an Aggregate ) that needs to contain an arbitrary number of STMTTRN elements. Hopefully the docs/comments in Aggregate.__init__() clear things up somewhat.

TL;DR Just pass the STMTTRN instances (i.e. the contained items) into BANKTRANLIST as positional arguments. Pass the attributes (dtstart, dtend) as keyword arguments.

[csingley]

By the way - Plaid can eat shit & die, but enabling the kind of script you are working on is the entire reason I released this library. Yes you can indeed use ofxtools for these applications.

[philliphall]

Thank you so much for your prompt and helpful response!

I am curious about your experience with Plaid. Should I reconsider my life choices?

I still can't say that I truly understand what is happening in the class definitions with Aggregate and SubAggregate - I marvel at your brilliance and acknowledge I have a lot yet to learn. Using the .append function got me where I need to be. If you think it'd be helpful, I'd be happy to contribute another example for https://ofxtools.readthedocs.io/en/latest/generating.html that includes transactions.

Basically...

banktranlist = BANKTRANLIST(dtstart=dtstart, dtend=dtend)

banktranlist.append(STMTTRN(trntype=trans['trntype'],
                                                dtposted=trans['dtposted'],
                                                trnamt=trans['amount'],
                                                fitid=trans['transaction_id'],
                                                checknum=trans['check_number'], 
                                                name=trans['merchant_name']))

[csingley]

Generally the way to build these deeply nested messages is to proceed bottom-up. So I'll tend to write something like this.

trn1 = STMTTRN(
    trntype=trans['trntype'],
    dtposted=trans['dtposted'],
    trnamt=trans['amount'],
    fitid=trans['transaction_id'],
    checknum=trans['check_number'], 
    name=trans['merchant_name'],
)

trn2 = STMTTRN(
    trntype=trans2['trntype'],
    dtposted=trans2['dtposted'],
    trnamt=trans2['amount'],
    fitid=trans2['transaction_id'],
    checknum=trans2['check_number'], 
    name=trans2['merchant_name'],
)

banktranlist = BANKTRANLIST(trn1, trn2, dtstart=dtstart, dtend=dtend)

Of course, as you get into it you'll find the massive verbosity and repetition will start driving you to create functions to map your input format to ofxtools model classes... for a reasonably fully worked example of which see ofxtools.Client.

I am curious about your experience with Plaid. Should I reconsider my life choices?

You should always reconsider your life choices - I recommend doing so while showering.

All these "fintech" startups, the Patagonia vest-clad legions who staff them, and the Sand Hill VCs who fund them ... they can pretty much all eat shit and die together, with the possible exception of Stripe (whose services I haven't used, but who at least do a decent job of passing themselves off as actual humans instead of glitter-eyed thralls of the Green Manalishi).

Plaid's tainted blood manifested right from the beginning - there was no juvenile phase of "don't be evil" type ideals; they burst right out of the crib humming the Zero-Sum Anthem and hustling marks. The "growth hacking"... how about you just hand over all your financial account passwords; what could possibly go wrong??

But really this kind of thing is about par for the course in post-great America. What earns my particular ire, in this forum, is their having joined up with the supergroup of assholes that is Financial Data Exchange - who has taken custody of OFX protocol, and is now holding the pillow tightly over its face to accelerate its demise. They can't work fast enough to replace it with FDX, which offers strong security guarantees (backed by public-key cryptography) that you and I cannot gain direct access to our own financial data, but instead is comprehensively architected to structurally require some intermediary rentier parasite to unlock what they have gone to some effort to lock away.

OFX is going the way of the dodo, and some sort of open-source fdxtools is infeasible by design.

A pox on all their houses.
Power to the people, and ban the fucking bomb.

We now return you to your regularly scheduled accounting software program.

[philliphall]

I very thoroughly enjoyed that entire post! This may be yet another way that the human tendency to value convenience over quality leads us into slavery. Or maybe it will lead us into crypto... cause that's never caused anyone to lose any sleep...

[csingley]

P.S. If you want to contribute docs, that's great - you are entirely welcome to do so. The few dozen of us who still cling to this legacy protocol will be most appreciative.

[philliphall]

Got my code functional and just have to come back with a huge thank you for this module and for your help! I vastly underestimated the complexity of compliance with the OFX standard - couldn't have done it without you.

Of course I am very amateur and you will likely find lots of things I didn't do right, but if any of it is useful to "The few dozen of us who still cling to this legacy protocol", you are more than welcome.

https://github.com/philliphall/plaid2qfx

[philliphall]

Suggested update to generating.rst (renamed as .txt so I can upload it here. Sorry, too lazy to fork and pull... still haven't documented my own script yet!)

generating.txt