Updating the LLVM backend

[tmcdonell]

There have been a bunch of updates and bug fixes in the CUDA backend since the LLVM backends were originally written, and we should make sure we have got those fixes in LLVM too. This is a place to keep track of all the relevant changes.

• New memory manager
• Segfaults with profiling turned on #260
• 'invalid device context' and other concurrency issues #261
• Tracking other resources with Lifetimes for correct finalisation

[tmcdonell]

@robeverest do you remember any other bugs you have squashed that may be relevant? I only had a quick look through the history.

[tmcdonell]

Similarly, we should back port any changes the LLVM backend made to CUDA. In particular, I did some experiments with different data structures for the memory tables and at that time IntMap was the winner (by a large margin, if I remember correctly). We should do more comprehensive tests though. Judy arrays are another candidate I intended to look into.

[robeverest]

Having a look through the commit history, I can't see much that immediately springs to mind as needing to be moved over to the LLVM backend. The only thing that does, and I'm not sure whether you're including this under the new memory manager, is the tracking of resources other than allocated arrays (i.e. contexts, streams, programs and events) with lifetimes.

I would not be surprised if IntMaps were a faster alternative to hashtables. We should also do some testing to see how much time looking up arrays in the memory table takes of the total runtime. I have a suspicion that tracking the time the array was last accessed has added to that, because every read is now also a write.

Also, if there's anything you're unsure about with the memory manager API, just let me know.

[tmcdonell]

Right, I saw that more things are wrapped in lifetimes now. I will look into that.

That is a good point. We should definitely do some benchmarking of the LRU in that case.

The memory manager is a bit hanky so I have been beating it with a sack full of doorknobs. So far it has not resisted arrest, but I will let you know how that goes.

[tmcdonell]

Closing as the LLVM backends are feature complete at this point, and I don't think the remaining points should keep it open:

• Segfaults with profiling turned on #260: I can't test this at the moment because both cuda (Build fails with library profiling tmcdonell/cuda#43) and llvm-general (Abort build when compiling with library-profiling enabled bscarlet/llvm-general#195) fail to compile with profiling enabled
• 'invalid device context' and other concurrency issues #261: I haven't encountered any 'invalid context errors', so this seems to be fine...?