Skip to content
This repository has been archived by the owner on Apr 19, 2020. It is now read-only.

Consider adding support for off-heap vectors #9

Open
densh opened this issue Mar 17, 2015 · 4 comments
Open

Consider adding support for off-heap vectors #9

densh opened this issue Mar 17, 2015 · 4 comments

Comments

@densh
Copy link
Owner

densh commented Mar 17, 2015

Continuos arrays are not a perfect match for paged regions (due to maximum allocation size restriction.) To make it easier to work with collections of data in regions we can introduce a different collection (e.g. Vector) that stores data in linked (potentially tree-based) groups of small arrays that fit well in pages.

@densh densh added this to the 0.1 milestone Mar 17, 2015
@densh densh removed this from the 0.1 milestone Apr 24, 2015
@densh
Copy link
Owner Author

densh commented May 20, 2015

Another idea: Instead of making arrays more complicated we can just allocate larger arrays directly from system memory and free them once region ends. This would put them on slow path but won't require users to specially structure their code to fit into region page system.

@densh
Copy link
Owner Author

densh commented May 21, 2015

Yet another idea: Use variable-size pages in memory pooling layer. This way region can just request for page as big as necessary to allocate large array.

@densh densh added this to the 0.1 milestone May 21, 2015
@DarkDimius
Copy link
Collaborator

allocate larger arrays directly from system memory and free them once region ends

By doing this you're just offloading the same problem to OS allocator. It's likely to be more capable of solving it, but the problem still persists.

@densh densh changed the title Add support for blocked arrays Consider adding support for off-heap vectors May 28, 2015
@densh densh removed this from the 0.1 milestone May 28, 2015
@densh densh added the Maybe label May 28, 2015
@densh densh added the Feature label Jun 5, 2015
@densh
Copy link
Owner Author

densh commented Jul 19, 2015

Another use-case: efficient implementation of size-changing operations on arrays like filter as discussed in #39 (comment)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants