Arachne is a simplistic net & spider toolbox. It contains the utilities you might have use for when storing link data mined on the Internet. It can also be used to validate and normalize data.
This repository is going to be populated with a selection of restructured utilities that already live in the system ecology at Jajja.
The project to gather and simplify tools currently used for crawling and handling net resources is ongoing. The current priority is:
- Documentation and examples
- URLs, Parameters (parsing and resolve)
- HTTP, Cookies, Proxies (possibly by wrapping an existing client)