inline article media

[max-mapper]

Have you looked into getting the images from articles as well? From what I understand Kiwix has distributions where they bundle images in (for example wikipedia_en_all_2016-12.zim is 62GB)... I actually wrote a parser for their format recently https://www.npmjs.com/package/zimmer but havent finished doing the actual import-to-dat part :P

[derhuerst]

The ZIM parser is pretty cool! The reason I went for the wiki dumps is that they're being frequently built by Wikimedia itself, not sure about the ZIMs. But that would be an option too. For now, I'd like to focus on bringing a whole Wikipedia (e.g. enwiki) into dat, but I repeatedly stumble over joehand/dat-next#122 (comment) . It always fails after 600 articles.

[derhuerst]

For images & additional data to be stored, I guess it would be wise to have each article as a directory, containing an index.md or article.md index.html or article.html.

[derhuerst]

FYI: As of 8111d48, there is now one dir per article, containing index.html and index.json with metadata.

Thinking about media, it seems to me that it would make sense not to store the media next to the articles, as one image can be used by multiple articles.

[derhuerst]

https://github.com/openzim/mwoffliner might also be interesting.