BaseXenForoForumAdapters

New sites forums.spacebattles.com and forums.sufficientvelocity.com

...Both base_xenforoforum adapters sharing almost all their code.

Forums are not really designed to host story, but people use them that way anyway. I recently started reading a fandom (the Worm web serial by Wildbow) and the largest collection of stories I found for it was on a couple of near identical forum sites.

The biggest problem with downloading stories posted to a forum is picking out the 'chapter' posts from all the other posts, since most stories on these sites combine chapters and reader comments. Plus, there's more than one way such stories are organized.

I've identified and supported three different story forms:

• First, a story thread can have 'Threadmarks' which link to specific posts. If a thread URL is given, and the thread has 2 or more Threadmarks, each Threadmarked post is considered a chapter. (A number of stories I found had a single Threadmark for an 'Index' post. It's not uncommon to also have an Index post (see below), but Threadmarks take priority.)

  • Example: https://forums.spacebattles.com/threads/queen-of-the-swarm-worm.323254/

• Second, a story thread can have an 'Index' in the first post containing links to chapter posts. If a thread URL is given, it's assumed that the first post of the thread is the Index. As an added complication, sometimes the Index post also contains the first chapter. The first post/index is included as the first chapter titled 'First Post'--unless it's also linked to explicitly in the post itself.

  • Example: https://forums.spacebattles.com/threads/oracle-worm-exalted.297826/

• Third, a story thread can have an Index post that isn't the first post of the thread. If a specific post URL is given, that post is used as the Index and included as 'First Post' if not explicitly linked. (I don't have the example off hand, but remember seeing one that included the most up-to-date index in the same post as a later chapter without linking to it. I have no way to detect that.)

  • Example: https://forums.spacebattles.com/posts/12990114/

• In either Index post case, an Index post can also contain links to chapter posts in other threads. Only links to posts on the same site will be included. It's common for long running stories to have several threads, which apparently precludes using Threadmarks.

  • Example: https://forums.spacebattles.com/threads/wail-of-the-banshee-3-boxhead-blues-and-the-zombocalypse.309577/

• In all cases, the title of the story is taken from the thread title.

In addition to the supported story forms, there are some common UNsupported story forms:

• I've seen stories that contain multiple chapter posts but lack either Threadmarks or an Index post. These stories are not supported because of the difficulty of distinguishing chapter posts from comment posts.

  • Example: https://forums.spacebattles.com/threads/walking-on-corpses-altpower-taylor-worm.313243/

• It's also common to for authors post short works or even several chapters in a single 'Discussion' thread.

  • Example: https://forums.spacebattles.com/threads/wormverse-ideas-recs-and-fic-discussion-thread-35.305746/page-75#post-14833156

Other details:

• strip_chapter_numbers:false is explicitly set for [base_xenforoforum] because of how many chapters labeled simply '1.1' etc I saw.

• Some fairly complex replace_metadata magic is included by default for [base_xenforoforum] to find 'tags' in the thread title and include them as tags, but not in the title. And remove 'Thread 5' etc.

  • Example: Way of the Open Hand (Worm/D&D) becomes Title: Way of the Open Hand with Tags: Worm, D&D

I have yet to find an effective 'update notice' mechanism for these sites. 'Watching' a thread works for story threads that have fallen dormant, but it's common for discussion (and derailments) to continue for weeks after the last chapter update.