External blogposts from substack appear locally but throws an error upon deployment

[aqcheng]

Acknowledge the following

• I carefully read and followed the Getting Started guide.
• I read through FAQ and searched through the past issues, none of which addressed my issue.
• The issue I am raising is a potential bug in al-folio and not just a usage question.
  [For usage questions, please post in the Discussions instead of raising an issue.]

Describe the bug
I have a website that has been working fine. I recently decided to move my blog to substack and use the external RSS feed instead, but I am getting the error

/home/runner/work/aqcheng.github.io/aqcheng.github.io/vendor/bundle/ruby/3.0.0/gems/feedjira-3.2.2/lib/feedjira.rb:63:in `parse': No valid parser for XML. (Feedjira::NoParserAvailable)

and the website fails to deploy. It works again when I remove the external feed from the _config.yml file. The website displays the blogs fine locally. Full error message is attached.
deploy errmsg.txt

To Reproduce
Steps to reproduce the behavior:

In _config.yml, add an external RSS feed i.e.

external_sources:
   - name: substack.com
     rss_url: https://aqcheng.substack.com/feed

Expected behavior
Github deployment mimics local deployment.

System (please complete the following information):

• Jekyll version 4.3.3
• Ruby version 3.1.4 locally, 3.0.2 on Github pages

[alshedivat]

I tried adding your external source to config and didn't run into any issues building it locally from the master branch. my only guess is maybe you just need to update ruby to 3.2.2? (both locally and in your github runner config)

[aqcheng]

How do I update ruby in my github runner config? Also, I didn't run into any issues building it locally; it's the deployment from github that is problematic

[alshedivat]

are you using github actions for automatic deployment? if that's the case, you just need to update the version in .github/workflows/deploy.yml config:

al-folio/.github/workflows/deploy.yml

Lines 21 to 25 in 1d84621

	- name: Setup Ruby
	uses: ruby/setup-ruby@v1
	with:
	ruby-version: "3.2.2"
	bundler-cache: true

[aqcheng]

I tried both 3.1.4 (the version I have locally) and 3.2.2, but with no luck. Same error message. I'm positive the problem is with feedjira because it deploys fine when I don't include the external feed, but the ruby, feedjira, and jekyll versions on the Github pages deployment are all identical to my local ruby installation.

I am using bundle exec jekyll serve to deploy the website locally (i.e. not using docker), and I believe my website was cloned from v0.6.0. I've customized my website pretty extensively so I think it'd be hard to update it. Do you have any idea what the issue could be?

[george-gca]

Before anything try to replicate the GitHub website locally. Do exactly what is done in the deploy action, more specifically:

export JEKYLL_ENV=production
bundle exec jekyll build --lsi

The --lsi is just if you use the related posts part. Then serve the site locally with python3 -m http.server -d _site/ and test it in http://localhost:8000/. When you build the site with JEKYLL_ENV=production it does some changes on the build process.

---

One thing that might be causing the issue is older libraries. For instance in #2073 the site of some users were throwing some errors due to deprecated old libraries. Maybe if you update them it could work. Also as I suggested there you could get the tip of master branch and the tip of yours and run a diff between the two directories with a tool like meld or winmerge. They make the process of manual merge a little easier.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[harmonbhasin]

Had a similar issue with trying to include my substack. After printing out the variable xml, it seems that I'm not even getting the rss feed, but rather am getting the html for a page that says 'Just a moment...', which I assume is CloudFlare blocking Github.

I found that others had a similar issue, Athou/commafeed#1138. I'm looking into solutions, but I figured I'd leave a comment.

[harmonbhasin]

Came up with a solution, it's kind of rough, in external-posts.rb, I added the following line after reading in the xml file,

if xml.include?('<!DOCTYPE html>') xml = File.read('./substack.rss') end

where 'substack.rss' is generated by calling the following bash command curl https://your.substack.com/feed > substack.rss. This won't update every time you post to substack since it's relying on the file, but it's a good enough temporary fix.

[george-gca]

What if you somehow set the user agent used to fetch this information? Like it is done in here?