Discourse email formatting

[ryantm]

The formatting after the weekly is emailed to Discourse is always quite bad. I have been manually editing it. It would be nice if it worked better.

https://discourse.nixos.org/t/nixos-weekly-06-nixos-weekly/7896

[ryantm]

The formatting is still messed up. I tried looking int this repo for information about how @domenkozar posts to Discourse and I don't see anything.

[domenkozar]

Emails are sent from mailchimp and then forwarded to discourse via ifttt.com

I haven't look into this because the email I get from mailchimp renders correctly.

I don't have much control over ifttt or discourse, but I can try using something else than ifttt to see if that fixes it.

[domenkozar]

Looking at what ifttt.com sends: <h1>News</h1> <h2>Announcements</h2>, meaning that it must be discourse email to markdown conversion that is imperfect.

[ryantm]

Raw email as reported by Discourse (click to expand)

Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133])
	by mail-receiver-stardestroyer03.localdomain (Postfix) with ESMTPS id 7E5221613D0
	for <[email protected]>; Fri, 27 Nov 2020 16:11:19 +0000 (UTC)
Received: by mail-il1-x133.google.com with SMTP id a19so5060269ilm.3
        for <[email protected]>; Fri, 27 Nov 2020 08:11:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=dev-si.20150623.gappssmtp.com; s=20150623;
        h=from:mime-version:date:message-id:subject:to;
        bh=kDWChQV280u+GWaWSpbLmUW2vuQTBCk5J3iZoEHn660=;
        b=TAyC0sDPCXFm5VRSOVHAZHv+X6qTmd6Z8LQjYeLo1ZXh5Xv3/dBNpsQOv+6yLmFD0h
         sNM00QhFoJEVGwAL7V0leXqurPVUydJRSndjNjcmE2zdDLYD0Mk+rNslEKPoaIJVBKnh
         9K/GsruvtLh2+ih732ktvje8mPtcBfxb5CLZNnTgmae2b9B+vmJebU7BGubwokQWkd24
         /PG2fuXGonux0BYpHNME2IL11JFUt2zS9c2LsSfedIT2tP2Kg4QPH7Q2eo7RPhBqPt8O
         ftuUNvd71h+phMncHL5IEzLXT/lkJ54qNTwQpoOvB8ATDjWP8Z+hm8Sj+ANmOoBvLHgT
         IMXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:mime-version:date:message-id:subject:to;
        bh=kDWChQV280u+GWaWSpbLmUW2vuQTBCk5J3iZoEHn660=;
        b=dor5nA/6xT6/jMgVJSJfzrHaDhCLP4aU3nEuAcpr9XzDwnCgvnWs3yULV+pqa3B9lg
         Z37Gwyfx113HrrKQAMwRel6Uh2mWRLYolqOQfDcou6eoOfUnQ00IjHL+8YggEFH8ROxC
         Ou1ThEl5E5GgiP4qoXbl3DorG+8UvSdMVvFb+/eoQb1wcfiwS5JXq4EIhWcbpfPOM3Fx
         51khZIYxOmQ1dMA6bQHZSY/h/e/qq4bqEOsT7ZMW18VLYit8qy/CUJb6SamdU+YSeKRd
         0+V/PcgJgev+/VoRUmYvqH+ihq89Koxk4parjgbMTDTgjjLho5wUVUPHRLiN9DZSsI2N
         rIRg==
X-Gm-Message-State: AOAM530VvJMF2CbhtwVl3E8Rr6awT+TXUJFuoNH4c3A70v2xulTkBq32
	48XtkD5EQBWG6wwoVoMrM0UTRgwqAJCUlU2aDBfqFa5oxtNmyA==
X-Google-Smtp-Source: ABdhPJyDCTrLERvaHMzHuHp9jB2YvYE0SLEErWsxrVs5uyeaBO8cVZtkul+CNP+5P71NrO2mbACx/gsbZSsDTjV+3xM=
X-Received: by 2002:a92:c64b:: with SMTP id 11mr7560388ill.224.1606493478415;
 Fri, 27 Nov 2020 08:11:18 -0800 (PST)
Received: from 427071021612 named unknown by gmailapi.google.com with
 HTTPREST; Fri, 27 Nov 2020 11:11:18 -0500
From: [email protected]
Mime-Version: 1.0
Date: Fri, 27 Nov 2020 11:11:17 -0500
Message-ID: <CAMvcdZSt6iXGq6-f5oX6AiAPk4-1HqxuOAQO+2uG2eWCq854fw@mail.gmail.com>
Subject: NixOS Weekly #09 - NixOS Weekly
To: [email protected]
Content-Type: multipart/alternative; boundary="0000000000001d8d0105b518e9c5"

--0000000000001d8d0105b518e9c5
Content-Type: text/plain; charset="UTF-8"

News Announcements

   - Announcing Nixpkgs Database
   <https://kamadorueda.github.io/nixpkgs-db/#/about>

A database and search engine for the entire history of each package at
Nixpkgs <https://github.com/nixos/nixpkgs>.

   - Cachix: write access control for binary caches
   <https://blog.cachix.org/posts/2020-11-09-write-access-control-for-binary-caches/>

The trust of who created binary cache contents comes from who owns the
private signing key.

This approach has a number of disadvantages when it comes to lifecycle in
teams, so API tokens are now the default way to manage write access.

   -

   NixOS 21.03 Call for Release Manager
   <https://discourse.nixos.org/t/21-03-call-for-release-manager/10204>
   -

   nixpkgs: Python 2.7 packages are no longer built
   <https://discourse.nixos.org/t/python-2-7-and-3-9-and-hydra-builds>
   -

   nixpkgs: Haskell packages now follow nightly
   <https://discourse.nixos.org/t/new-ghc-default-version-8-10-2-package-versions-now-from-stackage-nightly>
   -

   Mach-nix now has conda packages as a backend
   <https://discourse.nixos.org/t/mach-nix-create-python-environments-quick-and-easy/6858/86>
   -

   Running NixOS on Pinecube
   <https://github.com/danielfullmer/pinecube-nixos>
   -

   Our New Nix Deployment Tool: deploy-rs
   <https://serokell.io/blog/deploy-rs>

Tutorials

   -

   How to Setup Prometheus, Grafana and Loki on NixOS
   <https://christine.website/blog/prometheus-grafana-loki-nixos-2020-11-20>
   -

   Deploying NixOS using Terraform
   <https://nix.dev/tutorials/deploying-nixos-using-terraform.html>
   -

   Setting up i3 on Thinkpads running NixOS
   <https://www.srid.ca/i3-setup.html>
   -

   How to use NixOS for lightweight integration tests
   <http://www.haskellforall.com/2020/11/how-to-use-nixos-for-lightweight.html>
   -

   Lightweight Linux VMs on NixOS <https://www.srid.ca/2012301.html>
   -

   XMonad + Polybar on NixOS <https://gvolpe.com/blog/xmonad-polybar-nixos/>

Resources

   -

   Eight Months of NixOS <https://catgirl.ai/log/nixos-experience/>
   -

   One week of NixOS <https://jae.moe/blog/2020/11/one-week-of-nixos/>
   -

   Moving into NixOS
   <https://foo-dogsquared.github.io/blog/posts/moving-into-nixos/>
   -

   Automatically OCR scanned PDFs in NixOS
   <https://dev.to/jwoudenberg/automatically-ocr-scanned-pdfs-in-nixos-25d>

Annotated code showing you how to automatically create searchable scans of
all the documents you receive in the post.
Jobs

   -

   Internal Tooling & SRE at NoRedInk - Nix, Haskell, k8s
   <https://www.noredink.com/job_post?gh_jid=2443611>
   -

   DevOps Engineer (Munich or Remote)
   <https://genesis-jobs.personio.de/job/278290>

Contribute to NixOS Weekly Newsletter

This work would not be possible without the many contributions of the
community.

You can help too! Create or comment on the pull request
<https://github.com/NixOS/nixos-weekly/pulls> for the next edition or look
at the issue tracker <https://github.com/NixOS/nixos-weekly/issues> to add
other improvements.

--0000000000001d8d0105b518e9c5
Content-Type: text/html; charset="UTF-8"

<h1>News</h1>
<h2>Announcements</h2>
<ul>
<li><a href="https://kamadorueda.github.io/nixpkgs-db/#/about">Announcing Nixpkgs Database</a></li>
</ul>
<p>A database and search engine for the entire history of each package at <a href="https://github.com/nixos/nixpkgs">Nixpkgs</a>.</p>
<ul>
<li><a href="https://blog.cachix.org/posts/2020-11-09-write-access-control-for-binary-caches/">Cachix: write access control for binary caches</a></li>
</ul>
<p>The trust of who created binary cache contents comes from who owns the private signing key.</p>
<p>This approach has a number of disadvantages when it comes to lifecycle in teams, so API tokens are now the default way to manage write access.</p>
<ul>
<li>
<p><a href="https://discourse.nixos.org/t/21-03-call-for-release-manager/10204">NixOS 21.03 Call for Release Manager</a></p>
</li>
<li>
<p><a href="https://discourse.nixos.org/t/python-2-7-and-3-9-and-hydra-builds">nixpkgs: Python 2.7 packages are no longer built</a></p>
</li>
<li>
<p><a href="https://discourse.nixos.org/t/new-ghc-default-version-8-10-2-package-versions-now-from-stackage-nightly">nixpkgs: Haskell packages now follow nightly</a></p>
</li>
<li>
<p><a href="https://discourse.nixos.org/t/mach-nix-create-python-environments-quick-and-easy/6858/86">Mach-nix now has conda packages as a backend</a></p>
</li>
<li>
<p><a href="https://github.com/danielfullmer/pinecube-nixos">Running NixOS on Pinecube</a></p>
</li>
<li>
<p><a href="https://serokell.io/blog/deploy-rs">Our New Nix Deployment Tool: deploy-rs</a></p>
</li>
</ul>
<h2>Tutorials</h2>
<ul>
<li>
<p><a href="https://christine.website/blog/prometheus-grafana-loki-nixos-2020-11-20">How to Setup Prometheus, Grafana and Loki on NixOS</a></p>
</li>
<li>
<p><a href="https://nix.dev/tutorials/deploying-nixos-using-terraform.html">Deploying NixOS using Terraform</a></p>
</li>
<li>
<p><a href="https://www.srid.ca/i3-setup.html">Setting up i3 on Thinkpads running NixOS</a></p>
</li>
<li>
<p><a href="http://www.haskellforall.com/2020/11/how-to-use-nixos-for-lightweight.html">How to use NixOS for lightweight integration tests</a></p>
</li>
<li>
<p><a href="https://www.srid.ca/2012301.html">Lightweight Linux VMs on NixOS</a></p>
</li>
<li>
<p><a href="https://gvolpe.com/blog/xmonad-polybar-nixos/">XMonad + Polybar on NixOS</a></p>
</li>
</ul>
<h2>Resources</h2>
<ul>
<li>
<p><a href="https://catgirl.ai/log/nixos-experience/">Eight Months of NixOS</a></p>
</li>
<li>
<p><a href="https://jae.moe/blog/2020/11/one-week-of-nixos/">One week of NixOS</a></p>
</li>
<li>
<p><a href="https://foo-dogsquared.github.io/blog/posts/moving-into-nixos/">Moving into NixOS</a></p>
</li>
<li>
<p><a href="https://dev.to/jwoudenberg/automatically-ocr-scanned-pdfs-in-nixos-25d">Automatically OCR scanned PDFs in NixOS</a></p>
</li>
</ul>
<p>Annotated code showing you how to automatically create searchable scans of all the documents you receive in the post.</p>
<h2>Jobs</h2>
<ul>
<li>
<p><a href="https://www.noredink.com/job_post?gh_jid=2443611">Internal Tooling &amp; SRE at NoRedInk - Nix, Haskell, k8s</a></p>
</li>
<li>
<p><a href="https://genesis-jobs.personio.de/job/278290">DevOps Engineer (Munich or Remote)</a></p>
</li>
</ul>
<h1>Contribute to NixOS Weekly Newsletter</h1>
<p>This work would not be possible without the many contributions of the community.</p>
<p>You can help too! Create or comment on the <a href="https://github.com/NixOS/nixos-weekly/pulls">pull request</a> for the next edition or look at the <a href="https://github.com/NixOS/nixos-weekly/issues">issue tracker</a> to add other improvements.</p>


--0000000000001d8d0105b518e9c5--

[ryantm]

I think the problem is the text part of the email. edit: never mind, that doesn't seem likely

[ryantm]

text of Discourse post (click to expand)

# News## Announcements- [Announcing Nixpkgs Database](https://kamadorueda.github.io/nixpkgs-db/#/about)

A database and search engine for the entire history of each package at [Nixpkgs](https://github.com/nixos/nixpkgs).

- [Cachix: write access control for binary caches](https://blog.cachix.org/posts/2020-11-09-write-access-control-for-binary-caches/)

The trust of who created binary cache contents comes from who owns the private signing key.

This approach has a number of disadvantages when it comes to lifecycle in teams, so API tokens are now the default way to manage write access.

- [NixOS 21.03 Call for Release Manager](https://discourse.nixos.org/t/21-03-call-for-release-manager/10204)

- [nixpkgs: Python 2.7 packages are no longer built](https://discourse.nixos.org/t/python-2-7-and-3-9-and-hydra-builds)

- [nixpkgs: Haskell packages now follow nightly](https://discourse.nixos.org/t/new-ghc-default-version-8-10-2-package-versions-now-from-stackage-nightly)

- [Mach-nix now has conda packages as a backend](https://discourse.nixos.org/t/mach-nix-create-python-environments-quick-and-easy/6858/86)

- [Running NixOS on Pinecube](https://github.com/danielfullmer/pinecube-nixos)

- [Our New Nix Deployment Tool: deploy-rs](https://serokell.io/blog/deploy-rs)

## Tutorials- [How to Setup Prometheus, Grafana and Loki on NixOS](https://christine.website/blog/prometheus-grafana-loki-nixos-2020-11-20)

- [Deploying NixOS using Terraform](https://nix.dev/tutorials/deploying-nixos-using-terraform.html)

- [Setting up i3 on Thinkpads running NixOS](https://www.srid.ca/i3-setup.html)

- [How to use NixOS for lightweight integration tests](http://www.haskellforall.com/2020/11/how-to-use-nixos-for-lightweight.html)

- [Lightweight Linux VMs on NixOS](https://www.srid.ca/2012301.html)

- [XMonad + Polybar on NixOS](https://gvolpe.com/blog/xmonad-polybar-nixos/)

## Resources- [Eight Months of NixOS](https://catgirl.ai/log/nixos-experience/)

- [One week of NixOS](https://jae.moe/blog/2020/11/one-week-of-nixos/)

- [Moving into NixOS](https://foo-dogsquared.github.io/blog/posts/moving-into-nixos/)

- [Automatically OCR scanned PDFs in NixOS](https://dev.to/jwoudenberg/automatically-ocr-scanned-pdfs-in-nixos-25d)

Annotated code showing you how to automatically create searchable scans of all the documents you receive in the post.

## Jobs- [Internal Tooling & SRE at NoRedInk - Nix, Haskell, k8s](https://www.noredink.com/job_post?gh_jid=2443611)

- [DevOps Engineer (Munich or Remote)](https://genesis-jobs.personio.de/job/278290)

# Contribute to NixOS Weekly Newsletter

This work would not be possible without the many contributions of the community.

You can help too! Create or comment on the [pull request](https://github.com/NixOS/nixos-weekly/pulls) for the next edition or look at the [issue tracker](https://github.com/NixOS/nixos-weekly/issues) to add other improvements.

[ryantm]

It could very well be a problem with Discourse though, they have many bugs.

[domenkozar]

Are there settings on discourse to prefer html over plaintext when receiving email?

[ryantm]

Yes, and it is set to prefer html currently.

[ryantm]

[domenkozar]

That looks like discourse is parsing those emails wrong, is there a way to update discourse and see?

[domenkozar]

Probably due to https://github.com/discourse/discourse/blob/master/lib/html_to_markdown.rb

[ryantm]

Yeah. We are hosted by Discourse so we cannot update it to see, but we can submit failing tests or a fix. We could first try submitting a bug report on the Meta Discourse.

[ryantm]

I posted on the Discourse Meta https://meta.discourse.org/t/issues-with-html-to-markdown-converter-on-incoming-email/171518

[ryantm]

I just looked more closely at the HTML from your email and see that it has some bad formatting:

<ul>
<li><a href="https://kamadorueda.github.io/nixpkgs-db/#/about">Announcing Nixpkgs Database</a></li>
</ul>
<p>A database and search engine for the entire history of each package at <a href="https://github.com/nixos/nixpkgs">Nixpkgs</a>.</p>

The description paragraph is outside of the list item, so it doesn't seem like Discourse has any hope of realizing that should be matched up to the list item.

[domenkozar]

In that case it's a bug in WYSIWYG editor on mailchimp :(

Let's see if the newline issue was fixed by discourse.