Skip to content

Commit

Permalink
Merge branch 'zone-eu:master' into v3.6.5-custom
Browse files Browse the repository at this point in the history
  • Loading branch information
rjnz2023 authored Nov 16, 2023
2 parents 8f7a5d2 + a851f1e commit 6bb2418
Show file tree
Hide file tree
Showing 13 changed files with 10,108 additions and 60 deletions.
37 changes: 37 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
on:
push:
branches:
- master

permissions:
contents: write
pull-requests: write
id-token: write

name: release
jobs:
release-please:
runs-on: ubuntu-latest
steps:
- uses: google-github-actions/release-please-action@v3
id: release
with:
release-type: node
package-name: ${{env.NPM_MODULE_NAME}}
pull-request-title-pattern: 'chore${scope}: release ${version} [skip-ci]'
# The logic below handles the npm publication:
- uses: actions/checkout@v3
# these if statements ensure that a publication only occurs when
# a new release is created:
if: ${{ steps.release.outputs.release_created }}
- uses: actions/setup-node@v3
with:
node-version: 18
registry-url: 'https://registry.npmjs.org'
if: ${{ steps.release.outputs.release_created }}
- run: npm ci
if: ${{ steps.release.outputs.release_created }}
- run: npm publish --provenance --access public
env:
NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
if: ${{ steps.release.outputs.release_created }}
48 changes: 48 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Run tests

on:
push:
pull_request:

jobs:
test:
strategy:
matrix:
node: [16.x, 18.x, 20.x]
os: [ubuntu-20.04]
runs-on: ${{ matrix.os }}
# Service containers to run with `container-job`
services:
# Label used to access the service container
redis:
# Docker Hub image
image: redis
# Set health checks to wait until redis has started
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4

- name: Start MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 4.4
mongodb-port: 27017
mongodb-db: wildduck-test

- name: Use Node.js ${{ matrix.node }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
- run: which mongo
- run: npm install
- name: Run tests
run: npm run runtest
env:
APPCONF_dbs_redis: redis://127.0.0.1:6379/1
APPCONF_dbs_mongodb: 'mongodb://127.0.0.1:27017/wildduck-test?authSource=admin'
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ test/queuetest
config/production.*
config/development.*
*v8.log
package-lock.json*

7 changes: 7 additions & 0 deletions .ncurc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
module.exports = {
upgrade: true,
reject: [
// mongodb 5.x driver does not support callbacks, only promises
'mongodb'
]
};
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@postalsys:registry=https://registry.npmjs.org/
59 changes: 58 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,61 @@
# CHANGELOG
# Changelog

## [3.6.13](https://github.com/zone-eu/zone-mta/compare/v3.6.12...v3.6.13) (2023-10-12)


### Bug Fixes

* **deploy:** updated package-lock to fix release ([4248526](https://github.com/zone-eu/zone-mta/commit/42485265748bef6cbcf34eb570fdfae422128abb))

## [3.6.12](https://github.com/zone-eu/zone-mta/compare/v3.6.11...v3.6.12) (2023-10-12)


### Bug Fixes

* **queue:** mark emails with 'mailbox full' responses as rejected, not deferred ([c2040e2](https://github.com/zone-eu/zone-mta/commit/c2040e284e337e79c6987b3cf393c669d08a04c8))
* **queue:** replaced non-likely bounces from deferred to rejected ([c701efd](https://github.com/zone-eu/zone-mta/commit/c701efde4907ea969aef92e7dbfc517a993d8ce9))

## [3.6.11](https://github.com/zone-eu/zone-mta/compare/v3.6.10...v3.6.11) (2023-10-05)


### Bug Fixes

* **mail2http:** retry 5xx responses ([33aa6d5](https://github.com/zone-eu/zone-mta/commit/33aa6d5b3c726206563649137de2b42e9bb38bd7))

## [3.6.10](https://github.com/zone-eu/zone-mta/compare/v3.6.9...v3.6.10) (2023-09-29)


### Bug Fixes

* **bounces:** Fixed MS365 missing user bounce check regex ([3e0853a](https://github.com/zone-eu/zone-mta/commit/3e0853aef154bdfcdaf7d879017559991e03b507))

## [3.6.9](https://github.com/zone-eu/zone-mta/compare/v3.6.8...v3.6.9) (2023-09-18)


### Bug Fixes

* **deploy:** Added repository URL for automatic publishing ([6172534](https://github.com/zone-eu/zone-mta/commit/6172534567299e0c9c568446c6b62e87949e9272))

## [3.6.8](https://github.com/zone-eu/zone-mta/compare/v3.6.7...v3.6.8) (2023-09-18)


### Bug Fixes

* **readme:** Updated readme to trigger package publish ([b90d31b](https://github.com/zone-eu/zone-mta/commit/b90d31b610862bf1c9db9c4df41af1a41a905866))

## [3.4.1](https://github.com/zone-eu/zone-mta/compare/v3.4.0...v3.4.1) (2023-09-18)


### Bug Fixes

* **deploy:** Set up auto publishing to NPM ([#338](https://github.com/zone-eu/zone-mta/issues/338)) ([b1d6fef](https://github.com/zone-eu/zone-mta/commit/b1d6fef8931a4e40042694414318d1dd5fda2914))
* **tests:** Added Github action to run tests ([#341](https://github.com/zone-eu/zone-mta/issues/341)) ([908edf9](https://github.com/zone-eu/zone-mta/commit/908edf9321be5342426127d081997bde116ff23b))

## CHANGELOG

- v3.6.6 2023-05-24

- Bumped mx-connect for MTA-STS caching fix

- v3.6.3 2023-02-08

Expand Down
31 changes: 22 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ZoneMTA (internal code name X-699)

Modern outbound SMTP relay (MTA/MSA) built on Node.js and MongoDB (queue storage). It's kind of like Postfix for outbound but is able to use multiple local IP addresses and is easily extendable using plugins.
Outbound SMTP relay (MTA/MSA) built on Node.js and MongoDB (queue storage).

```
███████╗ ██████╗ ███╗ ██╗███████╗███╗ ███╗████████╗ █████╗
Expand All @@ -17,21 +17,15 @@ ZoneMTA is comparable to [Haraka](https://haraka.github.io/) but unlike Haraka i

There's also a web-based [administration interface](https://github.com/zone-eu/zmta-webadmin) (needs to be installed separately).

## Upgrade notes

ZoneMTA version 1.1 uses a different application configuration scheme than 1.0. See [zone-mta-template](https://github.com/zone-eu/zone-mta-template) for reference.

Also, there is no zone-mta command line application anymore, you need to include it as a module.

## Requirements

1. **Node.js** v8.0.0+ for running the app
1. **Node.js** v16+ for running the app
2. **MongoDB** for storing messages in the queue
3. **Redis** for locking and counters

## Quickstart

Assuming [Node.js](https://nodejs.org/en/download/package-manager/) (v8.0.0+), _MongoDB_ running on localhost and _git_. There must be nothing listening on ports 2525 (SMTP), 12080 (HTTP API) and 12081 (internal data channel). All these ports are configurable.
Assuming [Node.js](https://nodejs.org/en/download/package-manager/) (v16+), _MongoDB_ running on localhost and _git_. There must be nothing listening on ports 2525 (SMTP), 12080 (HTTP API) and 12081 (internal data channel). All these ports are configurable.

#### Create ZoneMTA application

Expand Down Expand Up @@ -199,6 +193,8 @@ Messages might get lost if the database gets into a corrupted state and it is no

You can assign a new IP to the IP pool using lower load share than other addresses by using `ratio` option (value in the range of 0 and 1 where 0 means that this IP is never used and 1 means that only this IP is used)

in `default.js` file:

```javascript
{
pools: {
Expand All @@ -213,6 +209,23 @@ You can assign a new IP to the IP pool using lower load share than other address
}
```

or in `pools.toml` file:

```
[[ipWarmPool]]
address="1.2.3.1"
name="host1.example.com"
[[ipWarmPool]]
address="1.2.3.2"
name="host2.example.com"
[[ipWarmPool]]
address="1.2.3.3"
name="warmup.example.com"
ratio="0.05"
```

Once your IP address is warm enough then you can either increase the load ratio for it or remove the parameter entirely to share load evenly between all addresses. Be aware though that every time you change pool structure it mixes up the address resolving, so a message that is currently deferred for greylisting does not get the same IP address that it previously used and thus might get greylisted again.

### Delivery to HTTP
Expand Down
64 changes: 32 additions & 32 deletions config/bounces.txt
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@
# 550 5.7.1 Unfortunately, messages from [1.2.3.4] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer
^550[ \-]5\.7\.\d+.*block list,defer,blacklist,Sender IP blocked

^550[ \-]5\.4\.1.*Recipient address rejected: Access denied.* \[.*\.prod\.protection\.outlook\.com\],reject,recipient,Unknown user
^550[ \-]5\.4\.1.*Recipient address rejected: Access denied.* \[.*\.prod\.protection\.outlook\.com,reject,recipient,Unknown user

# 550 5.7.606 Access denied, banned sending IP [1.2.3.4]. To request removal from this list please visit https://sender.office.com/ and follow the directions. For more information please go to http://go.microsoft.com/fwlink/?LinkID=526655 (AS16012609) [DB5EUR01FT031.eop-EUR01.prod.protection.outlook.com]
# 550 5.7.511 Access denied, banned sender[1.2.3.4]. To request removal from this list please forward this message to [email protected]. For more information please go to http://go.microsoft.com/fwlink/?LinkId=526653. AS(508) [AM5EUR02FT019.eop-EUR02.prod.protection.outlook.com]
Expand Down Expand Up @@ -406,65 +406,65 @@ IPTS04,defer,spam,temporarily deferred due to user complaints

^\d{3}[ \-]+4\.0\.0\b,defer,other,Unknown Issue
^\d{3}[ \-]+4\.1\.0\b,defer,envelope,Unknown Issue With Recipient's Email Address
^\d{3}[ \-]+4\.1\.1\b,defer,envelope,User Name In Email Address Not Valid
^\d{3}[ \-]+4\.1\.2\b,defer,envelope,Domain Name Of Email Address Not Valid
^\d{3}[ \-]+4\.1\.3\b,defer,envelope,Email Address Malformed
^\d{3}[ \-]+4\.1\.1\b,reject,envelope,User Name In Email Address Not Valid
^\d{3}[ \-]+4\.1\.2\b,reject,envelope,Domain Name Of Email Address Not Valid
^\d{3}[ \-]+4\.1\.3\b,reject,envelope,Email Address Malformed
^\d{3}[ \-]+4\.1\.4\b,defer,recipient,Multiple Mailboxes For Recipient
^\d{3}[ \-]+4\.1\.5\b,defer,recipient,Valid Email Address
^\d{3}[ \-]+4\.1\.6\b,defer,recipient,Email Address No Longer Accepts Or Forwards Messages
^\d{3}[ \-]+4\.1\.7\b,defer,envelope,Sender's Address Is Written Incorrectly
^\d{3}[ \-]+4\.1\.8\b,defer,config,Sending Address Is Bad Or Can't Accept Replies
^\d{3}[ \-]+4\.1\.9\b,defer,recipient,Message Routed To Non-Email System
^\d{3}[ \-]+4\.1\.6\b,reject,recipient,Email Address No Longer Accepts Or Forwards Messages
^\d{3}[ \-]+4\.1\.7\b,reject,envelope,Sender's Address Is Written Incorrectly
^\d{3}[ \-]+4\.1\.8\b,reject,config,Sending Address Is Bad Or Can't Accept Replies
^\d{3}[ \-]+4\.1\.9\b,reject,recipient,Message Routed To Non-Email System
^\d{3}[ \-]+4\.2\.0\b,defer,recipient,Unknown Issue With Mailbox
^\d{3}[ \-]+4\.2\.1\b,defer,capacity,Mailbox Is Not Accepting New Messages
^\d{3}[ \-]+4\.2\.2\b,defer,capacity,Mailbox Is Full
^\d{3}[ \-]+4\.2\.3\b,defer,message,Message Is Too Long
^\d{3}[ \-]+4\.2\.1\b,reject,capacity,Mailbox Is Not Accepting New Messages
^\d{3}[ \-]+4\.2\.2\b,reject,capacity,Mailbox Is Full
^\d{3}[ \-]+4\.2\.3\b,reject,message,Message Is Too Long
^\d{3}[ \-]+4\.2\.4\b,defer,recipient,Mailbox Has A Mailing List System Issue
^\d{3}[ \-]+4\.3\.0\b,defer,recipient,Unknown Issue With The Receiving System
^\d{3}[ \-]+4\.3\.1\b,defer,capacity,Email System Is Full
^\d{3}[ \-]+4\.3\.1\b,reject,capacity,Email System Is Full
^\d{3}[ \-]+4\.3\.2\b,defer,recipient,Receiving Email System Not Operating
^\d{3}[ \-]+4\.3\.3\b,defer,message,Message Features Not Supported By Receiving System
^\d{3}[ \-]+4\.3\.4\b,defer,message,Message Is Too Large
^\d{3}[ \-]+4\.3\.5\b,defer,recipient,Recipient System Incorrectly Configured
^\d{3}[ \-]+4\.3\.3\b,reject,message,Message Features Not Supported By Receiving System
^\d{3}[ \-]+4\.3\.4\b,reject,message,Message Is Too Large
^\d{3}[ \-]+4\.3\.5\b,reject,recipient,Recipient System Incorrectly Configured
^\d{3}[ \-]+4\.3\.6\b,defer,message,Message Priority Issue
^\d{3}[ \-]+4\.4\.0\b,defer,network,Unknown Issue With Network Connection
^\d{3}[ \-]+4\.4\.1\b,defer,network,Network Is Busy
^\d{3}[ \-]+4\.4\.2\b,defer,network,Network Connection Timed Out
^\d{3}[ \-]+4\.4\.3\b,defer,network,Network Issue With The Receiving System
^\d{3}[ \-]+4\.4\.4\b,defer,network,Email Routing Issue Due To Network
^\d{3}[ \-]+4\.4\.5\b,defer,network,Overloaded Network
^\d{3}[ \-]+4\.4\.6\b,defer,network,Network Issue Caused A Mail Loop
^\d{3}[ \-]+4\.4\.6\b,reject,network,Network Issue Caused A Mail Loop
^\d{3}[ \-]+4\.4\.7\b,defer,network,Message Expired Before Delivery Completed
^\d{3}[ \-]+4\.5\.0\b,defer,config,Unknown Issue With Your Email Server
^\d{3}[ \-]+4\.5\.1\b,defer,protocol,Bad Messaging Command In Transmission
^\d{3}[ \-]+4\.5\.2\b,defer,protocol,Email System Sent Bad Command
^\d{3}[ \-]+4\.5\.3\b,defer,message,Too Many Recipients For Message
^\d{3}[ \-]+4\.5\.4\b,defer,protocol,Email System Sent Bad Command
^\d{3}[ \-]+4\.5\.5\b,defer,protocol,Email Systems Were Incompatible
^\d{3}[ \-]+4\.5\.6\b,defer,protocol,Messaging Command In Transmission Too Long
^\d{3}[ \-]+4\.5\.1\b,reject,protocol,Bad Messaging Command In Transmission
^\d{3}[ \-]+4\.5\.2\b,reject,protocol,Email System Sent Bad Command
^\d{3}[ \-]+4\.5\.3\b,reject,message,Too Many Recipients For Message
^\d{3}[ \-]+4\.5\.4\b,reject,protocol,Email System Sent Bad Command
^\d{3}[ \-]+4\.5\.5\b,reject,protocol,Email Systems Were Incompatible
^\d{3}[ \-]+4\.5\.6\b,reject,protocol,Messaging Command In Transmission Too Long
^\d{3}[ \-]+4\.6\.0\b,defer,message,Unknown Issue With The Message Content
^\d{3}[ \-]+4\.6\.1\b,defer,message,Message Content Or Attachment Not Supported
^\d{3}[ \-]+4\.6\.2\b,defer,message,Message Content Not Accepted By Recipient Server
^\d{3}[ \-]+4\.6\.1\b,reject,message,Message Content Or Attachment Not Supported
^\d{3}[ \-]+4\.6\.2\b,reject,message,Message Content Not Accepted By Recipient Server
^\d{3}[ \-]+4\.6\.3\b,defer,message,Message Content Could Not Be Processed For Delivery
^\d{3}[ \-]+4\.6\.4\b,defer,network,Content Loss During Delivery
^\d{3}[ \-]+4\.6\.5\b,defer,network,Message Content Not Accessible To Recipient Server
^\d{3}[ \-]+4\.6\.6\b,defer,network,Email Content Could Not Be Accessed
^\d{3}[ \-]+4\.6\.7\b,defer,envelope,Email Address Format Not Supported
^\d{3}[ \-]+4\.6\.7\b,reject,envelope,Email Address Format Not Supported
^\d{3}[ \-]+4\.6\.8\b,defer,protocol,Email Servers Are Not Responding Appropriately
^\d{3}[ \-]+4\.6\.9\b,defer,protocol,Email Servers Are Not Communicating With Each Other
^\d{3}[ \-]+4\.6\.10\b,defer,protocol,Email Servers Are Not Responding Appropriately
^\d{3}[ \-]+4\.7\.0\b,defer,policy,Unknown Security Or Policy Concern
^\d{3}[ \-]+4\.7\.1\b,defer,block,Message Sender Blocked By Receiving Server
^\d{3}[ \-]+4\.7\.2,defer,auth,Not Authorized To Send To Mailing List
^\d{3}[ \-]+4\.7\.2,reject,auth,Not Authorized To Send To Mailing List
^\d{3}[ \-]+4\.7\.3,defer,other,Security Features Could Not Be Interpreted
^\d{3}[ \-]+4\.7\.4,defer,auth,User Authentication For Message Failed
^\d{3}[ \-]+4\.7\.5,defer,message,Message Authentication Issues
^\d{3}[ \-]+4\.7\.6,defer,message,Message Encryption Issues
^\d{3}[ \-]+4\.7\.7,defer,message,Potentially Corrupt Message
^\d{3}[ \-]+4\.7\.5,reject,message,Message Authentication Issues
^\d{3}[ \-]+4\.7\.6,reject,message,Message Encryption Issues
^\d{3}[ \-]+4\.7\.7,reject,message,Potentially Corrupt Message
^\d{3}[ \-]+4\.7\.8,defer,auth,User Authentication Failed
^\d{3}[ \-]+4\.7\.9,defer,auth,Security Issue With User Authentication
^\d{3}[ \-]+4\.7\.10\b,defer,auth,Stronger Security Or Encryption Needed
^\d{3}[ \-]+4\.7\.11\b,defer,auth,Encryption Needed For Network Connection
^\d{3}[ \-]+4\.7\.10\b,reject,auth,Stronger Security Or Encryption Needed
^\d{3}[ \-]+4\.7\.11\b,reject,auth,Encryption Needed For Network Connection
^\d{3}[ \-]+4\.7\.12\b,defer,auth,Authentication Required
^\d{3}[ \-]+4\.7\.13\b,defer,block,Sender Account Disabled
^\d{3}[ \-]+4\.7\.14\b,defer,spam,Potential Security Issue For Sender
Expand Down Expand Up @@ -543,7 +543,7 @@ IPTS04,defer,spam,temporarily deferred due to user complaints
# General errors
^450\b,defer,other,Mailbox unavailable
^451\b,defer,other,Local error in processing
^452\b,defer,capacity,Insufficient system storage
^452\b,reject,capacity,Insufficient system storage
^500\b,reject,protocol,Syntax error, command unrecognized
^501\b,reject,protocol,Syntax error in parameters or arguments
^502\b,reject,protocol,Command not implemented
Expand Down
7 changes: 7 additions & 0 deletions lib/queue-locker.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,10 @@ class QueueLocker {

lock(key, zone, domain, lockOwner, maxConnections, ttl) {
if (this.lockExists(key)) {
console.log('LOCK EXISTS', key);
return false;
}
console.log('SETTING LOCK', key);

/*
if (domain && this.domainIsSkipped(zone, domain)) {
Expand Down Expand Up @@ -113,13 +115,17 @@ class QueueLocker {

release(key) {
if (!key) {
console.log('NOT LOCKED 1', key);
return false;
}

if (!this.locks.has(key)) {
console.log('NOT LOCKED 2', key);
return false;
}

console.log('RELEASING LOCK', key);

let lock = this.locks.get(key);
this.locks.delete(key);

Expand Down Expand Up @@ -160,6 +166,7 @@ class QueueLocker {

releaseLockOwner(lockOwner) {
if (!this.lockOwners.has(lockOwner)) {
console.log('RELEASE OWNER', lockOwner);
return false;
}
let keys = [];
Expand Down
Loading

0 comments on commit 6bb2418

Please sign in to comment.