Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: simplify configuration #105

Merged
merged 4 commits into from
Oct 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Data source
# Data source
TWITTER_HANDLE=username

// API
# API
TWITTER_USERNAME=username
TWITTER_PASSWORD=password
MASTODON_INSTANCE=mastodon.social
Expand All @@ -10,12 +10,10 @@ BLUESKY_INSTANCE=
BLUESKY_IDENTIFIER=
BLUESKY_PASSWORD=

// Touitomamout
INSTANCE_ID=
EXECUTION=manual
# EXECUTION=pm2
# Touitomamout
SYNC_MASTODON=true
SYNC_BLUESKY=true
SYNC_FREQUENCY_MIN=30
SYNC_PROFILE_DESCRIPTION=false
SYNC_PROFILE_PICTURE=false
SYNC_PROFILE_HEADER=false
Expand Down
93 changes: 5 additions & 88 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,92 +27,9 @@ Please find the project documentation here:

[<img src="https://github.com/louisgrasset/touitomamout/raw/main/.github/docs/documentation-center.svg" width="300px"/>](https://louisgrasset.github.io/touitomamout/docs/discover)

## Dependencies
Kudos to the following projects that made Touitomamout project possible 🙏
- 🦤 [twitter-scraper](https://github.com/the-convocation/twitter-scraper)
- 🦣 [masto.js](https://github.com/neet/masto.js)
- ☁️[atproto](https://github.com/bluesky-social/atproto)

## Installation
**Clone** the project
```bash
git clone [email protected]:louisgrasset/touitomamout.git
```

**Install** dependencies & build the project
```bash
npm ci && npm run build
```
## Configuration
Touitomamout relies on two APIs:
- [Mastodon](https://docs.joinmastodon.org/client/intro/)
- [Twitter-Scraper](https://github.com/the-convocation/twitter-scraper)

### Mastodon configuration
In order to communicate with the mastodon instance, you'll have to generate an API Token. It is totally free. Reminder: your application name will be publicly visible.
1. Go to your account's application page: `https://{yourinstance.tld}/settings/applications/new`
2. Create a new application with the following scopes:
- `read:accounts`: get your mastodon account username
- `write:media`: post medias
- `write:statuses`: post toots
- `write:accounts`: update your profile
3. Populate the [Environment](#Environment) section with your `access token`.

### Twitter configuration
The tweets retrieval by itself can be done without Twitter credentials. But keep in mind that twitter currently blocks guests to access users' replies.
Touitomamout is trying to restore the previous session, so you'll not get spammed by the Twitter security team for each connection.


> **Note**
>
> The configuration allows you to sync a first account and authenticate with a secondary account for two reasons:
> 1. Currently, there is no simple way to authenticate with an account having 2FA enabled, so you may not want to lower your main account security.
> 2. Because this project is running with a non-official API, you may not want to put your account at risk.


### Environment
First things first, please copy the [`.env.example`](https://github.com/louisgrasset/touitomamout/blob/main/.env.example) file to `.env`.
Then, please fill each variable.

> **Warning**
>
> Do not forget to properly choose the `EXECUTION` variable.
> Two values are allowed:
> 1. `manual`: a simple node script execution
> 2. `pm2`: spawns as a new PM2 process, named with `touitomamout-${instance_id}` pattern.


### Multiple instances
This project supports a multiple instances mode. To do so, simply provide multiple `.env` files such as `.env.instance1` and `.env.instance2`.

`deploy` & `deploy:update` scripts will handle them properly.

## Run it

### Manually
You can simply run a `node ./dist/index.js .env` and have a one shot sync of your recent tweets.

> **Note**
>
> Don't forget to replace `.env` with the right .env filename.


### Cron
Because automation is cool, feel free to run that script everytime you need to.
Simply create your cron [here](https://crontab.guru).

### PM2 support
[PM2](https://pm2.keymetrics.io/) is a utility that allows you to monitor and run periodically your node scripts.
Thus, it can be useful for some users to deploy Touitomamout to a PM2 instance.

#### **PM2**: First run
```bash
npm run deploy
```

#### **PM2**: Update your instance after a code update
Your instance will be removed and will be generated again with the latest codebase.
Your `cache.instance.json` file is kept, so you won't have duplicated toots.
```bash
npm ci &&
npm run build &&
npm run deploy:update
```

### Docker
You can alternatively rely on docker and use the `docker-compose.yml` file.
27 changes: 0 additions & 27 deletions deployment/deploy.sh

This file was deleted.

43 changes: 43 additions & 0 deletions deployment/pm2.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/bin/sh

script_path=$(realpath "$0")
script_dir=$(dirname "$script_path")

touitomamout="$script_dir/../dist/index.js"

# For each env file found (except .env.example)
for env in .env*; do
if [[ $env != *.example ]]; then
# Prevent startup with PM2 if the internal DAEMON is set to true. PM2 will handle the cron itself.
daemon=$(grep "DAEMON=" "$env" | cut -d '=' -f 2)

if [[ daemon == "true" ]]; then
echo "DAEMON is enabled. Disable it before deploying Touitomamout with PM2. Please check your .env file."
exit 1
fi

# Handle the instance name
name=$(grep "TWITTER_HANDLE=" "$env" | cut -d '=' -f 2)

if [[ -z $name ]]; then
echo "No value found for TWITTER_HANDLE in $env. Please check your .env file."
exit 1
fi

# Handle the sync frequency
frequency=$(grep "SYNC_FREQUENCY_MIN=" "$env" | cut -d '=' -f 2)

if [[ -z $frequency ]]; then
echo "No value found for SYNC_FREQUENCY_MIN in $env. Using default value. Please check your .env file."
frequency=30
fi

# If called with --update flag, delete the instance before starting it again
if [[ $1 == "--update" ]]; then
pm2 del touitomamout-"$name"
fi

pm2 start "$touitomamout" --name touitomamout-"$name" --no-autorestart --cron "*/$frequency * * * *" -- "$env"
pm2 save
fi
done
13 changes: 13 additions & 0 deletions docker-compose.source.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
version: '3.9'

services:
touitomamout:
container_name: "touitomamout"
build:
context: ./ # ← This will build the image from the source code
restart: unless-stopped
environment:
- ENV_FILE=/data/.env
- STORAGE_DIR=/data
volumes:
- ./data:/data
6 changes: 3 additions & 3 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ version: '3.9'

services:
touitomamout:
build:
context: ./
container_name: "touitomamout"
image: louisgrasset/touitomamout:latest # Or "ghcr.io/louisgrasset/touitomamout:latest"
restart: unless-stopped
environment:
- ENV_FILE=/data/.env
- STORAGE_DIR=/data
- DAEMON_PERIOD_MIN=1
volumes:
- ./data:/data
16 changes: 0 additions & 16 deletions docs/docs/configuration/cron.md

This file was deleted.

41 changes: 41 additions & 0 deletions docs/docs/configuration/cron.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
sidebar_position: 3
title: Configure with Cron
tags: [
'configuration',
'cron',
]
---
import SectionPullInstallProject from "./sections/_section_pull_install_project.md"
import TipFileGenerator from "./sections/_tip-file-generator.md"

# ⏰ Cron configuration

## Installation
<SectionPullInstallProject/>

## Define environment variables
3 types of environment variables are required to run the sync:
- **Core variables**: required in situations, get them [here](/docs/resources/environment-variables#core-variables).
- **Sync variables**: required for the sync **you** want, get them [here](/docs/resources/environment-variables#synchronization-).
- **Cron variables**: required for the cron, get them [here](/docs/resources/environment-variables#configuration-with-cron-).

<TipFileGenerator />

Once your env variables are ready, inject them in a .env.{handle_to_sync} file. (e.g: `.env.signalapp`)

## Run it!

> You can create your own cron expression here: [crontab.guru](https://crontab.guru/).

First, enter the cron editor:
```bash
crontab -e
```

Then, add the following line to run the sync every 30 minutes:
```bash
*/30 * * * * cd /home/{user}/services/touitmamout && node ./dist/index.js .env
```

Finally, save and exit the editor.
16 changes: 0 additions & 16 deletions docs/docs/configuration/docker.md

This file was deleted.

71 changes: 71 additions & 0 deletions docs/docs/configuration/docker.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
sidebar_position: 2
title: Configure with Docker
tags: [
'configuration',
'docker',
]
---
import TipFileGenerator from "./sections/_tip-file-generator.md"

# 🐳 Docker configuration

## Define environment variables
3 types of environment variables are required to run the sync:
- **Core variables**: required in situations, get them [here](/docs/resources/environment-variables#core-variables)
- **Sync variables**: required for the sync **you** want, get them [here](/docs/resources/environment-variables#synchronization-)
- **Docker variables**: required for the docker container, get them [here](/docs/resources/environment-variables#configuration-with-docker-)

<TipFileGenerator />

## Create the touitomamout docker container
Two ways of creating it: either by using the image or by building it from the source code. The first one is a bit easier, but features will be the same in both cases.

---
📦 Releases are published on the following registries:
- [Docker Hub](https://hub.docker.com/r/louisgrasset/touitomamout) (`louisgrasset/touitomamout`)
- [Github (GHCR)](https://github.com/louisgrasset/touitomamout/pkgs/container/touitomamout) (`ghcr.io/louisgrasset/touitomamout`)

| Image tag | Description |
|-----------|----------------------------------------------------|
| `latest` | Corresponds to the latest release. |
| `dev` | Reflects the most recent changes on "main" branch. |
| `x.x.x` | A specific version. |

## From the image
```yml title="docker-compose.yml"
version: '3.9'

services:
touitomamout:
container_name: "touitomamout"
image: louisgrasset/touitomamout:latest # Or "ghcr.io/louisgrasset/touitomamout:latest"
restart: unless-stopped
environment:
- ENV_FILE=/data/.env
- STORAGE_DIR=/data
volumes:
- ./data:/data
```
## From the source code
```yml title="docker-compose.source.yml"
version: '3.9'

services:
touitomamout:
container_name: "touitomamout"
build:
context: ./ # ← This will build the image from the source code
restart: unless-stopped
environment:
- ENV_FILE=/data/.env
- STORAGE_DIR=/data
volumes:
- ./data:/data
```

## Run it!
Simply run this command to start the container.
```bash
docker-compose up -d
```
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,23 @@ tags: [
'manual sync',
]
---
import SectionPullInstallProject from "./sections/_section_pull_install_project.md"
import TipFileGenerator from "./sections/_tip-file-generator.md"

# 👏️️ Manual sync configuration

## Installation
<SectionPullInstallProject/>

## Define environment variables
3 types of environment variables are required to run the sync:
- **Core variables**: required in situations, get them [here](/docs/resources/environment-variables#core-variables).
- **Sync variables**: required for the sync **you** want, get them [here](/docs/resources/environment-variables#synchronization-).
- **Manual sync variables**: required for the manual sync, get them [here](/docs/resources/environment-variables#configuration-with-manual-sync-).

<TipFileGenerator />

## Run it!
```bash
node ./dist/index.js .env.{handle_to_sync}
```
Loading