Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DietPi-Software | Add Firefox Sync Server #3471

Merged
merged 9 commits into from
May 13, 2020
Merged

DietPi-Software | Add Firefox Sync Server #3471

merged 9 commits into from
May 13, 2020

Conversation

CedArctic
Copy link
Contributor

@CedArctic CedArctic commented Apr 16, 2020 •
edited
Loading

Status: Testing

  • Get installation and sync working

Commit list/description:

  • DietPi-Software | Added Firefox Sync Server by Mozilla
  • All other commits are fixes to the initial

Sources:
https://mozilla-services.readthedocs.io/en/latest/howtos/run-sync-1.5.html
https://github.com/mozilla-services/syncserver

Description:
This is Mozilla's Firefox Sync Server which manages syncing FF instance bookmarks, history, tabs and passwords across devices. Out of the box it runs on a Python server for small loads and can be configured to run behind nginx or Apache.

I've added an installation in dietpi-software to setup the basic python server (no nginx or apache yet, as LAN with only a handful of devices was the target). I'm currently running it on my Pi and managed to sync my devices. Please feel free to take a look and let me know on what can be improved.

Edit: Sync working on mobile devices as well

@CedArctic
Added Firefox Sync Server to Optimized Software
d785d64
@CedArctic
Fixes to Firefox Sync configuration and serving
87597ec
@CedArctic
Firefox Sync fixes
1695295 
- Fixed long description crashing dietpi-software
- Fixed if syntax error on 5320
- Fixed bash variable initialization syntax on 11233
- Enabled writing host IP to config
- Fixed service file
- Enabled service to start on boot
@CedArctic
Firefox Sync DB fixes
932e106 
Fixed ffsync user creation and directory ownership for the database
@CedArctic CedArctic changed the title Dev Add Firefox Sync Server Apr 16, 2020
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
MichaIng
MichaIng reviewed Apr 17, 2020
View reviewed changes
@MichaIng
Copy link
Owner

@CedArctic
Many thanks for your work! Very great implementation. I did a few suggestions, most importantly about the install dir, which should not be /etc. I know we use it in some other cases as well, ancient remain that we plan to change by times. Everything else pretty minor.

I'll see if I find time for some testing and merging before v6.29 release, else it will be the very first thing for v6.30!

@MichaIng MichaIng added this to the v6.29 milestone Apr 17, 2020
@CedArctic
Firefox Sync General Fixes
399add9 
- Changed install dir to /opt/firefox-sync
- Removed git updating logic
- Download_Install is used instead of git
- DietPi user data dir is not used as temporary storage during installation
- ffsync user home dir is set to /opt/firefox-sync
- Use database and config file are placed in dietpi_userdata/firefox-sync
- Added logic to modify the makefile to make it use the syncserver.ini config file located in dietpi_userdata/firefox-sync/syncserver.ini
@CedArctic
Copy link
Contributor Author

@MichaIng thank you for the swift and helpful response! It's a pleasure contributing to the project.
I've made changes according to your above feedback and even managed to move the config file out of the installation directory. Would be glad to hear your thoughts and further feedback.

In the mean time if you're testing it you can check out the Quickstart instructions here:
https://github.com/mozilla-services/syncserver
https://mozilla-services.readthedocs.io/en/latest/howtos/run-sync-1.5.html
Just a couple of notes:

  • When adding http://localhost:5000/token/1.0/sync/1.5 to Firefox, make sure to not ommit the token portion (the default URL doesn't have it and can be deceiving 😛 )
  • If sync is not working and the service logs print out something about a URL mismatch, try changing force_wsgi_environ to true in $G_FP_DIETPI_USERDATA/firefox-sync/syncserver.ini

@MichaIng MichaIng modified the milestones: v6.29, v6.30 May 1, 2020
@CedArctic
Copy link
Contributor Author

@MichaIng wanted to chime in with some feedback: I've tested FFSync extensively in the past few weeks running on my Pi2 and everything seems to be working as expected across all my devices (Win, Mac, Linux, Android)

@MichaIng MichaIng modified the milestones: v6.30, v6.31 May 10, 2020
@MichaIng MichaIng changed the title Add Firefox Sync Server DietPi-Software | Add Firefox Sync Server May 10, 2020
@MichaIng MichaIng closed this May 10, 2020
@MichaIng MichaIng reopened this May 10, 2020
@MichaIng MichaIng changed the base branch from dev to debug May 10, 2020 23:49
@MichaIng MichaIng changed the base branch from debug to dev May 10, 2020 23:49
MichaIng
MichaIng requested changes May 11, 2020
View reviewed changes
@MichaIng
Copy link
Owner

Btw ignore CodeFactor. I already fixed things in dev branch, so you could satisfy it by merging current dev branch. However I'll merge regardless of course.

@CedArctic
Merge pull request #1 from MichaIng/dev
ebf7987 
Merge latest dev changes
@CedArctic
Changes for Firefox Sync Server integration
7ea9e5c 
- Fixed dependencies list
- Emptied install dir if it existed before install
- Removed sudo where not needed
- Checked if database existed using -f rather than -a
- Modified config file only if a use modified one does not exist
- Used absolute path for make serve
- Switched ffsync user to its own user group
- Removed the user data directory on uninstall
@CedArctic
Copy link
Contributor Author

@MichaIng thanks for the feedback and guidance. I've reviewed your proposed changes and addressed them all except adding FFsync to DietPi-Services control which seems like a great idea, however I am not sure how to go about implementing it.

@MichaIng
Copy link
Owner

@CedArctic
Excellent, I'll run a final test tomorrow and then merge. I'll open a new PR to add DietPi-Services support afterwards.

@CedArctic
Copy link
Contributor Author

@MichaIng Sounds great! Let me know how the testing goes and if we need to enable force_wsgi_environ in the configuration file after all.

@MichaIng
Copy link
Owner

MichaIng commented May 13, 2020 •
edited
Loading

Testing:

G_CONFIG_INJECT 'DEV_GITOWNER=' 'DEV_GITOWNER=CedArctic' /boot/dietpi.txt
G_CONFIG_INJECT 'DEV_GITBRANCH=' 'DEV_GITBRANCH=dev' /boot/dietpi.txt
dietpi-update -1
dietpi-software install 177
  • Stretch
  • Buster

🈴 Bullseye does not have python(2)-virtualenv (yet), hence does not support Firefox Sync Server for now

MichaIng added 2 commits May 13, 2020 14:34
@MichaIng
v6.31
61e2e85 
+ DietPi-Software | Firefox Sync Server: Disable on Bullseye, since python(2)-virtualenv is not available there (yet)
+ DietPi-Software | Firefox Sync Server: Fix wrong archive directory name
+ DietPi-Software | Firefox Sync Server: Add documentation link to service file
+ DietPi-Software | Firefox Sync Server: Remove ffsync group as well on uninstall
+ DietPi-Software | Firefox Sync Server: Adding cosmetic/consistent tabs and empty lines
@MichaIng
v6.31
348ad58 
+ DietPi-Software | Firefox Sync Server: ffsync user is not added to dietpi group anymore, hence remove invalid "-a" option from usermod
@MichaIng
Copy link
Owner

MichaIng commented May 13, 2020 •
edited
Loading

@CedArctic
Okay, fixed mainly a wrong download directory name, now works fine, Bullseye needed to be disabled for now as above.

I wanted to test with Firefox, but I don't have a Mozilla account and authentication against Mozilla account server is still required.

Found out that one can run a Mozilla authentication server as well: https://github.com/mozilla/fxa
Respectively for authentication server only: https://github.com/mozilla/fxa/tree/master/packages/fxa-auth-server
which depends on: https://github.com/mozilla/fxa/tree/master/packages/fxa-auth-db-mysql

Seems to be a larger task, however at least I already found the Firefox settings to adjust this as well:

  • identity.fxaccounts.auth.uri
  • identity.fxaccounts.remote.oauth.uri (not sure if OAuth is included with above FxA auth server)

Worth it do start working on this as well?

Running a test on ARMv8 to cover pre-compiling...

@CedArctic
Copy link
Contributor Author

@MichaIng yes I was also considering taking a look into self-hosting an authentication server however I think I came across some info about difficulties when it came to running on ARM platforms. Definitely worth taking a look into all this and possibly adding the authentication server integration.

@MichaIng
Copy link
Owner

@CedArctic
Okay, lets make this an own issue/request. Test on ARMv8 went well, will merge.

@MichaIng MichaIng merged commit 89ae778 into MichaIng:dev May 13, 2020
@MichaIng MichaIng mentioned this pull request Jun 28, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MichaIng MichaIng MichaIng approved these changes

Assignees

@CedArctic CedArctic

Labels
Software request 💿
Projects
None yet
Milestone
v6.31
Development

Successfully merging this pull request may close these issues.
2 participants
@CedArctic @MichaIng