Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help adding a filterlist (watchlist-internet.at) #1663

Closed
stonecrusher opened this issue Jun 5, 2020 · 8 comments
Closed

Help adding a filterlist (watchlist-internet.at) #1663

stonecrusher opened this issue Jun 5, 2020 · 8 comments
Labels
directory-data changes to basic FilterLists data question user question

Comments

@stonecrusher
Copy link
Contributor

Hi!
The austrian government has an official place where they collect fraudulent domains.
I made a scraping script in python to convert those to PiHole domain format and uBO recommended format (which might not be necessary - it's just so you are able to temporarly disable the filter).
I wanted to make it available for others by listing it on your site.

Despite the wiki I am very unsure what to do and which IDs to choose in the data model. The metadata itself should be quite obvious: German language, Domain-only format, I am the maintainer but origin is from watchlist-internet.at
Any idea what would be best way to proceed?
Greetings, Stonecrusher

@collinbarrett
Copy link
Owner

Hey, sure. Sorry that it is difficult to add at the moment unless you are familiar with how we do things. The easiest place to start is just reply to this Issue with all of the relevant data about your list. Start here. You can click here to see the existing data to get a feel for the data we have for other lists. We have a few folks who help out when they can taking new GitHub Issues and adding them to the site.

@collinbarrett collinbarrett added directory-data changes to basic FilterLists data question user question labels Jun 5, 2020
@DandelionSprout
Copy link
Contributor

DandelionSprout commented Jun 6, 2020

Normally, making use of the IDs that are in active use, here's normally how to do it:

  1. Go to https://github.com/collinbarrett/FilterLists/blob/master/data/FilterList.json, right-click on "View raw", and choose Download.
  2. Open the file in Sublime Text (or Notepad++ or something similar).
  3. Add a section near the bottom of the file and fill out "id", "description", "homeUrl", "issuesUrl", "licenseId", "name", "syntaxId" and/or "viewUrl" for the list; and save the file.
  4. Due to GitHub's online editor only supporting files up to 1MB, we sadly have to assign the list's language before actually properly adding the lists. Thus, go to https://github.com/collinbarrett/FilterLists/edit/master/data/FilterListLanguage.json.
  5. "listId" is the "id" from FilterList.json. "languageId" is from a randomly numbered list of languages at Language.json. German is 167, for the record.
  6. When you've saved FilterListLanguage.json, this should usually create a fork at https://github.com/stonecrusher/FilterLists/tree/master.
  7. Go to https://github.com/stonecrusher/FilterLists/tree/master/data and click "Upload", and upload FilterList.json that you edited a few minutes prior.
  8. Submit the pull request, and me and/or Collin will double-check it from there.

@stonecrusher
Copy link
Contributor Author

Thank you for your kind help guys, that's amazing.
It's much less complicated than I thought, just editing FilterListLanguage.json and FilterList.json; the other stuff is just for looking up IDs (or creating new, if necessary).
It might be worth linking this issue in the readme. The two links on the topic there are a bit overwhelming, although it's not hard to add a new list.

I made this PR: #1668

Maybe it would be appropriate to create a new tag as the list is specialized in fraudulent sites and fake shops? I don't think it really fits in "malware". Or just no tag as most other lists.

@DandelionSprout
Copy link
Contributor

For simplicity's sake, scam-specific lists are normally grouped under "phishing", though "malware" can also be used. I'll see if I can get this issue report added as a quick guide somewhere in the repo's instructions sometime this weekend.

@DandelionSprout
Copy link
Contributor

DandelionSprout commented Jun 6, 2020

Also, oh, I forgot to give instructions on how to add tags.

It's pretty much the same as steps 4-5 above, but with FilterListTag.json instead.

@stonecrusher
Copy link
Contributor Author

I added the "phishing" tag in the PR. That matches quite good.
Didn't have that keyword in my mind and must have overlooked it in the list. Thanks!

@collinbarrett
Copy link
Owner

Closed by #1668 . Thanks, @stonecrusher !

@collinbarrett
Copy link
Owner

And, thanks for helping as always, @DandelionSprout !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
directory-data changes to basic FilterLists data question user question
Projects
None yet
Development

No branches or pull requests

3 participants