Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pi-hole FTL v4.1 #408

Merged
merged 135 commits into from
Dec 9, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
f9c3423
Merge pull request #340 from pi-hole/release/v4.0
DL6ER Aug 6, 2018
177f756
Merge pull request #341 from pi-hole/hotfix/v4.0-version
DL6ER Aug 6, 2018
c30356c
Implement detection for when a local configuration is a blocking inst…
DL6ER Aug 9, 2018
5528505
Determine query type earlier on
DL6ER Aug 9, 2018
64a895d
Allow all query types (not only A and AAAA) during database import
DL6ER Aug 9, 2018
a3ac3bc
Be able to return any query type through API
DL6ER Aug 9, 2018
124f81e
Remove limitation to only analyze A and AAAA queries
DL6ER Aug 9, 2018
835ae44
Correct comment
DL6ER Aug 9, 2018
1edbd99
Also call FTL_cache(...) for the other query types (so far, we only c…
DL6ER Aug 9, 2018
2f4dd87
Remove more checks that skipped PTR replies
DL6ER Aug 9, 2018
111f160
Add new reply type: REPLY_DOMAIN
DL6ER Aug 9, 2018
1f360dc
Don't forget to GC wildcard blocked entries
DL6ER Aug 9, 2018
baeceeb
Extend FTL_reply(...). Meanwhile, I removed some code duplication and…
DL6ER Aug 9, 2018
877c81b
Add RRNAME reply type (needed for TXT queries)
DL6ER Aug 9, 2018
2f82a58
Merge pull request #348 from pi-hole/new/show_config_NXDOMAIN_as_blocked
DL6ER Aug 9, 2018
82f5324
Merge branch 'development' into new/more_query_types
DL6ER Aug 9, 2018
1a242be
Add "ANALYZE_ONLY_A_AND_AAAA" config option to obtain pre-v4.1 behavi…
DL6ER Aug 9, 2018
77f3eda
Show "regex.list" as source of regex blocked queries
DL6ER Aug 12, 2018
5979012
Merge pull request #350 from pi-hole/tweak/log_show_regex.list_as_source
DL6ER Aug 12, 2018
0d7c7e7
Merge branch 'development' into new/more_query_types
DL6ER Aug 12, 2018
25042de
Review comments
DL6ER Aug 12, 2018
9ac8980
Merge pull request #349 from pi-hole/new/more_query_types
DL6ER Aug 13, 2018
679f9d9
Determine blocking status enabled/disabled by reading the new variabl…
DL6ER Aug 13, 2018
c97f490
Fix tense
DL6ER Aug 13, 2018
716c978
BLOCKING -> BLOCKING_ENABLED
DL6ER Aug 13, 2018
a5a491b
Add query type forting for ">getallqueries" request
DL6ER Aug 15, 2018
2b27f8a
Implement forward destination filtering
DL6ER Aug 15, 2018
79b2865
Add new query type QUERY_EXTERNAL_BLOCKED triggered by an NXDOMAIN re…
DL6ER Aug 15, 2018
8eb10ef
Fix missing branch on tag builds
AzureMarker Aug 16, 2018
8c14a6f
Put tag builds in a folder named after the tag
AzureMarker Aug 16, 2018
c0bda36
Don't return any data if user requested an invalid query type in geta…
DL6ER Aug 16, 2018
c549041
Fix typo in comment
DL6ER Aug 16, 2018
5809d32
Merge pull request #352 from pi-hole/new/allqueries_forwarddest_query…
AzureMarker Aug 16, 2018
ffff591
Merge pull request #354 from pi-hole/fix/missing-branch-tag-build
AzureMarker Aug 16, 2018
51234de
Merge pull request #353 from pi-hole/new/quad9_nxdomain_adbit
DL6ER Aug 17, 2018
5fa70ec
Outsource code into query_externally_blocked() to avoid duplication
DL6ER Aug 17, 2018
a426b8d
Detect externally blocked state based on returned IP address (OpenDNS…
DL6ER Aug 17, 2018
d3351bf
Check string against NULL before passing to strcmp()
DL6ER Aug 17, 2018
2c5f151
Merge pull request #355 from pi-hole/new/OpenDNS_blocking_detection
DL6ER Aug 17, 2018
13deb5e
Use the pihole Docker namespace
AzureMarker Aug 18, 2018
57310f1
Merge pull request #356 from pi-hole/tweak/ci-image
AzureMarker Aug 18, 2018
dfe27e7
Add PRIVACY_NOSTATS mode that skips all (!) analysis. We furthermore …
DL6ER Aug 19, 2018
60ccf71
Merge pull request #357 from pi-hole/new/privacy_nostats
AzureMarker Aug 19, 2018
0e3fcc1
Add DBIMPORT config boolean that allows skipping the initial import o…
DL6ER Aug 20, 2018
e20cee4
Optimize ">getallqueries-domain"
DL6ER Aug 20, 2018
a725f03
Optimize ">getallqueries-clients"
DL6ER Aug 20, 2018
697f140
Add validate_access() guards
DL6ER Aug 20, 2018
1dca54c
Ensure validate_access() calls are also valid if no counters.{domains…
DL6ER Aug 20, 2018
78719ef
Merge pull request #358 from pi-hole/new/noDBimport
AzureMarker Aug 20, 2018
6aa7ac5
Fix indentation
DL6ER Aug 21, 2018
9be8054
Merge pull request #361 from pi-hole/tweak/optimize-allqueries
AzureMarker Aug 21, 2018
1936d60
Reduce code duplication by outsourcing queryID finder loop
DL6ER Aug 21, 2018
c94fe85
Merge pull request #362 from pi-hole/tweak/outsource_findQueryID
AzureMarker Aug 23, 2018
8712dc3
Ensure proper file ownerships if pihole-FTL is started as root. We dr…
DL6ER Aug 27, 2018
f67de18
Fix syntax and use short-circuit evaluation
DL6ER Aug 27, 2018
1958755
Fix usage of wrong variable
DL6ER Aug 29, 2018
42997d5
Merge pull request #366 from pi-hole/fix/file_permissions_as_root
AzureMarker Aug 29, 2018
1232cac
Merge pull request #368 from pi-hole/fix/counter_externally_blocked
DL6ER Aug 29, 2018
89c1db8
Merge pull request #351 from pi-hole/new/change_enabled_disabled
DL6ER Aug 30, 2018
2d3e91d
Apply patch: Fix crash parsing a --synth-domain with no prefix.
DL6ER Aug 31, 2018
0e04d64
Merge pull request #369 from pi-hole/fix/synth-domain
AzureMarker Aug 31, 2018
33a05db
Add guarding {} around if statement where I for got it previously
DL6ER Sep 2, 2018
007a047
Detect upstream returned NULL address as externally blocked query
DL6ER Sep 2, 2018
b90f6a0
Also compare against IP adresses if serving content from cache
DL6ER Sep 2, 2018
f4e62e4
Merge pull request #371 from pi-hole/fix/ref1035-compilation-warning
AzureMarker Sep 2, 2018
cc5f8b5
Add clarifying comment
DL6ER Sep 2, 2018
8d34229
Merge pull request #372 from pi-hole/new/detect_NULL_reply_as_ext_blo…
DL6ER Sep 2, 2018
c9e468b
Don't forget to reset "new" flag on client struct when name is resolved
AzureMarker Sep 3, 2018
85fcbdf
Merge pull request #373 from pi-hole/fix/client-name-new-flag
AzureMarker Sep 3, 2018
2c5462e
Don't count locally blocked queries as externally blocked
AzureMarker Sep 3, 2018
f7d076c
Ignore dnsmasq compiled files
AzureMarker Sep 3, 2018
061b19e
Merge pull request #374 from pi-hole/fix/gravity-blocked-externally-b…
AzureMarker Sep 3, 2018
c9cde35
Merge pull request #375 from pi-hole/tweak/dnsamsq-obj-gitignore
AzureMarker Sep 3, 2018
330abda
Remove privacy.c
AzureMarker Sep 3, 2018
c07807e
Merge pull request #376 from pi-hole/fix/prvacy-file-error
AzureMarker Sep 4, 2018
b433bee
Properly exit FTL on receipt of SIGTERM (save remaining queries to da…
DL6ER Sep 8, 2018
b02c9cd
Merge pull request #378 from pi-hole/fix/SIGTERM-action
AzureMarker Sep 9, 2018
6165dc6
Fix exit message showing invalid program run time
AzureMarker Sep 9, 2018
7700e95
Merge pull request #380 from pi-hole/fix/exit-timer
AzureMarker Sep 9, 2018
8c7d216
Update SQLite from 3.24.0 to 3.25.0
DL6ER Sep 16, 2018
a4eabb7
Merge pull request #381 from pi-hole/update/SQLite/3.25.0
AzureMarker Sep 16, 2018
af067b0
Always return data for blocklist and cache in ">forward-dest"
DL6ER Sep 25, 2018
3e0b580
Don't return -1 as number of read whitelist domains if there is no wh…
DL6ER Sep 26, 2018
446ad03
Merge pull request #387 from pi-hole/tweak/always-return-artificial-u…
DL6ER Sep 26, 2018
7a74acc
Merge pull request #388 from pi-hole/fix/no_whitelist
DL6ER Sep 26, 2018
d2d24ac
Kill the >kill command
AzureMarker Oct 4, 2018
cdcc178
Merge pull request #395 from pi-hole/tweak/kill-kill
AzureMarker Oct 5, 2018
515f14f
Ensure findOverTimeID() cannot return negative indices. Fixes #397
DL6ER Oct 13, 2018
059693a
Merge pull request #399 from pi-hole/fix/avoid_negative_timeidx
AzureMarker Oct 15, 2018
18b6788
update broken referral link
Oct 25, 2018
f2f3566
Move resolution of client and upstream host names to a dedicated thre…
DL6ER Oct 25, 2018
f10b49c
Remove obsolete debug output
DL6ER Oct 25, 2018
6880fbd
Merge pull request #403 from pi-hole/fix/referral-readme
AzureMarker Oct 25, 2018
5135663
Call to ">reresolve" also re-resolves host names of all known forward…
DL6ER Oct 26, 2018
560bca5
Two minor code optimizations
DL6ER Oct 27, 2018
dd77cc4
Merge pull request #406 from pi-hole/tweak/minor_optimizations
AzureMarker Oct 27, 2018
1a3a595
Use "DNS client" as custom thread name for the new name resolving thread
DL6ER Oct 27, 2018
fd53505
Rename routines/variables to reflect "DNS client"
DL6ER Oct 27, 2018
e6157da
Also re-resolve upstream host names every hour
DL6ER Oct 27, 2018
9f86037
Change logic of onlynew variable: If onlynew is set and an item is NO…
DL6ER Oct 27, 2018
7b01d20
Merge pull request #405 from pi-hole/new/resolve_thread
AzureMarker Oct 27, 2018
980729d
Modify the correct corresponding counters for user-defined dnsmasq co…
DL6ER Nov 4, 2018
b9e178c
Merge pull request #411 from pi-hole/fix/user-wildcards
AzureMarker Nov 4, 2018
92bb389
Fix interchanged indices in dnsmasq.h. This bug resulted in sometimes…
DL6ER Nov 5, 2018
e744e08
Merge pull request #413 from pi-hole/fix/record_source
AzureMarker Nov 5, 2018
7fc77c6
Search for readable config file in alternative location in case the d…
DL6ER Nov 7, 2018
d104d6b
Make log file path customizable
DL6ER Nov 7, 2018
c747023
Make PID file path customizable
DL6ER Nov 7, 2018
e68d82d
Add standardized subroutine for reading file path locations as this c…
DL6ER Nov 7, 2018
4240373
Make PORTFILE customizable
DL6ER Nov 7, 2018
f5d8adf
Make SOCKETFILE customizable
DL6ER Nov 7, 2018
32498b5
Remove files.log. It is not used anymore and a relict from pre-v4.0 t…
DL6ER Nov 7, 2018
ce95d08
Remove hard-coded location of preEventHorizon and 01-pihole.conf. The…
DL6ER Nov 7, 2018
ce10699
Use new customizable paths for Travis' tests. Fallback to searching f…
DL6ER Nov 7, 2018
2f8a67f
Make WHITELISTFILE customizable
DL6ER Nov 7, 2018
0a51cfa
Make BLACKLISTFILE customizable
DL6ER Nov 7, 2018
ee6cf89
Make GRAVITYFILE customizable
DL6ER Nov 7, 2018
5e17439
Make REGEXLISTFILE customizable
DL6ER Nov 7, 2018
f7a5ef0
Make SETUPVARSFILE customizable
DL6ER Nov 7, 2018
109861f
Make AUDITLISTFILE customizable
DL6ER Nov 7, 2018
ac3a8ab
Fix incorrect standard log path
DL6ER Nov 7, 2018
46771ff
Improve check in getLogFilePath()
DL6ER Nov 7, 2018
eeae592
Ensure regex validation is working as expected if privacylevel > 0. T…
DL6ER Nov 9, 2018
68181fb
Ensure privacy level can only be increased from starting pihole-FTL o…
DL6ER Nov 9, 2018
41e9a8a
Don't warn if privacy level is reduced. This generates too much outpu…
DL6ER Nov 9, 2018
8de3079
Review comments
DL6ER Nov 10, 2018
ece52c2
Rename conf2 to snapConf
DL6ER Nov 10, 2018
fa39906
Simplify two if expressions using De Morgan's Law
DL6ER Nov 10, 2018
efa86f9
Add documentation for getpath(...)
DL6ER Nov 10, 2018
ebbb3e4
Reimplement skipping of "0.0.0.0" in Top Client output
DL6ER Nov 10, 2018
2e25dda
Merge pull request #417 from pi-hole/fix/privacy_levels_regex_filtering
DL6ER Nov 10, 2018
f9ef0d7
Check return value of sscanf() explicitly (if one element was sucessf…
DL6ER Nov 10, 2018
39d47f0
Merge pull request #416 from pi-hole/new/adjustable_file_locations
DL6ER Nov 10, 2018
d3641e9
Revert blacklist file path to the original
AzureMarker Nov 21, 2018
1b85d49
Merge pull request #421 from pi-hole/fix/blacklist-default-path
DL6ER Nov 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: 2

.job_template: &job_template
docker:
- image: thepihole/ftl-build:$CIRCLE_JOB
- image: pihole/ftl-build:$CIRCLE_JOB
steps:
- checkout
- run:
Expand All @@ -12,17 +12,20 @@ version: 2
- run:
name: "Build"
command: |
make CFLAGS="${CFLAGS}" GIT_BRANCH="${CIRCLE_BRANCH}" GIT_TAG="${CIRCLE_TAG}"
BRANCH=$([ -z "$CIRCLE_TAG" ] && echo "$CIRCLE_BRANCH" || echo "master")

make CFLAGS="${CFLAGS}" GIT_BRANCH="${BRANCH}" GIT_TAG="${CIRCLE_TAG}"
file pihole-FTL
- run:
name: "Upload"
command: |
FOLDER=$([ -z "$CIRCLE_TAG" ] && echo "$CIRCLE_BRANCH" || echo "$CIRCLE_TAG")
mv pihole-FTL "${BIN_NAME}"
sha1sum pihole-FTL-* > ${BIN_NAME}.sha1
wget https://ftl.pi-hole.net:8080/FTL-client
chmod +x ./FTL-client
[[ "$CIRCLE_PR_NUMBER" == "" ]] && ./FTL-client "${CIRCLE_BRANCH}" "${BIN_NAME}" "${FTL_SECRET}"
[[ "$CIRCLE_PR_NUMBER" == "" ]] && ./FTL-client "${CIRCLE_BRANCH}" "${BIN_NAME}.sha1" "${FTL_SECRET}"
[[ "$CIRCLE_PR_NUMBER" == "" ]] && ./FTL-client "${FOLDER}" "${BIN_NAME}" "${FTL_SECRET}"
[[ "$CIRCLE_PR_NUMBER" == "" ]] && ./FTL-client "${FOLDER}" "${BIN_NAME}.sha1" "${FTL_SECRET}"
rm ./FTL-client
ls -lah .

Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Generated binary files
/obj/
/dnsmasq/obj
pihole-FTL
socket-test

Expand Down
43 changes: 25 additions & 18 deletions FTL.h
Original file line number Diff line number Diff line change
Expand Up @@ -82,34 +82,36 @@
enum { DATABASE_WRITE_TIMER, EXIT_TIMER, GC_TIMER, LISTS_TIMER, REGEX_TIMER };
enum { QUERIES, FORWARDED, CLIENTS, DOMAINS, OVERTIME, WILDCARD };
enum { DNSSEC_UNSPECIFIED, DNSSEC_SECURE, DNSSEC_INSECURE, DNSSEC_BOGUS, DNSSEC_ABANDONED, DNSSEC_UNKNOWN };
enum { QUERY_UNKNOWN, QUERY_GRAVITY, QUERY_FORWARDED, QUERY_CACHE, QUERY_WILDCARD, QUERY_BLACKLIST };
enum { QUERY_UNKNOWN, QUERY_GRAVITY, QUERY_FORWARDED, QUERY_CACHE, QUERY_WILDCARD, QUERY_BLACKLIST, QUERY_EXTERNAL_BLOCKED };
enum { TYPE_A = 1, TYPE_AAAA, TYPE_ANY, TYPE_SRV, TYPE_SOA, TYPE_PTR, TYPE_TXT, TYPE_MAX };
enum { REPLY_UNKNOWN, REPLY_NODATA, REPLY_NXDOMAIN, REPLY_CNAME, REPLY_IP };
enum { PRIVACY_SHOW_ALL = 0, PRIVACY_HIDE_DOMAINS, PRIVACY_HIDE_DOMAINS_CLIENTS, PRIVACY_MAXIMUM };
enum { REPLY_UNKNOWN, REPLY_NODATA, REPLY_NXDOMAIN, REPLY_CNAME, REPLY_IP, REPLY_DOMAIN, REPLY_RRNAME };
enum { PRIVACY_SHOW_ALL = 0, PRIVACY_HIDE_DOMAINS, PRIVACY_HIDE_DOMAINS_CLIENTS, PRIVACY_MAXIMUM, PRIVACY_NOSTATS };
enum { MODE_IP, MODE_NX, MODE_NULL, MODE_IP_NODATA_AAAA };
enum { REGEX_UNKNOWN, REGEX_BLOCKED, REGEX_NOTBLOCKED };
enum { BLOCKING_DISABLED, BLOCKING_ENABLED, BLOCKING_UNKNOWN };

// Privacy mode constants
#define HIDDEN_DOMAIN "hidden"
#define HIDDEN_CLIENT "0.0.0.0"

// Static structs
typedef struct {
const char* conf;
const char* log;
const char* pid;
const char* port;
const char* snapConf;
char* log;
char* pid;
char* port;
char* db;
const char* socketfile;
char* socketfile;
} FTLFileNamesStruct;

typedef struct {
const char* log;
const char* preEventHorizon;
const char* whitelist;
const char* blacklist;
const char* gravity;
const char* regexlist;
const char* setupVars;
const char* auditlist;
const char* dnsmasqconfig;
char* whitelist;
char* blacklist;
char* gravity;
char* regexlist;
char* setupVars;
char* auditlist;
} logFileNamesStruct;

typedef struct {
Expand All @@ -134,6 +136,7 @@ typedef struct {
int reply_NXDOMAIN;
int reply_CNAME;
int reply_IP;
int reply_domain;
} countersStruct;

typedef struct {
Expand All @@ -145,10 +148,12 @@ typedef struct {
int DBinterval;
int port;
int maxlogage;
int privacylevel;
unsigned char privacylevel;
bool ignore_localhost;
unsigned char blockingmode;
bool regex_debugmode;
bool analyze_only_A_AAAA;
bool DBimport;
} ConfigStruct;

// Dynamic structs
Expand All @@ -164,10 +169,11 @@ typedef struct {
bool db;
int id; // the ID is a (signed) int in dnsmasq, so no need for a long int here
bool complete;
bool private;
unsigned char privacylevel;
unsigned long response; // saved in units of 1/10 milliseconds (1 = 0.1ms, 2 = 0.2ms, 2500 = 250.0ms, etc.)
unsigned char reply;
unsigned char dnssec;
bool AD;
} queriesDataStruct;

typedef struct {
Expand Down Expand Up @@ -275,3 +281,4 @@ extern pthread_t telnet_listenthreadv6;
extern pthread_t socket_listenthread;
extern pthread_t DBthread;
extern pthread_t GCthread;
extern pthread_t DNSclientthread;
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ If you'd rather not [donate](https://pi-hole.net/donate/) (_which is okay!_), th

- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
- [Digital Ocean](http://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
- [UNIXstickers.com](http://unixstickers.refr.cc/jacobs) _save $5 when you spend $9 using our affiliate link_
- [Stickermule](https://www.stickermule.com/unlock?ref_id=6055890701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
- [Pi-hole Swag Store](https://pi-hole.net/shop/) _affiliate link_
- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
- [DNS Made Easy](https://cp.dnsmadeeasy.com/u/133706) _affiliate link_
Expand Down
Loading