Commit graph

282 commits

Author SHA1 Message Date
Adam Warner
5f2217a079
Fix for gravity.list not being emptied with an empty/missing adlist file.
Variable output depending on content of /existence of adlist

Signed-off-by: Adam Warner <adamw@rner.email>
2018-04-30 21:56:33 +01:00
Adam Warner
3216d65353
Merge branch 'development' into new/RemoveDefaultBlocklists 2018-04-29 13:39:17 +01:00
Adam Warner
9c29bd627e
fix user blacklisting 2018-04-21 23:55:33 +01:00
DL6ER
d24b141d8b
Get rid of numBlocked file, FTLDNS can now determine number of domains thanks to the new lists format
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-04-21 23:58:42 +02:00
DL6ER
bac9e005ae
Generate gravity.list and black.list in simple domain lists format for FTLDNS. Leave local.list in HOSTS format
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-04-21 23:27:36 +02:00
Dan Schaper
d4d1ec2568
Merge pull request #2124 from pi-hole/development
Internal
2018-04-17 11:54:50 -07:00
Adam Warner
88d4108f92
- Add new whipstail to choose from example blocklists on new install
- remove adlists.default
- do not copy non-existent adlists.default on run of gravity

Signed-off-by: Adam Warner <adamw@rner.email>
2018-04-17 10:33:06 +01:00
DL6ER
ff47ab12fb
Save number of effectively blocked domains in /etc/pihole/numBlocked
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-03-30 21:01:51 +02:00
Riley Avron
f390671018 Improve scheme matching to avoid inadvertent blocking
Signed-off-by: Riley Avron <riley.avron@gmail.com>
2018-03-06 22:03:56 -08:00
Riley Avron
512ec7fb87 Remove unneeded ifs in awk
Signed-off-by: Riley Avron <riley.avron@gmail.com>
2018-03-06 22:03:43 -08:00
Adam Warner
24219422df
Merge branch 'development' into FTLDNSremoveDnsmasq 2018-03-06 20:43:27 +00:00
Adam Warner
ad1f210528
As we no longer add source lists to the whitelist by default, gravity falls over on new installs.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-03-06 18:29:28 +00:00
Adam Warner
cadab61a21
Merge branch 'development' into FTLDNS 2018-03-05 21:33:53 +00:00
DL6ER
7ed45d26c2
Remove " " at some places where they shouldn't be
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-26 17:26:51 +01:00
DL6ER
e8ec0dc701
Use pihole-FTL resolver in pihole and gravity
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-25 09:34:04 +01:00
Adam Warner
6b4685b333
much more elegant to export LC_ALL at the beginning of the script!
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:33:29 +00:00
Adam Warner
d4ac818a0b
Specifty LC_ALL=C when dealing with sorted lists.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:23:07 +00:00
DL6ER
fc5034696d
Don't auto-whitelist blocking list domains
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-16 19:42:33 +01:00
Dan Schaper
9f2ce0e296
Merge pull request #1910 from pi-hole/fix/BinaryFileLineInGravity
Potential fix for #1909
2018-01-28 03:23:19 -08:00
Adam Warner
55e10d8287
replace another grep -F as pointed out by @StarPicard
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-28 00:50:15 +00:00
Adam Warner
f1956119fb
proposed gravity function renames
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-14 20:38:39 +00:00
Adam Warner
876170767c
Print actual domain instead of grep output of "Binary file /etc/pihole/list.preEventHorizon matches" when a domain with a unicode char is in a source list.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-13 16:43:20 +00:00
Adam Warner
2d3906450a
add back in str local var, and change ticks to info on whitelist lines
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-18 20:14:53 +00:00
Adam Warner
5ecc46d85b
Accidentally too many spaces
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 14:03:12 +00:00
Adam Warner
ddd6f3dbe5
Grammar changes. Lets not work out programmatically whether we need to pluralise a word. It's not needed.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:55:52 +00:00
Adam Warner
761fd13a70
Remove quotes from the uniqDomains array to actually whitelist the sources
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:15:26 +00:00
Adam Warner
c24e5cd3f6
:codacy: :codacy2: :codacy3:
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:45:25 +00:00
Adam Warner
cc3035267f
Revert parsing method to that of 3.1.4. New Method is allowing illegal lists through, and that is causing issues such as blocking www.google.com. This method will just throw away invalid list entries.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:36:09 +00:00
Jacob Salmela
80e17ab721
potentially fixes #1806 by falling back to dig if getent fails
Signed-off-by: Jacob Salmela <github@decoy.email>
2017-12-07 22:38:47 -06:00
Mcat12
8d721d086c
Modify indentation 2017-11-21 18:37:38 -05:00
DL6ER
6f0bb30def
Improve indentation
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 20:58:27 +01:00
DL6ER
c54f04ef4c
Repair username:password filtering
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 20:55:47 +01:00
DL6ER
eb83081a5c
Clarify error message: File could also have been of zero size
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:36:34 +01:00
DL6ER
8976930e20
Properly detect if local file was successfully downloaded
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:35:58 +01:00
DL6ER
164a81776e
Allow to add local lists to gravity using e.g. file:///path/to/my.list in adlists.list
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:30:40 +01:00
DL6ER
39ab1e1ea7
When using the Pi-hole DHCP server, local host names are called "something.local". Thir PR ensures that clients that are connected via VPN are augmented by a similar suffix like "someother.vpn".
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-09-30 13:03:49 +02:00
spacedingo
60365ad36a Update gravity.sh 2017-09-26 09:25:47 +04:00
spacedingo
466e6d9b30 Redo pull #1687
Fix for case sensitive duplicate domains
The change (https://github.com/pi-hole/pi-hole/pull/1687) was recently undone.
2017-09-26 05:53:07 +04:00
WaLLy3K
b9bcfe36ff Place errors on newline 2017-09-24 11:57:15 +10:00
WaLLy3K
cd7c00ef8e Fix Ctrl-C inadvertently deleting domains files 2017-09-23 10:32:56 +10:00
WaLLy3K
3aa525b0c0 Ensure domains files are not deleted upon w/blist 2017-09-22 14:17:56 +10:00
WaLLy3K
47099e2855 Ensure resolver test occurs each second
* Ensure that gravity will run the second the resolver is available
* Increase timeout to 120s
2017-09-22 03:56:53 +10:00
WaLLy3K
2deb2bf03f Fix broken whitelist functionality 2017-09-22 02:23:43 +10:00
WaLLy3K
a6f9272d4b Fix gravity from only parsing one adlist URL
* Redirect `grep` correctly to $sources (instead of using `awk`)
* Redirect $sourceDomains correctly
* Replace use of ${COL_LIGHT_BLUE}
* Add numeric count informing user of unique source domains being whitelisted
2017-09-20 22:25:33 +10:00
Adam Warner
8d8482d60b
<<<$() back to < <()
Signed-off-by: Adam Warner <adamw@rner.email>
2017-09-19 19:02:50 +01:00
WaLLy3K
d02bf258af Improve non-standard list parsing
* Add 504 status (Gateway connection timed out)
* Add text for non-standard list parsing
* Improve adblock parsing
* Ensure adblock exception rules are removed from file
* Ensure "www." is not treated as a URL-format list
* Corrected typo
* Ensure script does not fail if "-f" is used when there are no blocklists generated

Signed off by WaLLy3K <wally3k@pi-hole.net>
2017-09-18 17:36:03 +10:00
WaLLy3K
c957124fad Optimised parsing of domains on IPv6 servers
* Remove WHITELIST_COMMAND
* Place IPv4/IPv6 availability test underneath setupVars.conf source
* Improved clarity on comments
* Define default lookupDomain on local line
* Use `getent hosts` instead of nslookup (faster)
* Make gravity_DNSLookup() function more readable
* Use bold on "Neutrino emissions detected"
* Swap conditionals around on adlists file handling
* Add comments to both gravity_Collapse() `awk`s
* Removed unnecessary "${str}" from gravity_Pull()
* Merge function variables into local line
* Place .phgbp suffice on mktemp, so patternbuffers can be cleaned up all at once in gravity_Cleanup()
* Removed success="false" from $httpCode case, placed empty success var in local
* Reordered $httpCode case numerically because I can
* Provide error if Dnsmasq format list is being parsed
* Remove IPv4 check when determining URL list (too slow on large lists)
* Check ${#sources[@]} to ensure we're checking the number of entries and not the character count
* Define empty plural in local line, removing unnecessary plural=;
* Optimised readability of gravity_Whitelist()
* Removed uninformative "Nothing to blacklist"/"No wildcards used" text
* Optimised parsing of domains into hosts format on IPv6 enabled servers
* Ensure /etc/hostname is non-zero
* Use `: >` instead of `rm` as consistent with the rest of the script
* Ensured that gravity_Cleanup() removes ${localList}.tmp
* Optimised readability of gravity_ParseUserDomains()
* Moved dnsRestart to ${var} case statement, renaming it to dnsRestartType for readability
* Set default $listType to ensure script passes "bash strict mode"
2017-09-15 22:39:17 +10:00
WaLLy3K
ff5411a93a Add 'Connection Refused' for 000 status
* Shift default dnsRestart value into unset default parameter expansion value
* Change nslookup timeout to 5 seconds 
* Use &> instead of >
* Standardise plural code
* Update some comments
* Add "000" when connection is refused
* Condense adblock detection logic
* Add Dnsmasq format detection and parsing logic
* Removed unnecessary echo
* Add dnsWasOffline variable to ensure that if DNS service has been stopped, that it doesn't start and also get reloaded
2017-09-14 20:23:49 +10:00
WaLLy3K
d3073e5e23 Fix array of unique $sourceDomains 2017-09-14 17:09:52 +10:00
WaLLy3K
8191ec01e5 Gravity Bugfixes
* Merge development OpenVPN code
* Determine which domain to resolve depending on existence of $localList
* Re-add code to remove $localList, preventing duplicate local entries
* Minor shellcheck validation fix
2017-09-14 16:39:30 +10:00