Commit graph

268 commits

Author SHA1 Message Date
Adam Warner
ad1f210528
As we no longer add source lists to the whitelist by default, gravity falls over on new installs.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-03-06 18:29:28 +00:00
Adam Warner
6b4685b333
much more elegant to export LC_ALL at the beginning of the script!
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:33:29 +00:00
Adam Warner
d4ac818a0b
Specifty LC_ALL=C when dealing with sorted lists.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:23:07 +00:00
DL6ER
fc5034696d
Don't auto-whitelist blocking list domains
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-16 19:42:33 +01:00
Dan Schaper
9f2ce0e296
Merge pull request #1910 from pi-hole/fix/BinaryFileLineInGravity
Potential fix for #1909
2018-01-28 03:23:19 -08:00
Adam Warner
55e10d8287
replace another grep -F as pointed out by @StarPicard
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-28 00:50:15 +00:00
Adam Warner
f1956119fb
proposed gravity function renames
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-14 20:38:39 +00:00
Adam Warner
876170767c
Print actual domain instead of grep output of "Binary file /etc/pihole/list.preEventHorizon matches" when a domain with a unicode char is in a source list.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-13 16:43:20 +00:00
Adam Warner
2d3906450a
add back in str local var, and change ticks to info on whitelist lines
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-18 20:14:53 +00:00
Adam Warner
5ecc46d85b
Accidentally too many spaces
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 14:03:12 +00:00
Adam Warner
ddd6f3dbe5
Grammar changes. Lets not work out programmatically whether we need to pluralise a word. It's not needed.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:55:52 +00:00
Adam Warner
761fd13a70
Remove quotes from the uniqDomains array to actually whitelist the sources
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:15:26 +00:00
Adam Warner
c24e5cd3f6
:codacy: :codacy2: :codacy3:
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:45:25 +00:00
Adam Warner
cc3035267f
Revert parsing method to that of 3.1.4. New Method is allowing illegal lists through, and that is causing issues such as blocking www.google.com. This method will just throw away invalid list entries.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:36:09 +00:00
Jacob Salmela
80e17ab721
potentially fixes #1806 by falling back to dig if getent fails
Signed-off-by: Jacob Salmela <github@decoy.email>
2017-12-07 22:38:47 -06:00
Mcat12
8d721d086c
Modify indentation 2017-11-21 18:37:38 -05:00
DL6ER
6f0bb30def
Improve indentation
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 20:58:27 +01:00
DL6ER
c54f04ef4c
Repair username:password filtering
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 20:55:47 +01:00
DL6ER
eb83081a5c
Clarify error message: File could also have been of zero size
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:36:34 +01:00
DL6ER
8976930e20
Properly detect if local file was successfully downloaded
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:35:58 +01:00
DL6ER
164a81776e
Allow to add local lists to gravity using e.g. file:///path/to/my.list in adlists.list
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-11-21 18:30:40 +01:00
DL6ER
39ab1e1ea7
When using the Pi-hole DHCP server, local host names are called "something.local". Thir PR ensures that clients that are connected via VPN are augmented by a similar suffix like "someother.vpn".
Signed-off-by: DL6ER <dl6er@dl6er.de>
2017-09-30 13:03:49 +02:00
spacedingo
60365ad36a Update gravity.sh 2017-09-26 09:25:47 +04:00
spacedingo
466e6d9b30 Redo pull #1687
Fix for case sensitive duplicate domains
The change (https://github.com/pi-hole/pi-hole/pull/1687) was recently undone.
2017-09-26 05:53:07 +04:00
WaLLy3K
b9bcfe36ff Place errors on newline 2017-09-24 11:57:15 +10:00
WaLLy3K
cd7c00ef8e Fix Ctrl-C inadvertently deleting domains files 2017-09-23 10:32:56 +10:00
WaLLy3K
3aa525b0c0 Ensure domains files are not deleted upon w/blist 2017-09-22 14:17:56 +10:00
WaLLy3K
47099e2855 Ensure resolver test occurs each second
* Ensure that gravity will run the second the resolver is available
* Increase timeout to 120s
2017-09-22 03:56:53 +10:00
WaLLy3K
2deb2bf03f Fix broken whitelist functionality 2017-09-22 02:23:43 +10:00
WaLLy3K
a6f9272d4b Fix gravity from only parsing one adlist URL
* Redirect `grep` correctly to $sources (instead of using `awk`)
* Redirect $sourceDomains correctly
* Replace use of ${COL_LIGHT_BLUE}
* Add numeric count informing user of unique source domains being whitelisted
2017-09-20 22:25:33 +10:00
Adam Warner
8d8482d60b
<<<$() back to < <()
Signed-off-by: Adam Warner <adamw@rner.email>
2017-09-19 19:02:50 +01:00
WaLLy3K
d02bf258af Improve non-standard list parsing
* Add 504 status (Gateway connection timed out)
* Add text for non-standard list parsing
* Improve adblock parsing
* Ensure adblock exception rules are removed from file
* Ensure "www." is not treated as a URL-format list
* Corrected typo
* Ensure script does not fail if "-f" is used when there are no blocklists generated

Signed off by WaLLy3K <wally3k@pi-hole.net>
2017-09-18 17:36:03 +10:00
WaLLy3K
c957124fad Optimised parsing of domains on IPv6 servers
* Remove WHITELIST_COMMAND
* Place IPv4/IPv6 availability test underneath setupVars.conf source
* Improved clarity on comments
* Define default lookupDomain on local line
* Use `getent hosts` instead of nslookup (faster)
* Make gravity_DNSLookup() function more readable
* Use bold on "Neutrino emissions detected"
* Swap conditionals around on adlists file handling
* Add comments to both gravity_Collapse() `awk`s
* Removed unnecessary "${str}" from gravity_Pull()
* Merge function variables into local line
* Place .phgbp suffice on mktemp, so patternbuffers can be cleaned up all at once in gravity_Cleanup()
* Removed success="false" from $httpCode case, placed empty success var in local
* Reordered $httpCode case numerically because I can
* Provide error if Dnsmasq format list is being parsed
* Remove IPv4 check when determining URL list (too slow on large lists)
* Check ${#sources[@]} to ensure we're checking the number of entries and not the character count
* Define empty plural in local line, removing unnecessary plural=;
* Optimised readability of gravity_Whitelist()
* Removed uninformative "Nothing to blacklist"/"No wildcards used" text
* Optimised parsing of domains into hosts format on IPv6 enabled servers
* Ensure /etc/hostname is non-zero
* Use `: >` instead of `rm` as consistent with the rest of the script
* Ensured that gravity_Cleanup() removes ${localList}.tmp
* Optimised readability of gravity_ParseUserDomains()
* Moved dnsRestart to ${var} case statement, renaming it to dnsRestartType for readability
* Set default $listType to ensure script passes "bash strict mode"
2017-09-15 22:39:17 +10:00
WaLLy3K
ff5411a93a Add 'Connection Refused' for 000 status
* Shift default dnsRestart value into unset default parameter expansion value
* Change nslookup timeout to 5 seconds 
* Use &> instead of >
* Standardise plural code
* Update some comments
* Add "000" when connection is refused
* Condense adblock detection logic
* Add Dnsmasq format detection and parsing logic
* Removed unnecessary echo
* Add dnsWasOffline variable to ensure that if DNS service has been stopped, that it doesn't start and also get reloaded
2017-09-14 20:23:49 +10:00
WaLLy3K
d3073e5e23 Fix array of unique $sourceDomains 2017-09-14 17:09:52 +10:00
WaLLy3K
8191ec01e5 Gravity Bugfixes
* Merge development OpenVPN code
* Determine which domain to resolve depending on existence of $localList
* Re-add code to remove $localList, preventing duplicate local entries
* Minor shellcheck validation fix
2017-09-14 16:39:30 +10:00
WaLLy3K
0a00936e99 Update resolver test & added more comments
* Add/update code comments
* Change resolver check to test for pi.hole
* Make resolver check timeout after 10 seconds
* Use > instead of &> where appropriate
* Make resolver check sleep for 30 seconds (effectively waiting up to 50s for dnsmasq to be resolvable)
* Provide confirmation upon success of resolver check availability
* Quotes and Braced remaining variables as appropriate
* Removed duplicate local
2017-08-28 11:36:02 +10:00
WaLLy3K
f24ab8508e WIP cleanup
* Changed supernova/eventHorizon variables to match their purpose
* Add gravity_DNSLookup() function
* Ensure all comments are clear and relevant
* Use && instead of || in gravity_Collapse()
* Renamed existing functions, and placed them in order of script execution
* Use \t instead of literal tab in gravity_ParseFileIntoDomains()
* Replace instances of "truncate" with : > (e.g: gravity_Schwarzschild())
* Ensure correct variables are local'd
* Use phrase "Cleaning up stray matter" when gravity_Cleanup() is called
* Add black/white/wildcard switches for list.sh
* Ensure necessary functions are called when modifying black/white/wildcards

Signed-off-by: WaLLy3K <wally3k@pi-hole.net>
2017-07-27 12:34:35 +10:00
WaLLy3K
6830b08723 Clean up and optimise Gravity
* Shellcheck validation
* Made variable names, function names, comments and output clearer to understand
* Quoted and braced variables and conditionals
* Fix adlists.list handling logic, and remove any CR line endings
* Make CP/MV/RM provide user-friendly output upon failure
* Change adlists.list retrieval logic
* Moved and fixed adlists.list domain parsing logic
* Create gravity_ParseFileAsDomains() function to handle parsing of source files
* If no changes to a list is detected, print no output
* Ensure each source blocklist has a final newline
* Format number output as currency
* Make array of adlists domain sources unique to prevent redundant whitelisting
* Merged bash IPv4/IPv6 hosts formatting IF statement into an awk one-liner
* Trap Ctrl-C cancellations and run gravity_Cleanup()
* Use new gravity_Cleanup() function on errors and script completion
* Ensure that dnsmasq uses force-reload when gravity is invoked
* Add --wildcard option to ensure dnsmasq is restarted upon b/wlisting of a wildcard

Signed-off-by: WaLLy3K <wally3k@pi-hole.net>
2017-07-24 21:24:34 +10:00
Mcat12
1e31fa89aa Remove /* from IPv6 as well as IPv4
Fixes Discourse issue: https://discourse.pi-hole.net/t/ipv6-aaaa-dns-issue/3830
2017-06-26 09:51:21 -04:00
Adam Warner
536585b846 Colourise Core Output Text (#1471)
* Define colours within COL_TABLE
* Do not output colours for non-terminal instances
* Removed ":::"
* Fixed indenting & spacing
* Made output consistent throughout project
* Reworded text to fit on standard 80 char wide Terminal screen
* Made 'sudo raspi-config' warning (insufficient disk space) only show on RPi
* Make "Installation/Update Complete" the final msg
* Remove redundant messages
* Simplify update available message
* Confirm user would like to begin uninstall
* If "git pull" string says "Already up-to-date.", place [i] before it
* Colour Temp/Interface output
* Made `pihole disable 5z` invalid
* Added error fallback if invalid argument (not s/m) is detected
* Quoted "$2" for consistency
* Updated help text
* L185/286: Replaced echo with redirect
* User agents for adblock.mahakala.is/adaway.org unnecessary 
* Print newline on confirmation of repository reset
* Add output to admin-related dnsmasq restarts
* Return error message for "pihole -q"
* Imply default checkout behaviour with y/N
* Fix uninstall failing to remove pihole user
* Print checkout 'git remote show origin' STDERR on new line
* Replaced checkout "AdminLTE" wording with "Web Admin"
2017-06-21 21:49:05 +10:00
DL6ER
05798fe07a
cp + rm === mv (well, almost) 2017-06-19 20:22:03 +02:00
DL6ER
8bad56e897
Improve displayed messages and overall logic 2017-06-17 14:24:30 +02:00
DL6ER
92e691408f
Remove useless cat 2017-06-17 14:14:07 +02:00
DL6ER
0a9c234127
Add "pihole -g -b" to *only* update black.list (saves a bunch of time when adding/changing only blacklisted files - won'tdownload lal lists, but only processes the blacklist and restars dnsmasq) 2017-06-17 13:57:27 +02:00
DL6ER
0283a1ab74
Introduce new file black.list for blacklist content 2017-06-17 13:50:10 +02:00
Dan Schaper
6a2b885988
Update default User Agent to modern browser.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2017-04-27 11:41:08 -07:00
Mcat12
5e308dbd51 Delete the temp files created by gravity 2017-04-18 20:35:06 -04:00
Mcat12
a96b4d28e1 Merge pull request #1365 from pi-hole/adlistImprovements
Adlist improvements
2017-04-06 16:15:50 -04:00
Mcat12
ce86205df0 Use correct agent when downloading adaway.org list
See https://discourse.pi-hole.net/t/2278
2017-04-02 12:44:00 -04:00