Commit graph

185 commits

Author SHA1 Message Date
Salmela
d646612a25 Block ads over IPv6
I’m pretty sure this works well.  Maybe someone else can try it out.
2015-12-31 20:26:05 -06:00
Jacob Salmela
26dcbfc1f9 hide minor error
An error shows up on the first install that `latentWhitelist.txt` doesn't exist and can't be removed.  Redirecting STDERR should fix this.
2015-12-27 12:25:35 -06:00
klaas01
a6313c9e9e Update gravity.sh
Fix kill command and only run as last step.
2015-12-26 14:37:51 -04:00
Jacob Salmela
c88982d0d5 chmod doesn't apply at initial install 2015-12-09 15:52:53 -06:00
Dan Schaper
d215fa9ade Resolve merge confict 2015-12-06 20:22:34 -08:00
Dan Schaper
cb91b8f5fa Add copyright headers 2015-12-06 05:55:50 -08:00
Dan Schaper
a230df0032 Fix errant ) 2015-12-06 05:33:22 -08:00
Dan Schaper
b391afef49 Interface pull for multiple platforms 2015-12-06 04:45:04 -08:00
Dan Schaper
d068593582 Get last IP for selected interface 2015-12-06 00:30:16 -08:00
Dan Schaper
d8034f4156 Get last IP listed for interface 2015-12-05 22:17:28 -08:00
Jacob Salmela
0c3409a028 fixes #111
Also clears out some errors users don't need to see
2015-12-04 21:41:37 -06:00
Jacob Salmela
3b9f7031d5 Merge pull request #96 from dschaper/feature/AtomicFunctions
Feature/atomic functions
2015-11-27 18:10:51 -06:00
Jacob Salmela
9fe531d1aa Merge pull request #94 from dschaper/bug/CleanupFiles
Bug Fix - Cleanup remnant files
2015-11-27 11:59:47 -06:00
Dan Schaper
9892d06c0d Code format cleanups 2015-11-26 15:56:37 -08:00
Dan Schaper
d1e475da89 Move black/white list check to pulsar 2015-11-26 15:48:52 -08:00
Dan Schaper
c45dc277b6 Tighten themeatics 2015-11-26 15:29:13 -08:00
Dan Schaper
29ae57465b Bug Fix - Cleanup Files 2015-11-25 19:51:07 -08:00
Dan Schaper
23856d65e2 IP address detection update 2015-11-25 16:35:36 -08:00
Cole Turner
03bbef07b5 pinholeIP should only use one IP
Updated piholeIP because if there are multiple network adapters, the gravity list will populate incorrectly with both IP addresses. Only one address should be used.
2015-11-25 13:36:39 -08:00
Dan Schaper
20399b8262 Build array of active domains to concatenate
Fix #86 Builds an array of active domains, and then loops through that list to pick files to concatenate after truncating Matter.
2015-11-23 13:12:11 -08:00
Dan Schaper
e2b518568e Forgot to start the directory check function 2015-11-23 12:39:47 -08:00
Dan Schaper
a5d29a1048 Fix function ordering
Unique is now called after the comments are scrubbed from the unified domain list.
2015-11-23 03:26:10 -08:00
Dan Schaper
4f1bb12d36 More functions
Splitting into more functions
2015-11-23 03:11:16 -08:00
Dan Schaper
af909e0ee6 Copyright issues
Some of the blocklists are copyright and need to be pulled and stored as is. They can be processed further in the script to remove comments.
2015-11-23 02:52:12 -08:00
Dan Schaper
fe46afd08d More functions
More refactoring
2015-11-23 01:47:24 -08:00
Dan Schaper
a1e609d005 Ugly hack permissions
Temporary hack for /etc/pihole directory and file permissions to pull out sudo requirement. (Irony is that is requires sudo.) This will be fixed later, but for now it's needed for existing installs so they don't break.
2015-11-23 01:16:00 -08:00
Dan Schaper
668d14e32d Add functions
Start to add functions for code reuse
2015-11-23 00:36:01 -08:00
Dan Schaper
f02c37ec07 Start code refactor, eliminate some repetitive code
Begin code refactor prepatation, prevent DRY where possible.
2015-11-22 23:49:38 -08:00
dschaper
689426180a Resolve merge conflicts 2015-11-15 13:59:51 +00:00
Dan Schaper
763aa6aa5c Change shebang to environment
Find bash in the environment, not hard coded to path
2015-11-11 00:43:00 -08:00
Jacob Salmela
a7d1b0b42c Revert "theme addition"
This reverts commit e1b8419a07.
2015-11-06 17:17:14 -06:00
Jacob Salmela
e1b8419a07 theme addition
Further adherence to the science/star trek theme
2015-11-06 17:16:14 -06:00
Jacob Salmela
932d410b66 anal retentive changes
OCD
2015-11-06 17:14:05 -06:00
Jacob Salmela
c297a86211 decluttering output
I love the adherence to the theme, but this line makes it difficult to
quickly read what is happening.
2015-11-06 17:10:17 -06:00
Jacob Salmela
1b65193fe4 new hostsfile url
This seems to be the more appropriate URL:
http://hosts-file.net/ad_servers.txt.

I compared it to the original URL using opendiff and they are exactly
the same.
2015-11-06 17:05:04 -06:00
Jacob Salmela
a06098bb16 remove filename from whitelist output
I also cleared the whitespace (or rather, Atom did).
I moved the pinhole.conf to it's current location
I commented out mahakala
2015-11-06 17:03:55 -06:00
Dan Schaper
15b9fb68d7 Remove filename from wc
Line 118 and Line 127 changed wc to remove filename from output.
2015-11-06 10:24:12 -08:00
Dan Schaper
642c31c361 Theme changes 2015-11-05 18:33:05 -08:00
Dan Schaper
8e21488542 hawson memory reductions 2015-11-05 18:22:17 -08:00
Dan Schaper
23eece9443 Upstream Branch pseudo-rebase 2015-11-05 18:11:34 -08:00
Jacob Salmela
a36352dfb6 Merge pull request #58 from Giermann/patch-1
add whitespace after $piholeIP
2015-11-04 21:02:58 -06:00
Jacob Salmela
4ad2b68ee1 reverting IPv6 so I can merge lighttpd.conf 2015-10-27 17:16:17 -05:00
Sven Giermann
d5ffec64f2 add whitespace after $piholeIP
In my case there had been no whitespace in piholeIP (I configured it statically because 'hostname -I' returns multiple IP addresses, which ends in errornous hosts file), so it's more safe to add one between IP and hostname.

Further, I removed the obsolete latentBlacklist variable.
2015-10-22 15:17:44 +02:00
Jacob Salmela
963eacfe05 commenting out mahakala list
Due to so many users having issues with this list blocking legitimate
domains such as microsoft.com, apple.com, xkcd.com and more, I am
turning it off by default.  While this drastically reduces the amount
of domains blocked, ad-blocking performance still seems to function
quite well.

Long-time users can simply uncomment the list, but this will make it so
new users have a more pleasant experience.

If you already have the mahakala list, you will need to remove it first
with this command:

sudo rm /etc/pihole/list.2.adblock.mahakala.is.domains

There are a few other lists I am looking at including, but this will
provide an immediate fix.
2015-10-11 09:28:11 -05:00
Jacob Salmela
ee40bc0423 forgot to redirect it to the file 2015-10-10 14:02:18 -05:00
Jacob Salmela
428d5e9a06 Revert "new command to print both the IPv4 and IPv6 address on their own line"
This reverts commit 88e6ce041d.
2015-10-10 14:01:10 -05:00
Jacob Salmela
88e6ce041d new command to print both the IPv4 and IPv6 address on their own line
I'm not 100% sure if you can add multiple IP addresses to one domain on
a single line, so I just broke it up into two.  It makes the file
bigger and longer, but I'll have to see how it works
2015-10-10 13:55:49 -05:00
Jacob Salmela
e3c0fdfeb7 new variables for IPv4 and IPv6
The IPv6 address looks for the one created from the MAC address.
2015-10-10 13:52:54 -05:00
Jacob Salmela
dfbf1a952f deleting tabs and spaces 2015-10-10 13:51:21 -05:00
Jacob Salmela
6976532ef5 Removing IP as input
Due to users having issues with the IP not being entered properly, I am restoring the original way to store `piholeIP` until a better solution can be formed.
2015-10-06 06:12:52 -05:00
Jacob Salmela
d52fdde2f8 using double bracket notation
Despite the script running successfully, it produced this error:

`/usr/local/bin/gravity.sh: line 8: [: missing `]'`

prior to this fix.
2015-10-03 11:06:50 -05:00
Jacob Salmela
533f707578 Merge pull request #45 from jofrep/IP-as-input-parameter
Allow external IP as an input script parameter
2015-10-03 11:05:52 -05:00
Jofre Palau
4ea397bc71 Removed extra blank space if no IP added 2015-09-28 17:40:13 +02:00
Jofre Palau
885069d440 Allow external IP as an input script parameter: ./gravity.sh 123.123.123.123 2015-09-18 23:06:29 +02:00
Jesse Becker
cfac2be334 toggle test case for noswap 2015-09-13 15:23:29 -04:00
Jesse Becker
4a4db7ab56 Allow for local settings to disable swap 2015-09-11 22:54:37 -04:00
Jacob Salmela
e19a6c3624 Merge pull request #38 from korhadris/master
Fixes #32 and fixes #35
2015-09-06 10:11:39 -05:00
jacobsalmela
fa77b7b69d increase swap to fix #37 memory error
This will increase the swap file to 500MB before downloading the lists.
 Most of the issue comes from the mahakala list, which is so large.  If
no swap file is found, one is created.
2015-08-25 18:01:54 -05:00
korhadris
98c94912e1 Replace use of grep -w with grep -x.
Prepend "^" to start of latentWhitelist.txt lines.

The -x switch requires a full line match of the regexp, where as -w
will try to find the match somewhere in the line, looking for work
breaks. Combined with turning the whitelist lines into full regexps,
this results in significantly faster parsing.

Having "^" prepended to the lines also keeps false whitelisting from
occuring, such as the following example:

If whitelist.txt contains "google.com" it would whitelist many other
sites that end in "google.com" as long as there is a non-word
character preceeding the google (such as "-", or ".").
2015-08-22 23:37:01 -07:00
korhadris
a26377d229 Append ad list sources to latentWhitelist.txt to prevent them from being filtered.
Additional fixes for #35. This will prevent our own sources from being
filtered out by competing source lists.
2015-08-22 21:44:41 -07:00
korhadris
e464c04490 Ignore domains in ad lists that do not contain . characters.
This will skip entries such as `localhost`, `android`, `debian` and
empty lines as listed in #35.
2015-08-22 17:47:22 -07:00
korhadris
bb7db11214 Changing printouts when updating sources to tell what is going on when
manually running gravity.sh

This will print "Getting $domain list... " for each domain, followed
by either "Done" if data was received and validated, or "Skipping
list because it does not have any new entries" if no updates were
needed.
2015-08-22 17:33:30 -07:00
korhadris
1f29d01694 Remove leading and trailing whitespace and . characters and
duplicate `.` characters as each list is stored.

Should fix #32.
2015-08-22 17:05:19 -07:00
korhadris
d6d192cb0a Use url variable to store ${sources[$i]} value to improve readability.
I also wanted to replace the for loop iterating over indices with
something like:

`for url in $sources[@]}`

It made the use of `$i` in the save location more annoying though.
2015-08-22 16:22:07 -07:00
korhadris
0ec6eab683 Appending ".$justDomainsExtension" to $saveLocation variable.
Every use of $saveLocation was adding this and making lines
longer.
2015-08-22 16:04:54 -07:00
korhadris
159b29b80b Replace spaces with tabs to make indentation consistent within the file. 2015-08-22 15:56:32 -07:00
Fourdee
9d99a4ef36 Patch 3 - Dont use /etc/hosts
/etc/pihole/gravity.list now stores the block list. Ensures the
/etc/hosts file is left untouched.
2015-07-30 17:24:24 +01:00
jacobsalmela
563db80b6d resolves #25
Will not count blank lines if they happen to exist.
2015-07-17 20:49:03 -05:00
jacobsalmela
8f961c1aaa resolves #22
This lets dnsmasq re-read the hosts file without disturbing the daemon.
 So any new entries can be used as soon as gravity.sh is finished
running
2015-07-17 13:05:38 -05:00
rmceoin
f6ccb4b658 Merge remote-tracking branch 'upstream/master' 2015-07-13 09:55:53 -07:00
rmceoin
37e926ce84 Parses host only file formats now. Previously only handled hosts file format.
Specifically, it can now handle the following list:
'http://mirror1.malwaredomains.com/files/justdomains'
2015-07-13 09:28:45 -07:00
jacobsalmela
5c4bfb84b0 uses a variable for hostname instead of raspberrypi
Some people use a hostname other than raspberrypi, so their hostname
did not resolve to 127.0.0.1.  I replaced that hardcoded value with a
variable so that does not happen.

I also added a few comments and minor formatting adjustments,.
2015-07-13 06:59:22 -05:00
rmceoin
552f980430 blacklist was being concatenated with wrong matter 2015-06-22 13:33:02 -07:00
rmceoin
66bb0e7bb3 Strip carriage returns on matter so that whitelists work correctly. Lines that had \r would not match. 2015-06-22 13:03:15 -07:00
rmceoin
e9324f8316 Use double brackets for the test. 2015-06-19 17:54:12 -07:00
rmceoin
67aba8c496 If exists, import a config file to allow for overriding script variables. 2015-06-19 13:31:51 -07:00
jacobsalmela
01ac3c1dd3 Ditching the use of the loopback
Pushing files so they are available when the new article gets posted.

If the Pi's loopback is set in the hosts file, clients using it as a
DNS server will try to connect to their own loopback, which does not
have a Web server.  So the real IP of the Pi is used.  It is
recommended to use a static IP since this will be acting as a server.

Made one small change from some hard coded values to a variable.
2015-06-13 22:01:12 -05:00
jacobsalmela
c563841714 changing the origin folder
Originally, I had this set to /run/shm (in RAM) but ran into errors
when the list reached 900,000 entries.
Then I moved it to /tmp.
Finally, I decided to just put the files in the pihole dir so they are
available after reboots.  This will help with only downloading the
lists when absolutely needed--respecting the bandwidth of the people
serving the lists.

It is also possible to add addn-hosts=/path/to/hosts.conf within the
dnsmasq.conf file if you don't want to use hosts.  For simplicity and
speed, I just use the regular hosts file.
2015-06-06 23:34:32 -05:00
jacobsalmela
457b70f5c5 add IPv6 support in the hosts file
Still need to get lighted to use IPv6.  I am doing this because some
ads can get through using IPv6 if the IPv4 version is blocked.  Also,
it seems to work fine as far as performance even though it doubles the
file size...

Also added a few comments for better documentation.
2015-06-04 08:21:44 -05:00
jacobsalmela
61c99ff145 forgot to change origin dir 2015-05-19 13:32:37 -05:00
jacobsalmela
56c776af22 hosts format script 2015-05-19 13:31:37 -05:00
jacobsalmela
18d6f4b747 initial commit 2014-09-26 09:28:44 -05:00
Jacob Salmela
8569c419c1 added file path 2014-06-10 20:22:26 -05:00
Jacob Salmela
2af19a0194 added cmd to restart dns 2014-06-08 10:14:54 -05:00
Jacob Salmela
2131149fda Create gravity.sh 2014-06-08 10:03:56 -05:00