diff --git a/README.md b/README.md index be0d4369..05bb47ba 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,84 @@ -# Raspberry Pi Ad Blocker -**A black hole for ads, hence Pi-hole** - -![Pi-hole](http://i.imgur.com/wd5ltCU.png) - -The Pi-hole is a DNS/Web server that will **block ads for any device on your network**. - -## Coverage -Featured on [MakeUseOf](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/) and [Lifehacker](http://lifehacker.com/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-co-1686093533)! - -## Automated Install +# Automated Install +##### Designed For Raspberry Pi B, B+, 2, and Zero (with an Ethernet adapter) 1. Install Raspbian -2. Set a **static** IP address -3. Run the command below +2. Run the command below -```curl -s "https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/automated%20install/basic-install.sh" | bash``` +### ```curl -L install.pi-hole.net | bash``` -Once installed, **configure any device to use the Raspberry Pi as its DNS server and the ads will be blocked**. You can also configure your router's DHCP options to assign the Pi as clients DNS server so they do not need to do it manually. +Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](http://pi-hole.net/faq/can-i-set-the-pi-hole-to-be-the-dns-server-at-my-router-so-i-dont-have-to-change-settings-for-my-devices/) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to [use the Raspberry Pi as its DNS server](http://pi-hole.net/faq/how-do-i-use-the-pi-hole-as-my-dns-server/). + +## Pi-hole Is Free, But Powered By Your Donations +[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "Free, but powered by donations")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate") + +## How Does It Work? +**Watch the 60-second video below to get a quick overview** + +[![Pi-hole exlplained](http://i.imgur.com/qNybJDX.png)](https://vimeo.com/135965232) + +## Pi-hole Projects +- [Go Bananas for CHiP-hole ad blocking](https://www.hackster.io/jacobsalmela/chip-hole-network-wide-ad-blocker-98e037) +- [Sky-Hole](http://dlaa.me/blog/post/skyhole) +- [Pi-hole in the Cloud!](http://blog.codybunch.com/2015/07/28/Pi-Hole-in-the-cloud/) +- [unRaid-hole](https://github.com/spants/unraidtemplates/blob/master/Spants/unRaid-hole.xml#L13)--[Repo and more info](http://lime-technology.com/forum/index.php?PHPSESSID=c0eae3e5ef7e521f7866034a3336489d&topic=38486.0) +- [Pi-hole on/off button](http://thetimmy.silvernight.org/pages/endisbutton/) +- [Minibian Pi-hole](http://munkjensen.net/wiki/index.php/See_my_Pi-Hole#Minibian_Pi-hole) + +## Coverage +- [MacObserver Podcast 585](http://www.macobserver.com/tmo/podcast/macgeekgab-585) +- [Medium: Block All Ads For $53](https://medium.com/@robleathern/block-ads-on-all-home-devices-for-53-18-a5f1ec139693#.gj1xpgr5d) +- [MakeUseOf: Adblock Everywhere, The Pi-hole Way](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/) +- [Lifehacker: Turn Your Pi Into An Ad Blocker With A Single Command](http://lifehacker.com/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-co-1686093533)! +- [Pi-hole on TekThing](https://youtu.be/8Co59HU2gY0?t=2m) +- [Pi-hole on Security Now! Podcast](http://www.youtube.com/watch?v=p7-osq_y8i8&t=100m26s) +- [Foolish Tech Show](https://youtu.be/bYyena0I9yc?t=2m4s) +- [Pi-hole on Ubuntu](http://www.boyter.org/2015/12/pi-hole-ubuntu-14-04/) +- [Catchpoint: iOS 9 Ad Blocking](http://blog.catchpoint.com/2015/09/14/ad-blocking-apple/) + +## Partnering With Optimal.com + +Pi-hole will be teaming up with [Rob Leathern's subscription service to avoid ads](https://medium.com/@robleathern/block-ads-on-all-home-devices-for-53-18-a5f1ec139693#.gj1xpgr5d). This service is unique and will help content-creators and publishers [still make money from visitors who are using an ad ablocker](http://techcrunch.com/2015/12/17/the-new-optimal/). + +## Technical Details + +The Pi-hole is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement. You can also [replace ads with any image you want](http://pi-hole.net/faq/is-it-possible-to-change-the-blank-page-that-takes-place-of-the-ads-to-something-else/) since it is just a simple Webpage taking place of the ads. A more detailed explanation of the installation can be found [here](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0). -[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "AdminLTE Presentation")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate") - ## Gravity -The [gravity.sh](https://github.com/jacobsalmela/pi-hole/blob/master/gravity.sh) does most of the magic. The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0). +The [gravity.sh](https://github.com/jacobsalmela/pi-hole/blob/master/gravity.sh) does most of the magic. The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0) (if you decide to use the [mahakala list](https://github.com/jacobsalmela/pi-hole/commit/963eacfe0537a7abddf30441c754c67ca1e40965)). ## Whitelist and blacklist -You can add a `whitelist.txt` or `blacklist.txt` in `/etc/pihole/` and the script will apply those files automatically. +You can add a `whitelist.txt` or `blacklist.txt` in `/etc/pihole/` and the script will apply those files automatically. Put one domain per line. ## Web Interface -I am also working on a [Web interface](https://github.com/jacobsalmela/AdminLTE#pi-hole-admin-dashboard) so you can view stats and change settings. +The [Web interface](https://github.com/jacobsalmela/AdminLTE#pi-hole-admin-dashboard) will be installed automatically so you can view stats and change settings. You can find it at: -## Custom Config File -If you want to use your own variables for the gravity script (i.e. storing the files in a different location) and don't want to have to change them every time there is an update to the script, create a file called `/etc/pihole/pihole.conf`. In it, you should add your own variables in a similar fashion as shown below: +`http://192.168.1.x/admin/index.php` -``` -origin=/var/run/pihole -adList=/etc/dnsmasq.d/adList +### API + +A basic read-only API can be accessed at `/admin/api.php`. It returns the following JSON: +```JSON +{ + "domains_being_blocked": "136708", + "dns_queries_today": "18108", + "ads_blocked_today": "14648", + "ads_percentage_today": 80.892423238348 +} ``` -See [this PR](https://github.com/jacobsalmela/pi-hole/pull/20) for more details. +![Web](http://i.imgur.com/m114SCn.png) -### How It Works -A technical and detailed description can be found [here](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0)! +## Real-time Statistics + +You can view [real-time stats](http://pi-hole.net/faq/install-the-real-time-lcd-monitor-chronometer/) via `ssh` or on an [2.8" LCD screen](http://amzn.to/1P0q1Fj). This is accomplished via [`chronometer.sh`](https://github.com/jacobsalmela/pi-hole/blob/master/advanced/Scripts/chronometer.sh). + +![Pi-hole LCD](http://i.imgur.com/nBEqycp.jpg) + +## Help +- See the [Wiki](https://github.com/jacobsalmela/pi-hole/wiki/Customization) entry for more details +- There is also an [FAQ section on pi-hole.net](http://pi-hole.net) ## Other Operating Systems This script will work for other UNIX-like systems with some slight **modifications**. As long as you can install `dnsmasq` and a Webserver, it should work OK. The automated install only works for a clean install of Raspiban right now since that is how the project originated. - -[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "AdminLTE Presentation")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate") diff --git a/advanced/Scripts/blacklist.sh b/advanced/Scripts/blacklist.sh new file mode 100644 index 00000000..11c3852c --- /dev/null +++ b/advanced/Scripts/blacklist.sh @@ -0,0 +1,182 @@ +#!/usr/bin/env bash +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. + +if [[ $# = 0 ]]; then + echo "Immediately blacklists one or more domains in the hosts file" + echo " " + echo "Usage: blacklist.sh domain1 [domain2 ...]" + echo " " + echo "Options:" + echo " -d, --delmode Remove domains from the blacklist" + echo " -nr, --noreload Update blacklist without refreshing dnsmasq" + echo " -f, --force Force updating of the hosts files, even if there are no changes" + echo " -q, --quiet output is less verbose" + exit 1 +fi + +#globals +blacklist=/etc/pihole/blacklist.txt +adList=/etc/pihole/gravity.list +reload=true +addmode=true +force=false +versbose=true +domList=() +domToRemoveList=() + + +piholeIPfile=/tmp/piholeIP +piholeIPv6file=/etc/pihole/.useIPv6 + +# Otherwise, the IP address can be taken directly from the machine, which will happen when the script is run by the user and not the installation script +IPv4dev=$(ip route get 8.8.8.8 | awk '{for(i=1;i<=NF;i++)if($i~/dev/)print $(i+1)}') +piholeIPCIDR=$(ip -o -f inet addr show dev $IPv4dev | awk '{print $4}' | awk 'END {print}') +piholeIP=${piholeIPCIDR%/*} + +modifyHost=false + + +if [[ -f $piholeIPv6file ]];then + # If the file exists, then the user previously chose to use IPv6 in the automated installer + piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }') +fi + + +function HandleOther(){ + #check validity of domain + validDomain=$(echo $1 | perl -ne'print if /\b((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,63}\b/') + + if [ -z "$validDomain" ]; then + echo $1 is not a valid argument or domain name + else + domList=("${domList[@]}" $validDomain) + fi +} + +function PopBlacklistFile(){ + #check blacklist file exists, and if not, create it + if [[ ! -f $blacklist ]];then + touch $blacklist + fi + for dom in "${domList[@]}" + do + if $addmode; then + AddDomain $dom + else + RemoveDomain $dom + fi + done +} + +function AddDomain(){ +#| sed 's/\./\\./g' + bool=false + grep -Ex -q "$1" $blacklist || bool=true + if $bool; then + #domain not found in the blacklist file, add it! + if $versbose; then + echo "** Adding $1 to blacklist file" + fi + echo $1 >> $blacklist + modifyHost=true + else + if $versbose; then + echo "** $1 already blacklisted! No need to add" + fi + fi +} + +function RemoveDomain(){ + + bool=false + grep -Ex -q "$1" $blacklist || bool=true + if $bool; then + #Domain is not in the blacklist file, no need to Remove + if $versbose; then + echo "** $1 is NOT blacklisted! No need to remove" + fi + else + #Domain is in the blacklist file, add to a temporary array + if $versbose; then + echo "** Un-blacklisting $dom..." + fi + domToRemoveList=("${domToRemoveList[@]}" $1) + modifyHost=true + fi +} + +function ModifyHostFile(){ + if $addmode; then + #add domains to the hosts file + if [[ -r $blacklist ]];then + numberOf=$(cat $blacklist | sed '/^\s*$/d' | wc -l) + plural=; [[ "$numberOf" != "1" ]] && plural=s + echo "** blacklisting a total of $numberOf domain${plural}..." + if [[ -n $piholeIPv6 ]];then + cat $blacklist | awk -v ipv4addr="$piholeIP" -v ipv6addr="$piholeIPv6" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' >> $adList + else + cat $blacklist | awk -v ipv4addr="$piholeIP" '{sub(/\r$/,""); print ipv4addr" "$0}' >>$adList + fi + + fi + else + + for dom in "${domToRemoveList[@]}" + do + #we need to remove the domains from the blacklist file and the host file + echo $dom | sed 's/\./\\./g' | xargs -I {} perl -i -ne'print unless /[^.]'{}'(?!.)/;' $adList + echo $dom | sed 's/\./\\./g' | xargs -I {} perl -i -ne'print unless /'{}'(?!.)/;' $blacklist + done + fi + +} + +function Reload() { + # Reload hosts file + echo "** Refresh lists in dnsmasq..." + + dnsmasqPid=$(pidof dnsmasq) + + if [[ $dnsmasqPid ]]; then + # service already running - reload config + sudo kill -HUP $dnsmasqPid + else + # service not running, start it up + sudo service dnsmasq start + fi +} + +################################################### + +for var in "$@" +do + case "$var" in + "-nr"| "--noreload" ) reload=false;; + "-d" | "--delmode" ) addmode=false;; + "-f" | "--force" ) force=true;; + "-q" | "--quiet" ) versbose=false;; + * ) HandleOther $var;; + esac +done + +PopBlacklistFile + +if $modifyHost || $force; then + echo "** Modifying Hosts File" + ModifyHostFile +else + if $versbose; then + echo "** No changes need to be made" + fi + exit 1 +fi + +if $reload; then + Reload +fi diff --git a/advanced/Scripts/chronometer.sh b/advanced/Scripts/chronometer.sh index 73219760..b29480a9 100755 --- a/advanced/Scripts/chronometer.sh +++ b/advanced/Scripts/chronometer.sh @@ -1,28 +1,135 @@ -#!/bin/bash -# Displays Pi-hole stats on the Adafruit PiTFT 2.8" touch screen -# Set the pi user to log in automatically and run this script from /etc/profile -for (( ; ; )) +#!/usr/bin/env bash +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. + + +#Functions############################################################################################################## +piLog="/var/log/pihole.log" +gravity="/etc/pihole/gravity.list" + +today=$(date "+%b %e") + +function CalcBlockedDomains(){ + if [ -e "$gravity" ]; then + #Are we IPV6 or IPV4? + if [[ -n $piholeIPv6 ]];then + #We are IPV6 + blockedDomainsTotal=$(wc -l /etc/pihole/gravity.list | awk '{print $1/2}') + else + #We are IPV4 + blockedDomainsTotal=$(wc -l /etc/pihole/gravity.list | awk '{print $1}') + fi + else + blockedDomainsTotal="Err." + fi +} + +function CalcQueriesToday(){ + if [ -e "$piLog" ];then + queriesToday=$(cat "$piLog" | grep "$today" | awk '/query/ {print $6}' | wc -l) + else + queriesToday="Err." + fi +} + +function CalcblockedToday(){ + if [ -e "$piLog" ] && [ -e "$gravity" ];then + blockedToday=$(cat $piLog | awk '/\/etc\/pihole\/gravity.list/ && !/address/ {print $6}' | wc -l) + else + blockedToday="Err." + fi +} + +function CalcPercentBlockedToday(){ + if [ "$queriesToday" != "Err." ] && [ "$blockedToday" != "Err." ]; then + #scale 2 rounds the number down, so we'll do scale 4 and then trim the last 2 zeros + percentBlockedToday=$(echo "scale=4; $blockedToday/$queriesToday*100" | bc) + percentBlockedToday=$(sed 's/.\{2\}$//' <<< "$percentBlockedToday") + fi +} + +function CheckIPv6(){ + piholeIPv6file="/etc/pihole/.useIPv6" + if [[ -f $piholeIPv6file ]];then + # If the file exists, then the user previously chose to use IPv6 in the automated installer + piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }') + fi +} + +function outputJSON(){ + CalcQueriesToday + CalcblockedToday + CalcPercentBlockedToday + + CheckIPv6 + CalcBlockedDomains + + printf '{"domains_being_blocked":"%s","dns_queries_today":"%s","ads_blocked_today":"%s","ads_percentage_today":"%s"}\n' "$blockedDomainsTotal" "$queriesToday" "$blockedToday" "$percentBlockedToday" +} + +function normalChrono(){ + for (( ; ; )) + do + clear + # Displays a colorful Pi-hole logo + toilet -f small -F gay Pi-hole + echo " $(ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -d':' -f2)" + echo "" + uptime | cut -d' ' -f11- + echo "-------------------------------" + # Uncomment to continually read the log file and display the current domain being blocked + #tail -f /var/log/pihole.log | awk '/\/etc\/pihole\/gravity.list/ {if ($7 != "address" && $7 != "name" && $7 != "/etc/pihole/gravity.list") print $7; else;}' + + #uncomment next 4 lines to use original query count calculation + #today=$(date "+%b %e") + #todaysQueryCount=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ {print $7}' | wc -l) + #todaysQueryCountV4=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[A\]/ {print $7}' | wc -l) + #todaysQueryCountV6=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[AAAA\]/ {print $7}' | wc -l) + + + CalcQueriesToday + CalcblockedToday + CalcPercentBlockedToday + + CalcBlockedDomains + + echo "Blocking: $blockedDomainsTotal" + #below commented line does not add up to todaysQueryCount + #echo "Queries: $todaysQueryCountV4 / $todaysQueryCountV6" + echo "Queries: $queriesToday" #same total calculation as dashboard + echo "Pi-holed: $blockedToday ($percentBlockedToday%)" + + sleep 5 + done +} + +function displayHelp(){ + echo "Displays stats about your piHole!" + echo " " + echo "Usage: chronometer.sh [optional:-j]" + echo "Note: If no option is passed, then stats are displayed on screen, updated every 5 seconds" + echo " " + echo "Options:" + echo " -j, --json output stats as JSON formatted string" + echo " -h, --help display this help text" + + exit 1 +} + +if [[ $# = 0 ]]; then + normalChrono +fi + +for var in "$@" do - clear - # Displays a colorful Pi-hole logo - toilet -f small -F gay Pi-hole - echo " $(ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -d':' -f2)" - echo "" - uptime | cut -d' ' -f11- - echo "-------------------------------" - # Uncomment to continually read the log file and display the current domain being blocked - #tail -f /var/log/pihole.log | awk '/\/etc\/pihole\/gravity.list/ {if ($7 != "address" && $7 != "name" && $7 != "/etc/pihole/gravity.list") print $7; else;}' - - today=$(date "+%b %e") - todaysQueryCount=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ {print $7}' | wc -l) - todaysQueryCountV4=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[A\]/ {print $7}' | wc -l) - todaysQueryCountV6=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[AAAA\]/ {print $7}' | wc -l) - todaysAdsEliminated=$(cat /var/log/pihole.log | grep "$today" | awk '/\/etc\/pihole\/gravity.list/ {print $7}' | wc -l) - dividend=$(echo "$todaysAdsEliminated/$todaysQueryCount" | bc -l) - fp=$(echo "$dividend*100" | bc -l) - percentAds=$(echo ${fp:0:4}) - - echo "Queries: $todaysQueryCountV4 / $todaysQueryCountV6" - echo "Pi-holed: $todaysAdsEliminated ($percentAds%)" - sleep 5 + case "$var" in + "-j" | "--json" ) outputJSON;; + "-h" | "--help" ) displayHelp;; + * ) exit 1;; + esac done diff --git a/advanced/Scripts/piholeLogFlush.sh b/advanced/Scripts/piholeLogFlush.sh new file mode 100755 index 00000000..832c98c9 --- /dev/null +++ b/advanced/Scripts/piholeLogFlush.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +# Flushes /var/log/pihole.log +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +#Pi-hole is free software: you can redistribute it and/or modify +#it under the terms of the GNU General Public License as published by +#the Free Software Foundation, either version 2 of the License, or +#(at your option) any later version. + +truncate -s 0 /var/log/pihole.log diff --git a/advanced/Scripts/updateDashboard.sh b/advanced/Scripts/updateDashboard.sh new file mode 100644 index 00000000..bcd92c1e --- /dev/null +++ b/advanced/Scripts/updateDashboard.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# +# this script will update the pihole web interface files. +# +# if this is the first time running this script after an +# existing installation, the existing web interface files +# will be removed and replaced with the latest master +# branch from github. subsequent executions of this script +# will pull the latest version of the web interface. +# +# @TODO: add git as requirement to basic-install.sh +# + +WEB_INTERFACE_GIT_URL="https://github.com/jacobsalmela/AdminLTE.git" +WEB_INTERFACE_DIR="/var/www/html/admin" + +main() { + prerequisites + if ! is_repo; then + make_repo + fi + update_repo +} + +prerequisites() { + + # must be root to update + if [[ $EUID -ne 0 ]]; then + sudo bash "$0" "$@" + exit $? + fi + + # web interface must already exist. this is a (lazy) + # check to make sure pihole is actually installed. + if [ ! -d "$WEB_INTERFACE_DIR" ]; then + echo "$WEB_INTERFACE_DIR not found. Exiting." + exit 1 + fi + + if ! type "git" > /dev/null; then + apt-get -y install git + fi +} + +is_repo() { + # if the web interface directory does not have a .git folder + # it means its using the master.zip archive from the install + # script. + if [ ! -d "$WEB_INTERFACE_DIR/.git" ]; then + return 1 + fi + return 0 +} + +# removes the web interface installed from the master.zip archive and +# replaces it with the current master branch from github +make_repo() { + # remove the non-repod interface and clone the interface + rm -rf $WEB_INTERFACE_DIR + git clone "$WEB_INTERFACE_GIT_URL" "$WEB_INTERFACE_DIR" +} + +# pulls the latest master branch from github +update_repo() { + # pull the latest commits + cd "$WEB_INTERFACE_DIR" + git pull +} + +main diff --git a/advanced/Scripts/whitelist.sh b/advanced/Scripts/whitelist.sh index aed6bd0f..8d871c51 100755 --- a/advanced/Scripts/whitelist.sh +++ b/advanced/Scripts/whitelist.sh @@ -1,13 +1,180 @@ -#!/bin/bash -# For each argument passed to this script +#!/usr/bin/env bash +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. + +if [[ $# = 0 ]]; then + echo "Immediately whitelists one or more domains in the hosts file" + echo " " + echo "Usage: whitelist.sh domain1 [domain2 ...]" + echo " " + echo "Options:" + echo " -d, --delmode Remove domains from the whitelist" + echo " -nr, --noreload Update Whitelist without refreshing dnsmasq" + echo " -f, --force Force updating of the hosts files, even if there are no changes" + echo " -q, --quiet output is less verbose" + exit 1 +fi + +#globals +whitelist=/etc/pihole/whitelist.txt +adList=/etc/pihole/gravity.list +reload=true +addmode=true +force=false +versbose=true +domList=() +domToRemoveList=() + +piholeIPfile=/tmp/piholeIP +piholeIPv6file=/etc/pihole/.useIPv6 + +# Otherwise, the IP address can be taken directly from the machine, which will happen when the script is run by the user and not the installation script +IPv4dev=$(ip route get 8.8.8.8 | awk '{for(i=1;i<=NF;i++)if($i~/dev/)print $(i+1)}') +piholeIPCIDR=$(ip -o -f inet addr show dev $IPv4dev | awk '{print $4}' | awk 'END {print}') +piholeIP=${piholeIPCIDR%/*} + +modifyHost=false + + +if [[ -f $piholeIPv6file ]];then + # If the file exists, then the user previously chose to use IPv6 in the automated installer + piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }') +fi + + +function HandleOther(){ + #check validity of domain + validDomain=$(echo $1 | perl -ne'print if /\b((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,63}\b/') + + if [ -z "$validDomain" ]; then + echo $1 is not a valid argument or domain name + else + domList=("${domList[@]}" $validDomain) + fi +} + +function PopWhitelistFile(){ + #check whitelist file exists, and if not, create it + if [[ ! -f $whitelist ]];then + touch $whitelist + fi + for dom in "${domList[@]}" + do + if $addmode; then + AddDomain $dom + else + RemoveDomain $dom + fi + done +} + +function AddDomain(){ +#| sed 's/\./\\./g' + bool=false + grep -Ex -q "$1" $whitelist || bool=true + if $bool; then + #domain not found in the whitelist file, add it! + if $versbose; then + echo "** Adding $1 to whitelist file" + fi + echo $1 >> $whitelist + modifyHost=true + else + if $versbose; then + echo "** $1 already whitelisted! No need to add" + fi + fi +} + +function RemoveDomain(){ + + bool=false + grep -Ex -q "$1" $whitelist || bool=true + if $bool; then + #Domain is not in the whitelist file, no need to Remove + if $versbose; then + echo "** $1 is NOT whitelisted! No need to remove" + fi + else + #Domain is in the whitelist file, add to a temporary array and remove from whitelist file + if $versbose; then + echo "** Un-whitelisting $dom..." + fi + domToRemoveList=("${domToRemoveList[@]}" $1) + modifyHost=true + fi +} + +function ModifyHostFile(){ + if $addmode; then + #remove domains in from hosts file + if [[ -r $whitelist ]];then + # Remove whitelist entries + numberOf=$(cat $whitelist | sed '/^\s*$/d' | wc -l) + plural=; [[ "$numberOf" != "1" ]] && plural=s + echo "** Whitelisting a total of $numberOf domain${plural}..." + awk -F':' '{ print $1 }' $whitelist | sed 's/\./\\./g' | xargs -I {} perl -i -ne'print unless /[^.]'{}'(?!.)/;' $adList + fi + else + #we need to add the removed domains to the hosts file + for rdom in "${domToRemoveList[@]}" + do + if [[ -n $piholeIPv6 ]];then + echo "**Blacklisting $rdom on IPv4 and IPv6" + echo $rdom | awk -v ipv4addr="$piholeIP" -v ipv6addr="$piholeIPv6" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' >> $adList + else + echo "**Blacklisting $rdom on IPv4" + echo $rdom | awk -v ipv4addr="$piholeIP" '{sub(/\r$/,""); print ipv4addr" "$0}' >>$adList + fi + echo $rdom| sed 's/\./\\./g' | xargs -I {} perl -i -ne'print unless /'{}'(?!.)/;' $whitelist + done + fi +} + +function Reload() { + # Reload hosts file + echo "** Refresh lists in dnsmasq..." + dnsmasqPid=$(pidof dnsmasq) + + if [[ $dnsmasqPid ]]; then + # service already running - reload config + sudo kill -HUP $dnsmasqPid + else + # service not running, start it up + sudo service dnsmasq start + fi +} + +################################################### + for var in "$@" do - echo "Whitelisting $var..." - # Use sed to search for the domain in /etc/pihole/gravity.list and remove it using an in-place edit - sed -i "/$var/d" /etc/pihole/gravity.list - # Also add the domain to the whitelist.txt in /etc/pihole - echo "$var" >> /etc/pihole/whitelist.txt + case "$var" in + "-nr"| "--noreload" ) reload=false;; + "-d" | "--delmode" ) addmode=false;; + "-f" | "--force" ) force=true;; + "-q" | "--quiet" ) versbose=false;; + * ) HandleOther $var;; + esac done -echo "** $# domain(s) whitelisted." -# Force dnsmasq to reload /etc/pihole/gravity.list -kill -HUP $(pidof dnsmasq) \ No newline at end of file + +PopWhitelistFile + +if $modifyHost || $force; then + echo "** Modifying Hosts File" + ModifyHostFile +else + if $versbose; then + echo "** No changes need to be made" + exit 1 + fi +fi + +if $reload; then + Reload +fi diff --git a/advanced/dnsmasq.conf b/advanced/dnsmasq.conf index 25b2138e..25f52a57 100644 --- a/advanced/dnsmasq.conf +++ b/advanced/dnsmasq.conf @@ -1,12 +1,50 @@ +# If you want dnsmasq to read another file, as well as /etc/hosts, use +# this. addn-hosts=/etc/pihole/gravity.list + +# The following two options make you a better netizen, since they +# tell dnsmasq to filter out queries which the public DNS cannot +# answer, and which load the servers (especially the root servers) +# unnecessarily. If you have a dial-on-demand link they also stop +# these requests from bringing up the link unnecessarily. + +# Never forward plain names (without a dot or domain part) domain-needed +# Never forward addresses in the non-routed address spaces. bogus-priv + +# If you don't want dnsmasq to read /etc/resolv.conf or any other +# file, getting its servers from this file instead (see below), then +# uncomment this. no-resolv + +# Add other name servers here, with domain specs if they are for +# non-public domains. server=8.8.8.8 server=8.8.4.4 -interface=eth0 + +# If you want dnsmasq to listen for DHCP and DNS requests only on +# specified interfaces (and the loopback) give the name of the +# interface (eg eth0) here. +interface=@INT@ +# Or which to listen on by address (remember to include 127.0.0.1 if +# you use this.) listen-address=127.0.0.1 + +# Set the cachesize here. cache-size=10000 + +# For debugging purposes, log each DNS query as it passes through +# dnsmasq. log-queries log-facility=/var/log/pihole.log + +# Normally responses which come from /etc/hosts and the DHCP lease +# file have Time-To-Live set as zero, which conventionally means +# do not cache further. If you are happy to trade lower load on the +# server for potentially stale date, you can set a time-to-live (in +# seconds) here. local-ttl=300 + +# This allows it to continue functioning without being blocked by syslog, and allows syslog to use dnsmasq for DNS queries without risking deadlock +log-async diff --git a/advanced/lighttpd.conf b/advanced/lighttpd.conf index 1c3ed076..3998269a 100644 --- a/advanced/lighttpd.conf +++ b/advanced/lighttpd.conf @@ -1,38 +1,51 @@ server.modules = ( + "mod_access", + "mod_accesslog", "mod_expire", "mod_compress", "mod_redirect", + "mod_setenv", "mod_rewrite" ) - -server.document-root = "/var/www" + +server.document-root = "/var/www/html" +server.error-handler-404 = "pihole/index.html" server.upload-dirs = ( "/var/cache/lighttpd/uploads" ) server.errorlog = "/var/log/lighttpd/error.log" server.pid-file = "/var/run/lighttpd.pid" server.username = "www-data" server.groupname = "www-data" server.port = 80 - - +accesslog.filename = "/var/log/lighttpd/access.log" +accesslog.format = "%{%s}t|%V|%r|%s|%b" + + index-file.names = ( "index.php", "index.html", "index.lighttpd.html" ) url.access-deny = ( "~", ".inc" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) - + compress.cache-dir = "/var/cache/lighttpd/compress/" compress.filetype = ( "application/javascript", "text/css", "text/html", "text/plain" ) - + # default listening port for IPv6 falls back to the IPv4 port include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port include_shell "/usr/share/lighttpd/create-mime.assign.pl" include_shell "/usr/share/lighttpd/include-conf-enabled.pl" - -# Set access to 1 day for better query performance when the list gets so large -# http://jacobsalmela.com/raspberry-pi-block-ads-adtrap/#comment-2013820434 -$HTTP["url"] =~ "^/pihole/" { - expire.url = ("" => "access plus 1 days") + +# If the URL starts with /admin, it is the Web interface +$HTTP["url"] =~ "^/admin/" { + # Create a response header for debugging using curl -I + setenv.add-response-header = ( "X-Pi-hole" => "The Pi-hole Web interface is working!" ) +} + +# If the URL does not start with /admin, then it is a query for an ad domain +$HTTP["url"] =~ "^(?!/admin)/.*" { + # Create a response header for debugging using curl -I + setenv.add-response-header = ( "X-Pi-hole" => "A black hole for Internet advertisements." ) + + # Set the cache to 1 day for better performance + expire.url = ("" => "access plus 1 days") + + # Send the query into the black hole + url.rewrite = (".*" => "pihole/index.html" ) } - -# Rewrites all URLs to the /var/www/pihole/index.html -$HTTP["host"] =~ ".*" { - url.rewrite = (".*" => "pihole/index.html") -} \ No newline at end of file diff --git a/advanced/pihole.cron b/advanced/pihole.cron index a707607e..57f1c10c 100644 --- a/advanced/pihole.cron +++ b/advanced/pihole.cron @@ -1 +1,15 @@ -@weekly sudo /usr/local/bin/gravity.sh +# Pi-hole: Update the ad sources once a week on Sunday at 01:59 +# Download any updates from the ad lists +59 1 * * 7 root /usr/local/bin/gravity.sh + +# Pi-hole: Update the Web interface shortly after gravity runs +# This should also update the version number if it is changed in the dashboard repo +30 2 * * 7 root /usr/local/bin/updateDashboard.sh + +# Pi-hole: Parse the log file before it is flushed and save the stats to a database +# This will be used for a historical view of your Pi-hole's performance +#50 23 * * * root /usr/local/bin/dailyLog.sh + +# Pi-hole: Flush the log daily at 11:58 so it doesn't get out of control +# Stats will be viewable in the Web interface thanks to the cron job above +58 23 * * * root /usr/local/bin/piholeLogFlush.sh diff --git a/automated install/basic-install.sh b/automated install/basic-install.sh index f0a7d139..bdbcb3e8 100755 --- a/automated install/basic-install.sh +++ b/automated install/basic-install.sh @@ -1,8 +1,13 @@ +<<<<<<< HEAD #!/bin/bash +======= +#!/usr/bin/env bash +>>>>>>> master # Pi-hole: A black hole for Internet advertisements # by Jacob Salmela # Network-wide ad blocking via your Raspberry Pi # +<<<<<<< HEAD # pi-hole.net/donate # # Install with this command (from your Pi): @@ -33,6 +38,55 @@ IPv4gw=$(ip route show | awk '/default\ via/ {print $3}') #IPv6linkLocal=$(ip addr show | awk '/inet/ && /scope\ link/ && /fe80/ {print $2}' | cut -d'/' -f1) availableInterfaces=$(ip link show | awk -F' ' '/[0-9]: [a-z]/ {print $2}' | grep -v "lo" | cut -d':' -f1) +======= +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# pi-hole.net/donate +# +# Install with this command (from your Pi): +# +# curl -L install.pi-hole.net | bash + +######## VARIABLES ######### +# Must be root to install +if [[ $EUID -eq 0 ]];then + echo "You are root." +else + echo "sudo will be used for the install." + # Check if it is actually installed + # If it isn't, exit because the install cannot complete + if [[ $(dpkg-query -s sudo) ]];then + export SUDO="sudo" + else + echo "Please install sudo or run this as root." + exit 1 + fi +fi + + +tmpLog=/tmp/pihole-install.log +instalLogLoc=/etc/pihole/install.log + +# Find the rows and columns +rows=$(tput lines) +columns=$(tput cols) + +# Divide by two so the dialogs take up half of the screen, which looks nice. +r=$(( rows / 2 )) +c=$(( columns / 2 )) + +# Find IP used to route to outside world +IPv4dev=$(ip route get 8.8.8.8 | awk '{for(i=1;i<=NF;i++)if($i~/dev/)print $(i+1)}') +IPv4addr=$(ip -o -f inet addr show dev $IPv4dev | awk '{print $4}' | awk 'END {print}') +IPv4gw=$(ip route get 8.8.8.8 | awk '{print $3}') + +availableInterfaces=$(ip -o link | awk '{print $2}' | grep -v "lo" | cut -d':' -f1) +>>>>>>> master dhcpcdFile=/etc/dhcpcd.conf ####### FUCNTIONS ########## @@ -40,6 +94,7 @@ backupLegacyPihole() { if [[ -f /etc/dnsmasq.d/adList.conf ]];then echo "Original Pi-hole detected. Initiating sub space transport" +<<<<<<< HEAD sudo mkdir -p /etc/pihole/original/ sudo mv /etc/dnsmasq.d/adList.conf /etc/pihole/original/adList.conf.$(date "+%Y-%m-%d") sudo mv /etc/dnsmasq.conf /etc/pihole/original/dnsmasq.conf.$(date "+%Y-%m-%d") @@ -47,6 +102,14 @@ if [[ -f /etc/dnsmasq.d/adList.conf ]];then sudo mv /etc/lighttpd/lighttpd.conf /etc/pihole/original/lighttpd.conf.$(date "+%Y-%m-%d") sudo mv /var/www/pihole/index.html /etc/pihole/original/index.html.$(date "+%Y-%m-%d") sudo mv /usr/local/bin/gravity.sh /etc/pihole/original/gravity.sh.$(date "+%Y-%m-%d") +======= + $SUDO mkdir -p /etc/pihole/original/ + $SUDO mv /etc/dnsmasq.d/adList.conf /etc/pihole/original/adList.conf.$(date "+%Y-%m-%d") + $SUDO mv /etc/dnsmasq.conf /etc/pihole/original/dnsmasq.conf.$(date "+%Y-%m-%d") + $SUDO mv /etc/resolv.conf /etc/pihole/original/resolv.conf.$(date "+%Y-%m-%d") + $SUDO mv /etc/lighttpd/lighttpd.conf /etc/pihole/original/lighttpd.conf.$(date "+%Y-%m-%d") + $SUDO mv /var/www/pihole/index.html /etc/pihole/original/index.html.$(date "+%Y-%m-%d") + $SUDO mv /usr/local/bin/gravity.sh /etc/pihole/original/gravity.sh.$(date "+%Y-%m-%d") else : fi @@ -57,6 +120,271 @@ welcomeDialogs() # Display the welcome dialog whiptail --msgbox --backtitle "Welcome" --title "Pi-hole automated installer" "This installer will transform your Raspberry Pi into a network-wide ad blocker!" $r $c +# Support for a part-time dev +whiptail --msgbox --backtitle "Plea" --title "Free and open source" "The Pi-hole is free, but powered by your donations: http://pi-hole.net/donate" $r $c + +# Explain the need for a static address +whiptail --msgbox --backtitle "Initating network interface" --title "Static IP Needed" "The Pi-hole is a SERVER so it needs a STATIC IP ADDRESS to function properly. + +In the next section, you can choose to use your current network settings (DHCP) or to manually edit them." $r $c +} + +chooseInterface() +{ +# Turn the available interfaces into an array so it can be used with a whiptail dialog +interfacesArray=() +firstloop=1 + +while read -r line +do +mode="OFF" +if [[ $firstloop -eq 1 ]]; then + firstloop=0 + mode="ON" +fi +interfacesArray+=("$line" "available" "$mode") +done <<< "$availableInterfaces" + +# Find out how many interfaces are available to choose from +interfaceCount=$(echo "$availableInterfaces" | wc -l) +chooseInterfaceCmd=(whiptail --separate-output --radiolist "Choose An Interface" $r $c $interfaceCount) +chooseInterfaceOptions=$("${chooseInterfaceCmd[@]}" "${interfacesArray[@]}" 2>&1 >/dev/tty) +for desiredInterface in $chooseInterfaceOptions +do + piholeInterface=$desiredInterface + echo "Using interface: $piholeInterface" + echo ${piholeInterface} > /tmp/piholeINT +done +} + +use4andor6() +{ +# Let use select IPv4 and/or IPv6 +cmd=(whiptail --separate-output --checklist "Select Protocols" $r $c 2) +options=(IPv4 "Block ads over IPv4" on + IPv6 "Block ads over IPv6" off) +choices=$("${cmd[@]}" "${options[@]}" 2>&1 >/dev/tty) +for choice in $choices +do + case $choice in + IPv4) + echo "IPv4 selected." + useIPv4=true + ;; + IPv6) + echo "IPv6 selected." + useIPv6=true + ;; + esac +done +} + +useIPv6dialog() +{ +piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }') +whiptail --msgbox --backtitle "IPv6..." --title "IPv6 Supported" "$piholeIPv6 will be used to block ads." $r $c +$SUDO mkdir -p /etc/pihole/ +$SUDO touch /etc/pihole/.useIPv6 +} + +getStaticIPv4Settings() +{ +# Ask if the user wannts to use DHCP settings as their static IP +if (whiptail --backtitle "Calibrating network interface" --title "Static IP Address" --yesno "Do you want to use your current network settings as a static address? + + IP address: $IPv4addr + Gateway: $IPv4gw" $r $c) then + # If they choose yes, let the user know that the IP address will not be available via DHCP and may cause a conflict. + whiptail --msgbox --backtitle "IP information" --title "FYI: IP Conflict" "It is possible your router could still try to assign this IP to a device, which would cause a conflict. But in most cases the router is smart enough to not do that. + + If you are worried, either manually set the address, or modify the DHCP reservation pool so it does not include the IP you want. + + It is also possible to use a DHCP reservation, but if you are going to do that, you might as well set a static address." $r $c + # Nothing else to do since the variables are already set above +>>>>>>> master +else + # Otherwise, we need to ask the user to input their desired settings. + # Start by getting the IPv4 address (pre-filling it with info gathered from DHCP) + # Start a loop to let the user enter their information with the chance to go back and edit it if necessary + until [[ $ipSettingsCorrect = True ]] + do + # Ask for the IPv4 address + IPv4addr=$(whiptail --backtitle "Calibrating network interface" --title "IPv4 address" --inputbox "Enter your desired IPv4 address" $r $c $IPv4addr 3>&1 1>&2 2>&3) + if [[ $? = 0 ]];then + echo "Your static IPv4 address: $IPv4addr" + # Ask for the gateway + IPv4gw=$(whiptail --backtitle "Calibrating network interface" --title "IPv4 gateway (router)" --inputbox "Enter your desired IPv4 default gateway" $r $c $IPv4gw 3>&1 1>&2 2>&3) + if [[ $? = 0 ]];then + echo "Your static IPv4 gateway: $IPv4gw" + # Give the user a chance to review their settings before moving on + if (whiptail --backtitle "Calibrating network interface" --title "Static IP Address" --yesno "Are these settings correct? + IP address: $IPv4addr + Gateway: $IPv4gw" $r $c)then + # If the settings are correct, then we need to set the piholeIP + # Saving it to a temporary file us to retrieve it later when we run the gravity.sh script + echo ${IPv4addr%/*} > /tmp/piholeIP + echo $piholeInterface > /tmp/piholeINT + # After that's done, the loop ends and we move on + ipSettingsCorrect=True + else + # If the settings are wrong, the loop continues + ipSettingsCorrect=False + fi + else + # Cancelling gateway settings window + ipSettingsCorrect=False + echo "User canceled." + exit + fi + else + # Cancelling IPv4 settings window + ipSettingsCorrect=False + echo "User canceled." + exit + fi +done +# End the if statement for DHCP vs. static +fi +} + +setDHCPCD(){ +# Append these lines to dhcpcd.conf to enable a static IP +echo "interface $piholeInterface +static ip_address=$IPv4addr +static routers=$IPv4gw +static domain_name_servers=$IPv4gw" | $SUDO tee -a $dhcpcdFile >/dev/null +} + +setStaticIPv4(){ +if grep -q $IPv4addr $dhcpcdFile; then + # address already set, noop + : +else + setDHCPCD + $SUDO ip addr replace dev $piholeInterface $IPv4addr + echo "Setting IP to $IPv4addr. You may need to restart after the install is complete." +fi +} + +installScripts(){ +$SUDO curl -o /usr/local/bin/gravity.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh +$SUDO curl -o /usr/local/bin/chronometer.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/chronometer.sh +$SUDO curl -o /usr/local/bin/whitelist.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/whitelist.sh +$SUDO curl -o /usr/local/bin/blacklist.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/blacklist.sh +$SUDO curl -o /usr/local/bin/piholeLogFlush.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/piholeLogFlush.sh +$SUDO curl -o /usr/local/bin/updateDashboard.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/updateDashboard.sh +$SUDO chmod 755 /usr/local/bin/{gravity,chronometer,whitelist,blacklist,piholeLogFlush,updateDashboard}.sh +} + +installConfigs(){ +$SUDO mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig +$SUDO mv /etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.conf.orig +$SUDO curl -o /etc/dnsmasq.conf https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/dnsmasq.conf +$SUDO curl -o /etc/lighttpd/lighttpd.conf https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/lighttpd.conf +$SUDO sed -i "s/@INT@/$piholeInterface/" /etc/dnsmasq.conf +} + +stopServices(){ +$SUDO service dnsmasq stop || true +$SUDO service lighttpd stop || true +} + +installDependencies(){ +$SUDO apt-get update +$SUDO apt-get -y upgrade +$SUDO apt-get -y install dnsutils bc toilet figlet +$SUDO apt-get -y install dnsmasq +$SUDO apt-get -y install lighttpd php5-common php5-cgi php5 +$SUDO apt-get -y install git +} + +installWebAdmin(){ +$SUDO wget https://github.com/jacobsalmela/AdminLTE/archive/master.zip -O /var/www/master.zip +$SUDO unzip -oq /var/www/master.zip -d /var/www/html/ +$SUDO mv /var/www/html/AdminLTE-master /var/www/html/admin +$SUDO rm /var/www/master.zip 2>/dev/null +$SUDO touch /var/log/pihole.log +$SUDO chmod 644 /var/log/pihole.log +$SUDO chown dnsmasq:root /var/log/pihole.log +} + +installPiholeWeb(){ +$SUDO mkdir /var/www/html/pihole +$SUDO mv /var/www/html/index.lighttpd.html /var/www/html/index.lighttpd.orig +$SUDO curl -o /var/www/html/pihole/index.html https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/index.html +} + +installCron(){ +$SUDO curl -o /etc/cron.d/pihole https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/pihole.cron +} + +installPihole() +{ +installDependencies +stopServices +$SUDO chown www-data:www-data /var/www/html +$SUDO chmod 775 /var/www/html +$SUDO usermod -a -G www-data pi +$SUDO lighty-enable-mod fastcgi fastcgi-php +installScripts +installConfigs +installWebAdmin +installPiholeWeb +installCron +$SUDO /usr/local/bin/gravity.sh +} + +displayFinalMessage(){ + whiptail --msgbox --backtitle "Make it so." --title "Installation Complete!" "Configure your devices to use the Pi-hole as their DNS server using: + + $IPv4addr + $piholeIPv6 + +If you set a new IP address, you should restart the Pi. + +The install log is in /etc/pihole." $r $c +} + +######## SCRIPT ############ +# Start the installer +welcomeDialogs + +# Just back up the original Pi-hole right away since it won't take long and it gets it out of the way +backupLegacyPihole + +# Find interfaces and let the user choose one +chooseInterface + +# Let the user decide if they want to block ads over IPv4 and/or IPv6 +use4andor6 + +# Decide is IPv4 will be used +if [[ "$useIPv4" = true ]];then + echo "Using IPv4" + getStaticIPv4Settings + setStaticIPv4 +else + useIPv4=false + echo "IPv4 will NOT be used." +fi + +# Decide is IPv6 will be used +if [[ "$useIPv6" = true ]];then + useIPv6dialog + echo "Using IPv6." + echo "Your IPv6 address is: $piholeIPv6" +else + useIPv6=false + echo "IPv6 will NOT be used." +fi +} + +<<<<<<< HEAD +welcomeDialogs() +{ +# Display the welcome dialog +whiptail --msgbox --backtitle "Welcome" --title "Pi-hole automated installer" "This installer will transform your Raspberry Pi into a network-wide ad blocker!" $r $c + # Explain the need for a static address whiptail --msgbox --backtitle "Initating network interface" --title "Static IP Needed" "The Pi-hole is a SERVER so it needs a STATIC IP ADDRESS to function properly. @@ -272,10 +600,13 @@ else echo "IPv6 will NOT be used. Consider a donation at pi-hole.net/donate" fi +======= +>>>>>>> master # Install and log everything to a file installPihole | tee $tmpLog # Move the log file into /etc/pihole for storage +<<<<<<< HEAD sudo mv $tmpLog $instalLogLoc whiptail --msgbox --backtitle "Make it so." --title "Installation Complete!" "Configure your devices to use the Pi-hole as their DNS server using this IP: $IPv4addr. @@ -293,3 +624,11 @@ else sudo service dnsmasq start sudo service lighttpd start fi +======= +$SUDO mv $tmpLog $instalLogLoc + +displayFinalMessage + +$SUDO service dnsmasq start +$SUDO service lighttpd start +>>>>>>> master diff --git a/automated install/uninstall.sh b/automated install/uninstall.sh new file mode 100644 index 00000000..59eca67b --- /dev/null +++ b/automated install/uninstall.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# Completely uninstalls the Pi-hole +# (c) 2015 by Jacob Salmela +# This file is part of Pi-hole. +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. + +# Must be root to uninstall +if [[ $EUID -eq 0 ]];then + echo "You are root." +else + echo "sudo will be used for the install." + # Check if it is actually installed + # If it isn't, exit because the unnstall cannot complete + if [[ $(dpkg-query -s sudo) ]];then + export SUDO="sudo" + else + echo "Please install sudo or run this as root." + exit 1 + fi +fi + + +######### SCRIPT ########### +$SUDO apt-get -y remove --purge dnsutils bc toilet +$SUDO apt-get -y remove --purge dnsmasq +$SUDO apt-get -y remove --purge lighttpd php5-common php5-cgi php5 + +# Only web directories/files that are created by pihole should be removed. +echo "Removing the Pi-hole Web server files..." +$SUDO rm -rf /var/www/html/admin +$SUDO rm -rf /var/www/html/pihole +$SUDO rm /var/www/html/index.lighttpd.orig + +# If the web directory is empty after removing these files, then the parent html folder can be removed. +if [[ ! "$(ls -A /var/www/html)" ]]; then + $SUDO rm -rf /var/www/html +fi + +echo "Removing dnsmasq config files..." +$SUDO rm /etc/dnsmasq.conf /etc/dnsmasq.conf.orig + +# Attempt to preserve backwards compatibility with older versions +# to guarantee no additional changes were made to /etc/crontab after +# the installation of pihole, /etc/crontab.pihole should be permanently +# preserved. +if [[ -f /etc/crontab.orig ]]; then + echo "Initial Pi-hole cron detected. Restoring the default system cron..." + $SUDO mv /etc/crontab /etc/crontab.pihole + $SUDO mv /etc/crontab.orig /etc/crontab + $SUDO service cron restart +fi + +# Attempt to preserve backwards compatibility with older versions +if [[ -f /etc/cron.d/pihole ]];then + echo "Removing cron.d/pihole..." + $SUDO rm /etc/cron.d/pihole +fi + +echo "Removing config files and scripts..." +$SUDO rm /etc/dnsmasq.conf +$SUDO rm -rf /etc/lighttpd/ +$SUDO rm /var/log/pihole.log +$SUDO rm /usr/local/bin/gravity.sh +$SUDO rm /usr/local/bin/chronometer.sh +$SUDO rm /usr/local/bin/whitelist.sh +$SUDO rm /usr/local/bin/piholeLogFlush.sh +$SUDO rm -rf /etc/pihole/ diff --git a/gravity.sh b/gravity.sh index a40fe7e3..ddc47969 100755 --- a/gravity.sh +++ b/gravity.sh @@ -1,50 +1,83 @@ -#!/bin/bash +#!/usr/bin/env bash +# Pi-hole: A black hole for Internet advertisements +# (c) 2015 by Jacob Salmela +# Network-wide ad blocking via your Raspberry Pi # http://pi-hole.net # Compiles a list of ad-serving domains by downloading them from multiple sources +<<<<<<< HEAD piholeIPfile=/tmp/piholeIP +======= +# +# Pi-hole is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. + +piholeIPfile=/tmp/piholeIP +piholeIPv6file=/etc/pihole/.useIPv6 +>>>>>>> master if [[ -f $piholeIPfile ]];then # If the file exists, it means it was exported from the installation script and we should use that value instead of detecting it in this script piholeIP=$(cat $piholeIPfile) rm $piholeIPfile else # Otherwise, the IP address can be taken directly from the machine, which will happen when the script is run by the user and not the installation script +<<<<<<< HEAD piholeIP=$(ip -4 addr show | awk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' | sed '/^\s*$/d' | grep -v "127.0.0.1") fi # Ad-list sources--one per line in single quotes # The mahakala source is commented out due to many users having issues with it blocking legitimate domains. Uncomment at your own risk +======= + IPv4dev=$(ip route get 8.8.8.8 | awk '{for(i=1;i<=NF;i++)if($i~/dev/)print $(i+1)}') + piholeIPCIDR=$(ip -o -f inet addr show dev $IPv4dev | awk '{print $4}' | awk 'END {print}') + piholeIP=${piholeIPCIDR%/*} +fi + +if [[ -f $piholeIPv6file ]];then + # If the file exists, then the user previously chose to use IPv6 in the automated installer + piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }') +fi + +# Ad-list sources--one per line in single quotes +# The mahakala source is commented out due to many users having issues with it blocking legitimate domains. +# Uncomment at your own risk +>>>>>>> master sources=('https://adaway.org/hosts.txt' 'http://adblock.gjtech.net/?format=unix-hosts' #'http://adblock.mahakala.is/' -'http://hosts-file.net/.%5Cad_servers.txt' +'http://hosts-file.net/ad_servers.txt' 'http://www.malwaredomainlist.com/hostslist/hosts.txt' 'http://pgl.yoyo.org/adservers/serverlist.php?' 'http://someonewhocares.org/hosts/hosts' 'http://winhelp2002.mvps.org/hosts.txt') # Variables for various stages of downloading and formatting the list -adList=/etc/pihole/gravity.list -origin=/etc/pihole -piholeDir=/etc/pihole -justDomainsExtension=domains -matter=pihole.0.matter.txt -andLight=pihole.1.andLight.txt -supernova=pihole.2.supernova.txt -eventHorizon=pihole.3.eventHorizon.txt -accretionDisc=pihole.4.accretionDisc.txt -eyeOfTheNeedle=pihole.5.wormhole.txt +basename=pihole +piholeDir=/etc/$basename +adList=$piholeDir/gravity.list blacklist=$piholeDir/blacklist.txt -latentBlacklist=$origin/latentBlacklist.txt whitelist=$piholeDir/whitelist.txt -latentWhitelist=$origin/latentWhitelist.txt +latentWhitelist=$piholeDir/latentWhitelist.txt +justDomainsExtension=domains +matterandlight=$basename.0.matterandlight.txt +supernova=$basename.1.supernova.txt +eventHorizon=$basename.2.eventHorizon.txt +accretionDisc=$basename.3.accretionDisc.txt +eyeOfTheNeedle=$basename.4.wormhole.txt # After setting defaults, check if there's local overrides if [[ -r $piholeDir/pihole.conf ]];then echo "** Local calibration requested..." - . $piholeDir/pihole.conf + . $piholeDir/pihole.conf fi +########################### +# collapse - begin formation of pihole +function gravity_collapse() { + echo "** Neutrino emissions detected..." +<<<<<<< HEAD echo "** Neutrino emissions detected..." # Create the pihole resource directory if it doesn't exist. Future files will be stored here @@ -66,24 +99,38 @@ function createSwapFile() sudo dphys-swapfile swapon } - -if [[ -n "$noSwap" ]]; then - # if $noSwap is set, don't do anything - : -elif [[ -f /etc/dphys-swapfile ]];then - swapSize=$(cat /etc/dphys-swapfile | grep -m1 CONF_SWAPSIZE | cut -d'=' -f2) - if [[ $swapSize != 500 ]];then - mv /etc/dphys-swapfile /etc/dphys-swapfile.orig - echo "** Current swap size is $swapSize" - createSwapFile +======= + # Create the pihole resource directory if it doesn't exist. Future files will be stored here + if [[ -d $piholeDir ]];then + # Temporary hack to allow non-root access to pihole directory + # Will update later, needed for existing installs, new installs should + # create this directory as non-root + sudo chmod 777 $piholeDir + find "$piholeDir" -type f -exec sudo chmod 666 {} \; else - : + echo "** Creating pihole directory..." + mkdir $piholeDir fi -else - echo "** No swap file found. Creating one..." - createSwapFile -fi +} +>>>>>>> master +# patternCheck - check to see if curl downloaded any new files. +function gravity_patternCheck() { + patternBuffer=$1 + # check if the patternbuffer is a non-zero length file + if [[ -s "$patternBuffer" ]];then + # Some of the blocklists are copyright, they need to be downloaded + # and stored as is. They can be processed for content after they + # have been saved. + cp $patternBuffer $saveLocation + echo "List updated, transport successful..." + else + # curl didn't download any host files, probably because of the date check + echo "No changes detected, transport skipped..." + fi +} + +<<<<<<< HEAD # Loop through domain list. Download each one and remove commented lines (lines beginning with '# 'or '/') and blank lines for ((i = 0; i < "${#sources[@]}"; i++)) do @@ -114,22 +161,32 @@ do echo "Done." else echo "Skipping list because it does not have any new entries." +======= +# transport - curl the specified url with any needed command extentions +function gravity_transport() { + url=$1 + cmd_ext=$2 + agent=$3 + + # tmp file, so we don't have to store the (long!) lists in RAM + patternBuffer=$(mktemp) + heisenbergCompensator="" + if [[ -r $saveLocation ]]; then + # if domain has been saved, add file for date check to only download newer + heisenbergCompensator="-z $saveLocation" +>>>>>>> master fi -done -# Find all files with the .domains extension and compile them into one file and remove CRs -echo "** Aggregating list of domains..." -find $origin/ -type f -name "*.$justDomainsExtension" -exec cat {} \; | tr -d '\r' > $origin/$matter + # Silently curl url + curl -s $cmd_ext $heisenbergCompensator -A "$agent" $url > $patternBuffer + # Check for list updates + gravity_patternCheck $patternBuffer -# Append blacklist entries if they exist -if [[ -f $blacklist ]];then - numberOf=$(cat $blacklist | sed '/^\s*$/d' | wc -l) - echo "** Blacklisting $numberOf domain(s)..." - cat $blacklist >> $origin/$matter -else - : -fi + # Cleanup + rm -f $patternBuffer +} +<<<<<<< HEAD function gravity_advanced() ########################### { @@ -137,12 +194,92 @@ function gravity_advanced() echo "** $numberOf domains being pulled in by gravity..." # Remove carriage returns and preceding whitespace cat $origin/$andLight | sed $'s/\r$//' | sed '/^\s*$/d' > $origin/$supernova +======= +# spinup - main gravity function +function gravity_spinup() { + + # Loop through domain list. Download each one and remove commented lines (lines beginning with '# 'or '/') and # blank lines + for ((i = 0; i < "${#sources[@]}"; i++)) + do + url=${sources[$i]} + # Get just the domain from the URL + domain=$(echo "$url" | cut -d'/' -f3) + + # Save the file as list.#.domain + saveLocation=$piholeDir/list.$i.$domain.$justDomainsExtension + activeDomains[$i]=$saveLocation + + agent="Mozilla/10.0" + + echo -n " Getting $domain list: " + + # Use a case statement to download lists that need special cURL commands + # to complete properly and reset the user agent when required + case "$domain" in + "adblock.mahakala.is") + agent='Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0' + cmd_ext="-e http://forum.xda-developers.com/" + ;; + + "pgl.yoyo.org") + cmd_ext="-d mimetype=plaintext -d hostformat=hosts" + ;; + + # Default is a simple request + *) cmd_ext="" + esac + gravity_transport $url $cmd_ext $agent + done +} + +# Schwarzchild - aggregate domains to one list and add blacklisted domains +function gravity_Schwarzchild() { + + # Find all active domains and compile them into one file and remove CRs + echo "** Aggregating list of domains..." + truncate -s 0 $piholeDir/$matterandlight + for i in "${activeDomains[@]}" + do + cat $i |tr -d '\r' >> $piholeDir/$matterandlight + done +} + + +function gravity_Blacklist(){ + # Append blacklist entries if they exist + blacklist.sh -f -nr -q +} + + +function gravity_Whitelist() { + # Prevent our sources from being pulled into the hole + plural=; [[ "${sources[@]}" != "1" ]] && plural=s + echo "** Whitelisting ${#sources[@]} ad list source${plural}..." + + urls=() + for url in ${sources[@]} + do + tmp=$(echo "$url" | awk -F '/' '{print $3}') + urls=("${urls[@]}" $tmp) + done + + whitelist.sh -f -nr -q ${urls[@]} + + +} + +function gravity_unique() { +>>>>>>> master # Sort and remove duplicates - cat $origin/$supernova | sort | uniq > $origin/$eventHorizon - numberOf=$(cat $origin/$eventHorizon | sed '/^\s*$/d' | wc -l) + sort -u $piholeDir/$supernova > $piholeDir/$eventHorizon + numberOf=$(wc -l < $piholeDir/$eventHorizon) echo "** $numberOf unique domains trapped in the event horizon." +} + +function gravity_hostFormat() { # Format domain list as "192.168.x.x domain.com" echo "** Formatting domains into a HOSTS file..." +<<<<<<< HEAD cat $origin/$eventHorizon | awk '{sub(/\r$/,""); print "'"$piholeIP"' " $0}' > $origin/$accretionDisc # Copy the file over as /etc/pihole/gravity.list so dnsmasq can use it sudo cp $origin/$accretionDisc $adList @@ -162,15 +299,69 @@ if [[ -f $whitelist ]];then else rm $latentWhitelist fi +======= + # If there is a value in the $piholeIPv6, then IPv6 will be used, so the awk command modified to create a line for both protocols + if [[ -n $piholeIPv6 ]];then + cat $piholeDir/$eventHorizon | awk -v ipv4addr="$piholeIP" -v ipv6addr="$piholeIPv6" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' > $piholeDir/$accretionDisc + else + # Otherwise, just create gravity.list as normal using IPv4 + cat $piholeDir/$eventHorizon | awk -v ipv4addr="$piholeIP" '{sub(/\r$/,""); print ipv4addr" "$0}' > $piholeDir/$accretionDisc + fi + # Copy the file over as /etc/pihole/gravity.list so dnsmasq can use it + cp $piholeDir/$accretionDisc $adList +} -# Prevent our sources from being pulled into the hole -plural=; [[ "${#sources[@]}" != "1" ]] && plural=s -echo "** Whitelisting ${#sources[@]} ad list source${plural}..." -for url in ${sources[@]} -do - echo "$url" | awk -F '/' '{print "^"$3"$"}' | sed 's/\./\\./g' >> $latentWhitelist -done +# blackbody - remove any remnant files from script processes +function gravity_blackbody() { + # Loop through list files + for file in $piholeDir/*.$justDomainsExtension + do + # If list is in active array then leave it (noop) else rm the list + if [[ " ${activeDomains[@]} " =~ " ${file} " ]]; then + : + else + rm -f $file + fi + done +} -grep -vxf $latentWhitelist $origin/$matter > $origin/$andLight +function gravity_advanced() { +>>>>>>> master + + # Remove comments and print only the domain name + # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contigious + # This helps with that and makes it easier to read + # It also helps with debugging so each stage of the script can be researched more in depth + awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' $piholeDir/$matterandlight | sed -nr -e 's/\.{2,}/./g' -e '/\./p' > $piholeDir/$supernova + + numberOf=$(wc -l < $piholeDir/$supernova) + echo "** $numberOf domains being pulled in by gravity..." + + gravity_unique +} + +function gravity_reload() { + # Reload hosts file + echo "** Refresh lists in dnsmasq..." + dnsmasqPid=$(pidof dnsmasq) + + if [[ $dnsmasqPid ]]; then + # service already running - reload config + sudo kill -HUP $dnsmasqPid + else + # service not running, start it up + sudo service dnsmasq start + fi +} + + +gravity_collapse +gravity_spinup +gravity_Schwarzchild gravity_advanced +gravity_hostFormat +gravity_blackbody +gravity_Whitelist +gravity_Blacklist +gravity_reload