Merge pull request #11 from jacobsalmela/master

Upstream sync

README.md

@@ -1,68 +1,81 @@
 # Automated Install 
-#### Designed For Raspberry Pi B, B+, 2,
+##### Designed For Raspberry Pi B, B+, 2, and Zero (with an Ethernet adapter)
-##### and Zero (with an Ethernet adapter)
 
 1. Install Raspbian 
 2. Run the command below
 
 ### ```curl -L install.pi-hole.net | bash```
 
-![Pi-hole automated installation](http://i.imgur.com/Un7lBlj.png)
+Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](http://pi-hole.net/faq/can-i-set-the-pi-hole-to-be-the-dns-server-at-my-router-so-i-dont-have-to-change-settings-for-my-devices/) and then any device that connects to your network will have ads blocked without any further configuration.  Alternatively, you can manually set each device to [use the Raspberry Pi as its DNS server](http://pi-hole.net/faq/how-do-i-use-the-pi-hole-as-my-dns-server/).
 
-Once installed, **configure any device to use the Raspberry Pi as its DNS server and the ads will be blocked**.  You can also configure your router's DHCP options to assign the Pi as clients DNS server so they do not need to do it manually.  
+## Pi-hole Is Free, But Powered By Your Donations
+[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "Free, but powered by donations")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate")
 
-[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "AdminLTE Presentation")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate")
+## How Does It Work?
-
-# How Does It Work?
 **Watch the 60-second video below to get a quick overview**
 
-[![Pi-hole exlplained](http://i.imgur.com/wd5ltCU.png)](https://vimeo.com/135965232)
+[![Pi-hole exlplained](http://i.imgur.com/qNybJDX.png)](https://vimeo.com/135965232)
+
+## Pi-hole Projects
+- [Sky-Hole](http://dlaa.me/blog/post/skyhole)
+- [Pi-hole in the Cloud!](http://blog.codybunch.com/2015/07/28/Pi-Hole-in-the-cloud/)
+- [unRaid-hole](https://github.com/spants/unraidtemplates/blob/master/Spants/unRaid-hole.xml#L13)--[Repo and more info](http://lime-technology.com/forum/index.php?PHPSESSID=c0eae3e5ef7e521f7866034a3336489d&topic=38486.0)
+- [Pi-hole on/off button](http://thetimmy.silvernight.org/pages/endisbutton/)
 
 ## Coverage
+- [Medium: Block All Ads For $53](https://medium.com/@robleathern/block-ads-on-all-home-devices-for-53-18-a5f1ec139693#.gj1xpgr5d)
+- [MakeUseOf: Adblock Everywhere, The Pi-hole Way](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/)
+- [Lifehacker: Turn Your Pi Into An Ad Blocker With A Single Command](http://lifehacker.com/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-co-1686093533)!
+- [Pi-hole on TekThing](https://youtu.be/8Co59HU2gY0?t=2m)
+- [Pi-hole on Security Now! Podcast](http://www.youtube.com/watch?v=p7-osq_y8i8&t=100m26s)
+- [Foolish Tech Show](https://youtu.be/bYyena0I9yc?t=2m4s)
+- [Pi-hole on Ubuntu](http://www.boyter.org/2015/12/pi-hole-ubuntu-14-04/)
+- [Catchpoint: iOS 9 Ad Blocking](http://blog.catchpoint.com/2015/09/14/ad-blocking-apple/)
 
-### Security Now! Podcast
+## Partnering With Optimal.com
-Pi-hole is mentioned at 100 minutes and 26 seconds (the link brings you right there)
-[![Pi-hole on Security Now!](http://img.youtube.com/vi/p7-osq_y8i8/0.jpg)](http://www.youtube.com/watch?v=p7-osq_y8i8&t=100m26s)
 
-### Tech Blogs
+Pi-hole will be teaming up with [Rob Leathern's subscription service to avoid ads](https://medium.com/@robleathern/block-ads-on-all-home-devices-for-53-18-a5f1ec139693#.gj1xpgr5d).  This service is unique and will help content-creators and publishers [still make money from visitors who are using an ad ablocker](http://techcrunch.com/2015/12/17/the-new-optimal/).
-
-Featured on [MakeUseOf](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/) and [Lifehacker](http://lifehacker.com/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-co-1686093533)!
 
 ## Technical Details
 
+The Pi-hole is an **advertising-aware DNS/Web server**.  If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement.  You can also [replace ads with any image you want](http://pi-hole.net/faq/is-it-possible-to-change-the-blank-page-that-takes-place-of-the-ads-to-something-else/) since it is just a simple Webpage taking place of the ads.
+
 A more detailed explanation of the installation can be found [here](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0).
 
 ## Gravity
-The [gravity.sh](https://github.com/jacobsalmela/pi-hole/blob/master/gravity.sh) does most of the magic.  The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0).
+The [gravity.sh](https://github.com/jacobsalmela/pi-hole/blob/master/gravity.sh) does most of the magic.  The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0) (if you decide to use the [mahakala list](https://github.com/jacobsalmela/pi-hole/commit/963eacfe0537a7abddf30441c754c67ca1e40965)).
 
 ## Whitelist and blacklist
-You can add a `whitelist.txt` or `blacklist.txt` in `/etc/pihole/` and the script will apply those files automatically.
+You can add a `whitelist.txt` or `blacklist.txt` in `/etc/pihole/` and the script will apply those files automatically.  Put one domain per line.
 
 ## Web Interface
 The [Web interface](https://github.com/jacobsalmela/AdminLTE#pi-hole-admin-dashboard) will be installed automatically so you can view stats and change settings.  You can find it at:
 
 `http://192.168.1.x/admin/index.php`
 
+### API
+
+A basic read-only API can be accessed at `/admin/api.php`. It returns the following JSON:
+```JSON
+{
+	"domains_being_blocked": "136708",
+	"dns_queries_today": "18108",
+	"ads_blocked_today": "14648",
+	"ads_percentage_today": 80.892423238348
+}
+```
+
 ![Web](http://i.imgur.com/m114SCn.png)
 
-##  Custom Config File
+## Real-time Statistics
-If you want to use your own variables for the gravity script (i.e. storing the files in a different location) and don't want to have to change them every time there is an update to the script, create a file called `/etc/pihole/pihole.conf`. In it, you should add your own variables in a similar fashion as shown below:
 
-```
+You can view [real-time stats](http://pi-hole.net/faq/install-the-real-time-lcd-monitor-chronometer/) via `ssh` or on an [2.8" LCD screen](http://amzn.to/1P0q1Fj).  This is accomplished via [`chronometer.sh`](https://github.com/jacobsalmela/pi-hole/blob/master/advanced/Scripts/chronometer.sh).
-piholeDir=/var/run/pihole
-adList=/etc/dnsmasq.d/adList
-```
 
-See the [Wiki](https://github.com/jacobsalmela/pi-hole/wiki/Customization) entry for more details.
+![Pi-hole LCD](http://i.imgur.com/nBEqycp.jpg)
 
-### How It Works
+## Help
-A technical and detailed description can be found [here](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0)!
+- See the [Wiki](https://github.com/jacobsalmela/pi-hole/wiki/Customization) entry for more details
+- There is also an [FAQ section on pi-hole.net](http://pi-hole.net)
 
 ## Other Operating Systems
 This script will work for other UNIX-like systems with some slight **modifications**.  As long as you can install `dnsmasq` and a Webserver, it should work OK.  The automated install only works for a clean install of Raspiban right now since that is how the project originated.
-
-### Examples Of The Pi-hole On Other Operating Systems
-- [Sky-Hole](http://dlaa.me/blog/post/skyhole)
-- [Pi-hole in the Cloud!](http://blog.codybunch.com/2015/07/28/Pi-Hole-in-the-cloud/)
-
-[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif "AdminLTE Presentation")](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY "Donate")

advanced/pihole.cron

@@ -19,8 +19,8 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
 # Pi-hole: Parse the log file before it is flushed and save the stats to a database
 #          This will be used for a historical view of your Pi-hole's performance
-#50 11  * * *   root    /usr/local/bin/dailyLog.sh
+#50 23  * * *   root    /usr/local/bin/dailyLog.sh
 
 # Pi-hole: Flush the log daily at 11:58 so it doesn't get out of control
 #          Stats will be viewable in the Web interface thanks to the cron job above
-58 11   * * *   root    /usr/local/bin/piholeLogFlush.sh
+58 23   * * *   root    /usr/local/bin/piholeLogFlush.sh

automated install/basic-install.sh

@@ -20,12 +20,9 @@
 tmpLog=/tmp/pihole-install.log
 instalLogLoc=/etc/pihole/install.log
 
-# Get the screen size in case we need a full-screen message and so we can display a dialog that is sized nicely
-screenSize=$(stty -a | tr \; \\012 | egrep 'rows|columns' | cut '-d ' -f3)
-
 # Find the rows and columns
-rows=$(stty -a | tr \; \\012 | egrep 'rows' | cut -d' ' -f3)
+rows=$(tput lines)
-columns=$(stty -a | tr \; \\012 | egrep 'columns' | cut -d' ' -f3)
+columns=$(tput cols)
 
 # Divide by two so the dialogs take up half of the screen, which looks nice.
 r=$(( rows / 2 ))
@@ -36,10 +33,6 @@ IPv4dev=$(ip route get 8.8.8.8 | awk '{for(i=1;i<=NF;i++)if($i~/dev/)print $(i+1
 IPv4addr=$(ip -o -f inet addr show dev $IPv4dev | awk '{print $4}' | awk 'END {print}')
 IPv4gw=$(ip route get 8.8.8.8 | awk '{print $3}')
 
-# IPv6 support to be added later
-#IPv6eui64=$(ip addr show | awk '/scope\ global/ && /ff:fe/ {print $2}' | cut -d'/' -f1)
-#IPv6linkLocal=$(ip addr show | awk '/inet/ && /scope\ link/ && /fe80/ {print $2}' | cut -d'/' -f1)
-
 availableInterfaces=$(ip -o link | awk '{print $2}' | grep -v "lo" | cut -d':' -f1)
 dhcpcdFile=/etc/dhcpcd.conf
 
@@ -65,6 +58,9 @@ welcomeDialogs()
 # Display the welcome dialog
 whiptail --msgbox --backtitle "Welcome" --title "Pi-hole automated installer" "This installer will transform your Raspberry Pi into a network-wide ad blocker!" $r $c
 
+# Support for a part-time dev
+whiptail --msgbox --backtitle "Plea" --title "Free and open source" "The Pi-hole is free, but powered by your donations:  http://pi-hole.net/donate" $r $c
+
 # Explain the need for a static address
 whiptail --msgbox --backtitle "Initating network interface" --title "Static IP Needed" "The Pi-hole is a SERVER so it needs a STATIC IP ADDRESS to function properly.
 
@@ -75,9 +71,16 @@ chooseInterface()
 {
 # Turn the available interfaces into an array so it can be used with a whiptail dialog
 interfacesArray=()
+firstloop=1
+
 while read -r line
 do
-interfacesArray+=("$line" "available" "ON")
+mode="OFF"
+if [[ $firstloop -eq 1 ]]; then
+  firstloop=0
+  mode="ON"
+fi
+interfacesArray+=("$line" "available" "$mode")
 done <<< "$availableInterfaces"
 
 # Find out how many interfaces are available to choose from
@@ -97,7 +100,7 @@ use4andor6()
 # Let use select IPv4 and/or IPv6
 cmd=(whiptail --separate-output --checklist "Select Protocols" $r $c 2)
 options=(IPv4 "Block ads over IPv4" on
-         IPv6 "Block ads over IPv4" off)
+         IPv6 "Block ads over IPv6" off)
 choices=$("${cmd[@]}" "${options[@]}" 2>&1 >/dev/tty)
 for choice in $choices
 do
@@ -116,7 +119,10 @@ done
 
 useIPv6dialog()
 {
-whiptail --msgbox --backtitle "Coming soon..." --title "IPv6 not yet supported" "I need your help for IPv6.  Consider donating at: http://pi-hole.net/donate" $r $c
+piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }')
+whiptail --msgbox --backtitle "IPv6..." --title "IPv6 Supported" "$piholeIPv6 will be used to block ads." $r $c
+sudo mkdir -p /etc/pihole/
+sudo touch /etc/pihole/.useIPv6
 }
 
 getStaticIPv4Settings()
@@ -263,6 +269,17 @@ installCron
 sudo /usr/local/bin/gravity.sh
 }
 
+displayFinalMessage(){
+	whiptail --msgbox --backtitle "Make it so." --title "Installation Complete!" "Configure your devices to use the Pi-hole as their DNS server using:
+
+						$IPv4addr
+						$piholeIPv6
+
+If you set a new IP address, you should restart the Pi.
+
+The install log is in /etc/pihole." $r $c
+}
+
 ######## SCRIPT ############
 # Start the installer
 welcomeDialogs
@@ -288,16 +305,12 @@ fi
 
 # Decide is IPv6 will be used
 if [[ "$useIPv6" = true ]];then
-	# If only IPv6 is selected, exit because it is not supported yet
-	if [[ "$useIPv6" = true ]] && [[ "$useIPv4" = false ]];then
 	useIPv6dialog
-		exit
+	echo "Using IPv6."
-	else
+	echo "Your IPv6 address is: $piholeIPv6"
-		useIPv6dialog
-	fi
 else
 	useIPv6=false
-	echo "IPv6 will NOT be used.  Consider a donation at pi-hole.net/donate"
+	echo "IPv6 will NOT be used."
 fi
 
 # Install and log everything to a file
@@ -306,11 +319,7 @@ installPihole | tee $tmpLog
 # Move the log file into /etc/pihole for storage
 sudo mv $tmpLog $instalLogLoc
 
-whiptail --msgbox --backtitle "Make it so." --title "Installation Complete!" "Configure your devices to use the Pi-hole as their DNS server using this IP: $IPv4addr.
+displayFinalMessage
-
-If you set a new IP address, it should work fine, but you may want to reboot the Pi at some point.
-
-The install log is in /etc/pihole." $r $c
 
 sudo service dnsmasq start
 sudo service lighttpd start

automated install/webinterface.sh

@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+#
+# this script will update the pihole web interface files.
+#
+# if this is the first time running this script after an 
+# existing installation, the existing web interface files
+# will be removed and replaced with the latest master
+# branch from github. subsequent executions of this script
+# will pull the latest version of the web interface.
+#
+# @TODO: add git as requirement to basic-install.sh
+#
+
+WEB_INTERFACE_GIT_URL="https://github.com/jacobsalmela/AdminLTE.git"
+WEB_INTERFACE_DIR="/var/www/html/admin"
+
+main() {
+    prerequisites
+    if ! is_repo; then
+        make_repo
+    fi
+    update_repo
+}
+
+prerequisites() {
+
+    # must be root to update
+    if [[ $EUID -ne 0 ]]; then
+        sudo bash "$0" "$@"
+        exit $?
+    fi
+
+    # web interface must already exist. this is a (lazy)
+    # check to make sure pihole is actually installed.
+    if [ ! -d "$WEB_INTERFACE_DIR" ]; then
+        echo "$WEB_INTERFACE_DIR not found. Exiting."
+        exit 1
+    fi
+
+    if ! type "git" > /dev/null; then
+        apt-get -y install git
+    fi
+}
+
+is_repo() {
+    # if the web interface directory does not have a .git folder
+    # it means its using the master.zip archive from the install
+    # script.
+    if [ ! -d "$WEB_INTERFACE_DIR/.git" ]; then
+        return 1
+    fi
+    return 0
+}
+
+# removes the web interface installed from the master.zip archive and
+# replaces it with the current master branch from github
+make_repo() {
+    # remove the non-repod interface and clone the interface
+    rm -rf $WEB_INTERFACE_DIR
+    git clone "$WEB_INTERFACE_GIT_URL" "$WEB_INTERFACE_DIR"
+}
+
+# pulls the latest master branch from github
+update_repo() {
+    # pull the latest commits
+    cd "$WEB_INTERFACE_DIR"
+    git pull    
+}
+
+main

gravity.sh

@@ -11,6 +11,7 @@
 # (at your option) any later version.
 
 piholeIPfile=/tmp/piholeIP
+piholeIPv6file=/etc/pihole/.useIPv6
 if [[ -f $piholeIPfile ]];then
     # If the file exists, it means it was exported from the installation script and we should use that value instead of detecting it in this script
     piholeIP=$(cat $piholeIPfile)
@@ -22,6 +23,11 @@ else
     piholeIP=${piholeIPCIDR%/*}
 fi
 
+if [[ -f $piholeIPv6file ]];then
+    # If the file exists, then the user previously chose to use IPv6 in the automated installer
+    piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }')
+fi
+
 # Ad-list sources--one per line in single quotes
 # The mahakala source is commented out due to many users having issues with it blocking legitimate domains.
 # Uncomment at your own risk
@@ -183,7 +189,7 @@ function gravity_pulsar() {
         # regexp so it can be parsed out with grep -x
         awk -F '[# \t]' 'NF>0&&$1!="" {print "^"$1"$"}' $whitelist | sed 's/\./\\./g' > $latentWhitelist
 	else
-        rm $latentWhitelist >/dev/null
+        rm $latentWhitelist 2>/dev/null
 	fi
 
 	# Prevent our sources from being pulled into the hole
@@ -208,7 +214,13 @@ function gravity_unique() {
 function gravity_hostFormat() {
 	# Format domain list as "192.168.x.x domain.com"
 	echo "** Formatting domains into a HOSTS file..."
-	cat $piholeDir/$eventHorizon | awk '{sub(/\r$/,""); print "'"$piholeIP"' " $0}' > $piholeDir/$accretionDisc
+  # If there is a value in the $piholeIPv6, then IPv6 will be used, so the awk command modified to create a line for both protocols
+  if [[ -n $piholeIPv6 ]];then
+    cat $piholeDir/$eventHorizon | awk -v ipv4addr="$piholeIP" -v ipv6addr="$piholeIPv6" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' > $piholeDir/$accretionDisc
+  else
+    # Otherwise, just create gravity.list as normal using IPv4
+    cat $piholeDir/$eventHorizon | awk -v ipv4addr="$piholeIP" '{sub(/\r$/,""); print ipv4addr" "$0}' > $piholeDir/$accretionDisc
+  fi
 	# Copy the file over as /etc/pihole/gravity.list so dnsmasq can use it
 	cp $piholeDir/$accretionDisc $adList
 }
@@ -239,8 +251,21 @@ function gravity_advanced() {
 	echo "** $numberOf domains being pulled in by gravity..."
 
 	gravity_unique
+}
 
-	sudo kill -s -HUP $(pidof dnsmasq)
+function gravity_reload() {
+	# Reload hosts file
+	echo "** Refresh lists in dnsmasq..."
+
+	dnsmasqPid=$(pidof dnsmasq)
+
+	if [[ $dnsmasqPid ]]; then
+		# service already running - reload config
+		sudo kill -HUP $dnsmasqPid
+	else
+		# service not running, start it up
+		sudo service dnsmasq start
+	fi
 }
 
 gravity_collapse
@@ -250,3 +275,4 @@ gravity_pulsar
 gravity_advanced
 gravity_hostFormat
 gravity_blackbody
+gravity_reload