Merge pull request #4 from pi-hole/development

Development

README.md

@@ -68,12 +68,11 @@ Sending a donation using our links below is **extremely helpful** in offsetting
 
 ### Alternative support
 If you'd rather not [donate](https://pi-hole.net/donate/) (_which is okay!_), there are other ways you can help support us:
-
+- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
 - [Digital Ocean](http://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
 - [UNIXstickers.com](http://unixstickers.refr.cc/jacobs) _save $5 when you spend $9 using our affiliate link_
 - [Pi-hole Swag Store](https://pi-hole.net/shop/) _affiliate link_
 - [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
-- [Ho-ost](https://clients.ho-ost.com/aff.php?aff=19) _save 50% with our affiliate link_
 - [DNS Made Easy](https://cp.dnsmadeeasy.com/u/133706) _affiliate link_
 - [Vultr](http://www.vultr.com/?ref=7190426) _affiliate link_
 - Spreading the word about our software, and how you have benefited from it
@@ -99,9 +98,6 @@ While we are primarily reachable on our <a href="https://discourse.pi-hole.net/"
   <li><a href="https://discourse.pi-hole.net/c/faqs">Frequently Asked Questions</a></li>
   <li><a href="https://github.com/pi-hole/pi-hole/wiki">Pi-hole Wiki</a></li>
   <li><a href="https://discourse.pi-hole.net/c/feature-requests?order=votes">Feature Requests</a></li>
-</ul>
-<br/>
-<ul>
   <li><a href="https://discourse.pi-hole.net/">Discourse User Forum</a></li>
   <li><a href="https://www.reddit.com/r/pihole/">Reddit</a></li>
   <li><a href="https://gitter.im/pi-hole/pi-hole">Gitter</a> (Real-time chat)</li>
@@ -178,32 +174,13 @@ Pi-hole being a **advertising-aware DNS/Web server**, makes use of the following
 * [AdminLTE Dashboard](https://github.com/almasaeed2010/AdminLTE) - premium admin control panel based on Bootstrap 3.x
 
 While quite outdated at this point, [this original blog post about Pi-hole](https://jacobsalmela.com/2015/06/16/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0/) goes into **great detail** about how Pi-hole was originally setup and how it works. Syntactically, it's no longer accurate, but the same basic principles and logic still apply to Pi-hole's current state.
-
------
-
-## Pi-hole Projects
-- [The Big Blocklist Collection](https://wally3k.github.io)
-- [Docker Pi-hole container (x86 and ARM)](https://hub.docker.com/r/diginc/pi-hole/)
-- [Pi-Hole in the cloud](http://blog.codybunch.com/2015/07/28/Pi-Hole-in-the-cloud/)
-- [Pie in the Sky-Hole [A Pi-Hole in the cloud for ad-blocking via DNS]](https://dlaa.me/blog/post/skyhole)
-- [Pi-hole Enable/Disable Button](http://thetimmy.silvernight.org/pages/endisbutton/)
-- [Minibian Pi-hole](https://munkjensen.net/wiki/index.php/See_my_Pi-Hole#Minibian_Pi-hole)
-- [CHiP-hole: Network-wide Ad-blocker](https://www.hackster.io/jacobsalmela/chip-hole-network-wide-ad-blocker-98e037)
-- [Chrome Extension: Pi-Hole List Editor](https://chrome.google.com/webstore/detail/pi-hole-list-editor/hlnoeoejkllgkjbnnnhfolapllcnaglh) ([Source Code](https://github.com/packtloss/pihole-extension))
-- [Splunk: Pi-hole Visualiser](https://splunkbase.splunk.com/app/3023/)
-- [Adblocking with Pi-hole and Ubuntu 14.04 on VirtualBox](https://hbalagtas.blogspot.com.au/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
-- [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py)
-- [Pi-hole unRAID Template](https://forums.lime-technology.com/topic/36810-support-spants-nodered-mqtt-dashing-couchdb/)
-- [Copernicus: Windows Tray Application](https://github.com/goldbattle/copernicus)
-- [Let your blink1 device blink when Pi-hole filters ads](https://gist.github.com/elpatron68/ec0b4c582e5abf604885ac1e068d233f)
-- [Pi-hole metrics](https://github.com/nlamirault/pihole_exporter) exporter for [Prometheus](https://prometheus.io/)
-- [Magic Mirror with DNS Filtering](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)
-- [Pi-hole Droid: Android client](https://github.com/friimaind/pi-hole-droid)
-- [Windows DNS Swapper](https://github.com/roots84/DNS-Swapper), see [#1400](https://github.com/pi-hole/pi-hole/issues/1400)
-- [Pi-hole Visualizer](https://www.reddit.com/r/pihole/comments/82ikgb/pihole_visualizer_update/)
 -----
 
 ## Coverage
+- [Software Engineering Daily: Interview with the creator of Pi-hole](https://softwareengineeringdaily.com/2018/05/29/pi-hole-ad-blocker-hardware-with-jacob-salmela/)
+- [Bloomberg Business Week: Brotherhood of the Ad blockers](https://www.bloomberg.com/news/features/2018-05-10/inside-the-brotherhood-of-pi-hole-ad-blockers)
+- [Securing DNS across all of my devices with Pi-Hole + DNS-over-HTTPS + 1.1.1.1](https://scotthelme.co.uk/securing-dns-across-all-of-my-devices-with-pihole-dns-over-https-1-1-1-1/)
+- [Adafruit: installing Pi-hole on a Pi Zero W](https://learn.adafruit.com/pi-hole-ad-blocker-with-pi-zero-w/install-pi-hole)
 - [Lifehacker: Turn A Raspberry Pi Into An Ad Blocker With A Single Command](https://www.lifehacker.com.au/2015/02/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-command/)
 - [MakeUseOf: Adblock Everywhere: The Raspberry Pi-Hole Way](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/)
 - [Catchpoint: Ad-Blocking on Apple iOS9: Valuing the End User Experience](http://blog.catchpoint.com/2015/09/14/ad-blocking-apple/)
@@ -222,3 +199,12 @@ While quite outdated at this point, [this original blog post about Pi-hole](http
 - [CryptoAUSTRALIA: How We Tried 5 Privacy Focused Raspberry Pi Projects](https://blog.cryptoaustralia.org.au/2017/10/05/5-privacy-focused-raspberry-pi-projects/)
 - [CryptoAUSTRALIA: Pi-hole Workshop](https://blog.cryptoaustralia.org.au/2017/11/02/pi-hole-network-wide-ad-blocker/)
 - [Know How 355: Killing ads with a Raspberry Pi-Hole!](https://www.twit.tv/shows/know-how/episodes/355)
+
+-----
+
+## Pi-hole Projects
+- [The Big Blocklist Collection](https://wally3k.github.io)
+- [Pie in the Sky-Hole](https://dlaa.me/blog/post/skyhole)
+- [Copernicus: Windows Tray Application](https://github.com/goldbattle/copernicus)
+- [Magic Mirror with DNS Filtering](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)
+- [Windows DNS Swapper](https://github.com/roots84/DNS-Swapper)

automated install/basic-install.sh

@@ -977,7 +977,7 @@ setLogging() {
   local LogChoices
 
   # Ask if the user wants to log queries
-  LogToggleCommand=(whiptail --separate-output --radiolist "Do you want to log queries?\\n (Disabling will render graphs on the Admin page useless):" ${r} ${c} 6)
+  LogToggleCommand=(whiptail --separate-output --radiolist "Do you want to log queries?" "${r}" "${c}" 6)
   # The default selection is on
   LogChooseOptions=("On (Recommended)" "" on
       Off "" off)
@@ -1251,7 +1251,7 @@ installConfigs() {
 }
 
 install_manpage() {
-  # Copy Pi-hole man page and call mandb to update man page database
+  # Copy Pi-hole man pages and call mandb to update man page database
   # Default location for man files for /usr/local/bin is /usr/local/share/man
   # on lightweight systems may not be present, so check before copying.
   echo -en "  ${INFO} Testing man page installation"
@@ -1261,23 +1261,30 @@ install_manpage() {
     return
   elif [[ ! -d "/usr/local/share/man" ]]; then
     # appropriate directory for Pi-hole's man page is not present
-    echo -e "${OVER}  ${INFO} man page not installed"
+    echo -e "${OVER}  ${INFO} man pages not installed"
     return
-  elif [[ ! -d "/usr/local/share/man/man8" ]]; then
+  fi
+  if [[ ! -d "/usr/local/share/man/man8" ]]; then
     # if not present, create man8 directory
     mkdir /usr/local/share/man/man8
   fi
-  # Testing complete, copy the file & update the man db
+  if [[ ! -d "/usr/local/share/man/man5" ]]; then
-  cp ${PI_HOLE_LOCAL_REPO}/pihole.8 /usr/local/share/man/man8/pihole.8
+    # if not present, create man8 directory
+    mkdir /usr/local/share/man/man5
+  fi
+  # Testing complete, copy the files & update the man db
+  cp ${PI_HOLE_LOCAL_REPO}/manpages/pihole.8 /usr/local/share/man/man8/pihole.8
+  cp ${PI_HOLE_LOCAL_REPO}/manpages/pihole-FTL.8 /usr/local/share/man/man8/pihole-FTL.8
+  cp ${PI_HOLE_LOCAL_REPO}/manpages/pihole-FTL.conf.5 /usr/local/share/man/man5/pihole-FTL.conf.5
   if mandb -q &>/dev/null; then
     # Updated successfully
-    echo -e "${OVER}  ${TICK} man page installed and database updated"
+    echo -e "${OVER}  ${TICK} man pages installed and database updated"
     return
   else
     # Something is wrong with the system's man installation, clean up 
-    # our file, (leave everything how we found it).
+    # our files, (leave everything how we found it).
-    rm /usr/local/share/man/man8/pihole.8
+    rm /usr/local/share/man/man8/pihole.8 /usr/local/share/man/man8/pihole-FTL.8 /usr/local/share/man/man5/pihole-FTL.conf.5
-    echo -e "${OVER}  ${CROSS} man page db not updated, man page not installed"
+    echo -e "${OVER}  ${CROSS} man page db not updated, man pages not installed"
   fi
 }
 

automated install/uninstall.sh

@@ -173,7 +173,7 @@ removeNoPurge() {
 
     # If the pihole manpage exists, then delete and rebuild man-db
     if [[ -f /usr/local/share/man/man8/pihole.8 ]]; then
-        ${SUDO} rm -f /usr/local/share/man/man8/pihole.8
+        ${SUDO} rm -f /usr/local/share/man/man8/pihole.8 /usr/local/share/man/man8/pihole-FTL.8 /usr/local/share/man/man5/pihole-FTL.conf.5
         ${SUDO} mandb -q &>/dev/null
         echo -e "  ${TICK} Removed pihole man page"
     fi

manpages/pihole-FTL.8

@@ -0,0 +1,112 @@
+.TH "Pihole-FTL" "8" "pihole-FTL" "Pi-hole" "June 2018"
+.SH "NAME"
+pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
+.br
+.SH "SYNOPSIS"
+\fBservice pihole-FTL \fR(\fBstart\fR|\fBstop\fR|\fBrestart\fR)
+.br
+
+\fBpihole-FTL debug\fR
+.br
+\fBpihole-FTL test\fR
+.br
+\fBpihole-FTL -v\fR
+.br
+\fBpihole-FTL -t\fR
+.br
+\fBpihole-FTL -b\fR
+.br
+\fBpihole-FTL -f\fR
+.br
+\fBpihole-FTL -h\fR
+.br
+\fBpihole-FTL dnsmasq-test\fR
+.br
+\fBpihole-FTL --\fR (\fBoptions\fR)
+.br
+
+.SH "DESCRIPTION"
+Pi-hole : The Faster-Than-Light (FTL) Engine is a lightweight, purpose-built daemon used to provide statistics needed for the Pi-hole Web Interface, and its API can be easily integrated into your own projects. Although it is an optional component of the Pi-hole ecosystem, it will be installed by default to provide statistics. As the name implies, FTL does its work \fIvery\fR \fIquickly\fR!
+.br
+
+Usage
+.br
+
+\fBservice pihole-FTL start\fR
+.br
+    Start the pihole-FTL daemon
+.br
+
+\fBservice pihole-FTL stop\fR
+.br
+    Stop the pihole-FTL daemon
+.br
+
+\fBservice pihole-FTL restart\fR
+.br
+    If the pihole-FTP daemon is running, stop and then start, otherwise start.
+.br
+
+Command line arguments
+.br
+
+\fBdebug\fR
+.br
+    Don't go into daemon mode (stay in foreground) + more verbose logging
+.br
+
+\fBtest\fR
+.br
+    Start FTL and process everything, but shut down immediately afterwards
+.br
+
+\fB-v, version\fR
+.br
+    Don't start FTL, show only version
+.br
+
+\fB-t, tag\fR
+.br
+    Don't start FTL, show only git tag
+.br
+
+\fB-b, branch\fR
+.br
+    Don't start FTL, show only git branch FTL was compiled from
+.br
+
+\fB-f, no-daemon\fR
+.br
+    Don't go into background (daemon mode)
+.br
+
+\fB-h, help\fR
+.br
+    Don't start FTL, show help
+.br
+
+\fBdnsmasq-test\fR
+.br
+    Test resolver config file syntax
+.br
+
+\fB--\fR  (options)
+.br
+    Pass options to internal dnsmasq resolver
+.br
+.SH "EXAMPLE"
+Command line arguments can be arbitrarily combined, e.g:
+.br
+
+\fBpihole-FTL debug test\fR
+.br
+
+Start ftl in foreground with more verbose logging, process everything and shutdown immediately
+.br
+.SH "SEE ALSO"
+\fBpihole\fR(8), \fBpihole-FTL.conf\fR(5)
+.br
+.SH "COLOPHON"
+
+Get sucked into the latest news and community activity by entering Pi-hole's orbit. Information about Pi-hole, and the latest version of the software can be found at https://pi-hole.net
+.br

manpages/pihole-FTL.conf.5

@@ -0,0 +1,102 @@
+.TH "pihole-FTL.conf" "5" "pihole-FTL.conf" "pihole-FTL.conf" "June 2018"
+.SH "NAME"
+
+pihole-FTL.conf - FTL's config file
+.br
+.SH "DESCRIPTION"
+
+/etc/pihole/pihole-FTL.conf will be read by \fBpihole-FTL(8)\fR on startup.
+.br
+
+\fBSOCKET_LISTENING=localonly|all\fR
+.br
+    Listen only for local socket connections or permit all connections
+.br
+
+\fBQUERY_DISPLAY=yes|no\fR
+.br
+    Display all queries? Set to no to hide query display
+.br
+
+\fBAAAA_QUERY_ANALYSIS=yes|no\fR
+.br
+    Allow FTL to analyze AAAA queries from pihole.log?
+.br
+
+\fBRESOLVE_IPV6=yes|no\fR
+.br
+    Should FTL try to resolve IPv6 addresses to host names?
+.br
+
+\fBRESOLVE_IPV4=yes|no\fR
+.br
+    Should FTL try to resolve IPv4 addresses to host names?
+.br
+
+\fBMAXDBDAYS=365\fR
+.br
+    How long should queries be stored in the database?
+.br
+    Setting this to 0 disables the database
+.br
+
+\fBDBINTERVAL=1.0\fR
+.br
+    How often do we store queries in FTL's database [minutes]?
+.br
+
+\fBDBFILE=/etc/pihole/pihole-FTL.db\fR
+.br
+    Specify path and filename of FTL's SQLite long-term database.
+.br
+    Setting this to DBFILE= disables the database altogether
+.br
+
+\fBMAXLOGAGE=24.0\fR
+.br
+    Up to how many hours of queries should be imported from the database and logs?
+.br
+    Maximum is 744 (31 days)
+.br
+
+\fBFTLPORT=4711\fR
+.br
+    On which port should FTL be listening?
+.br
+
+\fBPRIVACYLEVEL=0|1|2|3\fR
+.br
+    Which privacy level is used?
+.br
+    0 - show everything
+.br
+    1 - hide domains
+.br
+    2 - hide domains and clients
+.br
+    3 - paranoia mode (hide everything)
+.br
+
+\fBIGNORE_LOCALHOST=no|yes\fR
+.br
+    Should FTL ignore queries coming from the local machine?
+.br
+
+\fBBLOCKINGMODE=IP|IP-AAAA-NODATA|NXDOMAIN|NULL\fR
+.br
+    How should FTL reply to blocked queries?
+.br
+
+For each setting, the option shown first is the default.
+.br
+.SH "SEE ALSO"
+
+\fBpihole\fR(8), \fBpihole-FTL\fR(8)
+.br
+.SH "COLOPHON"
+
+Pi-hole : The Faster-Than-Light (FTL) Engine is a lightweight, purpose-built daemon used to provide statistics needed for the Pi-hole Web Interface, and its API can be easily integrated into your own projects. Although it is an optional component of the Pi-hole ecosystem, it will be installed by default to provide statistics. As the name implies, FTL does its work \fIvery quickly\fR!
+.br
+
+Get sucked into the latest news and community activity by entering Pi-hole's orbit. Information about Pi-hole, and the latest version of the software can be found at https://pi-hole.net
+.br

manpages/pihole.8

@@ -21,13 +21,13 @@ Pi-hole : A black-hole for internet advertisements
 .br
 \fBpihole -c\fR [-j|-r|-e]
 .br
-\fBpihole\fR \fB-d\fR [-a] 
+\fBpihole\fR \fB-d\fR [-a]
 .br
 \fBpihole -f
 .br
 pihole -r
 .br
-pihole -t 
+pihole -t
 .br
 pihole -g\fR
 .br
@@ -84,9 +84,9 @@ Available commands and options:
       --nuke            Removes all entries in a list
 .br
 
-\fB-d, debug\fR [-a]	
+\fB-d, debug\fR [-a]
 .br
-    Start a debugging session 
+    Start a debugging session
 .br
 
       -a                Enable automated debugging
@@ -96,8 +96,7 @@ Available commands and options:
 .br
     Flush the Pi-hole log
 .br
-		
+
-.br
 \fB-r, reconfigure\fR
 .br
     Reconfigure or Repair Pi-hole subsystems
@@ -129,11 +128,10 @@ Available commands and options:
 .br
       -l, privacylevel  <level> Set privacy level (0 = lowest, 3 = highest)
 .br
-		
+
-.br
 \fB-c, chronometer\fR	[options]
 .br
-    Calculates stats and displays to an LCD 
+    Calculates stats and displays to an LCD
 .br
 
     (Chronometer Options):
@@ -145,12 +143,12 @@ Available commands and options:
       -e, --exit        Output stats and exit witout refreshing
 .br
 
-\fB-g, updateGravity\fR	
+\fB-g, updateGravity\fR
 .br
     Update the list of ad-serving domains
 .br
 
-\fB-q, query\fR [option]	
+\fB-q, query\fR [option]
 .br
     Query the adlists for a specified domain
 .br
@@ -164,7 +162,7 @@ Available commands and options:
       -all              Return all query matches within a block list
 .br
 
-\fB-h, --help, help\fR	
+\fB-h, --help, help\fR
 .br
     Show a help dialog
 .br
@@ -178,21 +176,20 @@ Available commands and options:
 .br
       on                Enable the Pi-hole log at /var/log/pihole.log
 .br
-      off 		        Disable and flush the Pi-hole log at /var/log/pihole.log
+      off               Disable and flush the Pi-hole log at /var/log/pihole.log
 .br
       off noflush       Disable the Pi-hole log at /var/log/pihole.log
 .br
-	
+
+\fB-up, updatePihole\fR [--check-only]
 .br
-\fB-up, updat\fBe\fR\fR\fBPihole\fR [--check-only]	
+    Update Pi-hole subsystems
-.br
-    Update Pi-hole subsystems 
 .br
 
       --check-only      Exit script before update is performed.
 .br
 
-\fB-v, version\fR [repo] [options]	
+\fB-v, version\fR [repo] [options]
 .br
     Show installed versions of Pi-hole, Web Interface &amp; FTL
 .br
@@ -234,8 +231,7 @@ Available commands and options:
 .br
     Disable Pi-hole subsystems, optionally for a set duration
 .br
-		
+
-.br
     (time options):	
 .br
       #s                Disable Pi-hole functionality for # second(s)
@@ -248,7 +244,7 @@ Available commands and options:
     Restart Pi-hole subsystems
 .br
 
-\fBcheckout\fR [repo] [branch]	
+\fBcheckout\fR [repo] [branch]
 .br
     Switch Pi-hole subsystems to a different Github branch
 .br
@@ -281,50 +277,43 @@ Some usage examples
 .br
     \fBpihole -b -d noads.example.com\fR  Remove "noads.example.com" from blacklist 
 .br
-    \fBpihole -wild example.com\fR            Add "example.com" as wildcard - would block ads.example.com, www.example.com etc.
+    \fBpihole -wild example.com\fR        Add "example.com" as wildcard - would block ads.example.com, www.example.com etc.
 .br
 
-    Changing the web ui password
+    Changing the Web Interface password
-.br
-	
 .br
+
     \fBpihole -a -p ExamplePassword\fR    Change the password to "ExamplePassword"
 .br
 
     Updating lists from internet sources
 .br
-	
+
-.br
     \fBpihole -g\fR                       Update the list of ad-serving domains
 .br
 
     Displaying version information
 .br
-	
+
-.br
     \fBpihole -v -a -c\fR                 Display the current version of AdminLTE
 .br
-	
+
-.br
     Temporarily disabling Pi-hole
 .br
-	
+
-.br
     \fBpihole disable 5m\fR               Disable Pi-hole functionality for five minutes
 .br
-	
+
-.br
     Switching Pi-hole subsystem branches
 .br
-	
+
-.br
     \fBpihole checkout master\fR          Switch to master branch
 .br
     \fBpihole checkout core dev\fR        Switch to core development branch
 .br
 .SH "SEE ALSO"
 
-dnsmasq(8), lighttpd(8)
+\fBlighttpd\fR(8), \fBpihole-FTL\fR(8)
 .br
 .SH "COLOPHON"
 

pihole

@@ -541,12 +541,13 @@ Switch Pi-hole subsystems to a different Github branch
 
 Repositories:
   core [branch]       Change the branch of Pi-hole's core subsystem
-  web [branch]        Change the branch of Admin Console subsystem
+  web [branch]        Change the branch of Web Interface subsystem
   ftl [branch]        Change the branch of Pi-hole's FTL subsystem
 
 Branches:
   master              Update subsystems to the latest stable release
-  dev                 Update subsystems to the latest development release"
+  dev                 Update subsystems to the latest development release
+  branchname          Update subsystems to the specified branchname"
     exit 0
   fi
 
@@ -610,8 +611,8 @@ Debugging Options:
   -t, tail            View the live output of the Pi-hole log
 
 Options:
-  -a, admin           Admin Console options
+  -a, admin           Web interface options
-                        Add '-h' for more info on admin console usage
+                        Add '-h' for more info on Web Interface usage
   -c, chronometer     Calculates stats and displays to an LCD
                         Add '-h' for more info on chronometer usage
   -g, updateGravity   Update the list of ad-serving domains
@@ -622,7 +623,7 @@ Options:
                         Add '-h' for more info on query usage
   -up, updatePihole   Update Pi-hole subsystems
                         Add '--check-only' to exit script before update is performed.
-  -v, version         Show installed versions of Pi-hole, Admin Console & FTL
+  -v, version         Show installed versions of Pi-hole, Web Interface & FTL
                         Add '-h' for more info on version usage
   uninstall           Uninstall Pi-hole from your system
   status              Display the running status of Pi-hole subsystems