diff --git a/README.md b/README.md index 6f8813fa..4a6a86c6 100644 --- a/README.md +++ b/README.md @@ -8,37 +8,37 @@

-## The multi-platform, network-wide ad blocker +## Pi-hole®: The multi-platform, network-wide ad blocker -Block ads for **all** your devices _without_ the need to install client-side software. The Pi-hole™ blocks ads at the DNS-level, so all your devices are protected. - -- Web Browsers -- Cell Phones -- Smart TV's -- Internet-connected home automation -- Anything that communicates with the Internet +Block ads for **all** your devices _without_ the need to install client-side software.

-## Your Support Still Matters +## Executive Summary +The Pi-hole blocks ads at the DNS-level, so all your devices are protected. -Digital Ocean helps with our infrastructure, but our developers are all volunteers so *your donations help keep us innovating*. Sending a donation using our links below helps us offset a portion of our monthly costs. +- **Easy-to-install** - our intelligent installer walks you through the process with no additional software needed on client devices +- **Universal** - ads are blocked in _non-browser locations_ such as ad-supported mobile apps and smart TVs +- **Quick** - installation takes less than ten minutes and it [_really_ is _that easy_](https://discourse.pi-hole.net/t/new-pi-hole-questions/3971/5?u=jacob.salmela) +- **Informative** - an administrative Web interface shows ad-blocking statistics +- **Lightweight** - designed to run on [minimal resources](https://discourse.pi-hole.net/t/hardware-software-requirements/273) +- **Scalable** - even in large environments, [Pi-hole can handle hundreds of millions of queries](https://pi-hole.net/2017/05/24/how-much-traffic-can-pi-hole-handle/) (with the right hardware specs) +- **Powerful** - advertisements are blocked over IPv4 _and_ IPv6 +- **Fast** - it speeds up high-cost, high-latency networks by caching DNS queries and saves bandwidth by not downloading advertisement elements +- **Versatile** - Pi-hole can function also function as a DHCP server -- ![Paypal](https://assets.pi-hole.net/static/paypal.png) [Donate via PayPal](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY) -- ![Bitcoin](https://assets.pi-hole.net/static/Bitcoin.png) Bitcoin Address: 1GKnevUnVaQM2pQieMyeHkpr8DXfkpfAtL - -### One-Step Automated Install +# One-Step Automated Install 1. Install a [supported operating system](https://discourse.pi-hole.net/t/hardware-software-requirements/273/1) 2. Run the command below (it downloads [this script](https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh) in case you want to read over it first!) -### `curl -sSL https://install.pi-hole.net | bash` +#### `curl -sSL https://install.pi-hole.net | bash` -#### Alternative Semi-Automated Install Methods +## Alternative Semi-Automated Install Methods _If you wish to read over the script before running it, run `nano basic-install.sh` to open the file in a text viewer._ -##### Clone our repository and run the automated installer from your device. +### Clone our repository and run the automated installer from your device. ``` git clone --depth 1 https://github.com/pi-hole/pi-hole.git Pi-hole @@ -53,91 +53,233 @@ wget -O basic-install.sh https://install.pi-hole.net bash basic-install.sh ``` -Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to use Pi-hole™ as their DNS server. +Once installed, [configure your router to have **DHCP clients use the Pi-hole as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. -## What is Pi-hole™ and how do I install it? +If your router does not support setting the DNS server, you can [use Pi-hole's built in DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026); just be sure to disable DHCP on your router first. + +Alternatively, you can manually set each device to use Pi-hole as their DNS server. + +# What is Pi-hole and how do I install it?

+# Pi-hole Is Free, But Powered By Your Donations -## Get Help Or Connect With Us On The Web +[Digital Ocean](http://www.digitalocean.com/?refcode=344d234950e1) helps with our infrastructure, but [our developers](https://github.com/orgs/pi-hole/people) are all volunteers so *your donations help keep us innovating*. + +- ![Paypal](https://assets.pi-hole.net/static/paypal.png) [Donate via PayPal](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY) +- ![Bitcoin](https://assets.pi-hole.net/static/Bitcoin.png) Bitcoin Address: 1GKnevUnVaQM2pQieMyeHkpr8DXfkpfAtL + +## Other Ways To Support Us +### Affiliate Links +If you'd rather not send money, there are [other ways to support us](https://pi-hole.net/donate): you can sign up for services through our affiliate links, which will also help us offset some of the costs associated with keeping Pi-hole operational; or you can support us in some non-tangible ways as listed below. + +### Contributing Code Via Pull Requests + +We don't work on Pi-hole for monetary reasons; we work on it because we think it's fun and we think our software is important in today's world. To that end, we welcome all contributors--from novices to masters. + +If you feel you have some code to contribute, we're happy to take a look. Just make sure to fill out our template when submitting a pull request. We're all volunteers on the project and without all the information in the template, it's very difficult for us to quickly get the code merged in. + +You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh) and the [debug script](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/piholeDebug.sh) have an abundance of comments. These are two important scripts but we think they can also be a valuable resource to those who want to learn how to write scripts or code a program, which is why they are fully commented. So we encourage anyone who likes to tinker to read through it and submit a PR for us to review. + +### Presenting About Pi-hole + +Word-of-mouth has immensely helped our project grow. If you are going to be presenting about Pi-hole at a conference, meetup, or even for a school project, [get a hold of us for some free swag](https://pi-hole.net/2017/05/17/giving-a-presentation-on-pi-hole-contact-us-first-for-some-goodies-and-support/) to hand out to your audience. + +# Overview Of Features + +## The Dashboard (Web Interface) + +The [dashboard](https://github.com/pi-hole/AdminLTE#pi-hole-admin-dashboard) will (by default) be enabled during installation so you can view stats, change settings, and configure your Pi-hole. + +![Pi-hole Dashboard](https://assets.pi-hole.net/static/dashboard.png) + +There are several ways to [access the dashboard](https://discourse.pi-hole.net/t/how-do-i-access-pi-holes-dashboard-admin-interface/3168): + +1. `http:///admin/` +2. `http:/pi.hole/admin/` (when using Pi-hole as your DNS server) +3. `http://pi.hole/` (when using Pi-hole as your DNS server) + +### The Query Log + +If enabled, the query log will show all of the DNS queries requested by clients using Pi-hole as their DNS server. Forwarded domains will show in green, and blocked (_Pi-holed_) domains will show in red. You can also white or black list domains from within this section. + +

+ +

+ +The query log and graphs are what have helped people [discover what sort of traffic is traversing their networks](https://pi-hole.net/2017/07/06/round-3-what-really-happens-on-your-network/). + +#### Long-term Statistics +Using our Faster-Than-Light Engine ([FTL](https://github.com/pi-hole/FTL)), Pi-hole can store all of the domains queried in a database for retrieval or analysis later on. You can view this data as a graph, individual queries, or top clients/advertisers. + +

+ +

+ +### Whitelist And Blacklist + +Domains can be [whitelisted](https://discourse.pi-hole.net/t/commonly-whitelisted-domains/212) and/or [blacklisted](https://discourse.pi-hole.net/t/commonly-blacklisted-domains/305) using either the dashboard or [the `pihole` command](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738). + +

+ +

+ +#### Additional Blocklists +By default, Pi-hole blocks over 100,000 known ad-serving domains. You can expand the blocking power of your Pi-hole by [adding additional lists](https://discourse.pi-hole.net/t/how-do-i-add-additional-block-lists-to-pi-hole/259) such as the ones found on [The Big Blocklist Collection](https://wally3k.github.io/). + +

+ +

+ +### Enable And Disable Pi-hole +Sometimes you may want to stop using Pi-hole or turn it back on. You can trigger this via the dashboard or command line. + +

+ +

+ +### Tools + +

+ +

+ + +#### Update Ad Lists +This runs `gravity` to download any newly-added domains from your source lists. + +#### Query Ad Lists +You can find out what list a certain domain was on. This is useful for troubleshooting sites that may not work properly due to a blocked domain. + +#### `tail`ing Log Files +You can [watch the log files](https://discourse.pi-hole.net/t/how-do-i-watch-and-interpret-the-pihole-log-file/276) in real time to help debug any issues, or just see what's happening with your Pi-hole. + +#### Pi-hole Debugger +If you are having trouble with your Pi-hole, this is the place to go. You can run the debugger and it will attempt to diagnose any issues and then link to an FAQ with instructions on rectifying the problem. + +

+ +

+ +If run [via the command line](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#debug), you will see red/yellow/green text, which makes it easy to identify any problems. + +

+ +

+ + +After the debugger has finished, you have the option to upload it to our secure server for 48 hours. All you need to do then is provide one of our developers the unique token generated by the debugger (this is usually done via [our forums](https://discourse.pi-hole.net/c/bugs-problems-issues)). + +

+ +

+ +However, most of the time, you will be able to solve any issues without any intervention from us. But if you can't, we're always around to help out. + +### Settings + +The settings page lets you control and configure your Pi-hole. You can do things like: + +- view networking information +- flush logs or disable the logging of queries +- [enable Pi-hole's built-in DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026) +- [manage block lists](https://discourse.pi-hole.net/t/how-do-i-add-additional-block-lists-to-pi-hole/259) +- exclude domains from the graphs and enable privacy options +- configure upstream DNS servers +- restart Pi-hole's services +- back up some of Pi-hole's important files +- and more! + +

+ +

+ + +## Built-in DHCP Server + +Pi-hole ships with a [built-in DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026). This allows you to let your network devices use Pi-hole as their DNS server if your router does not let you adjust the DHCP options. + +One nice feature of using Pi-hole's DHCP server if you can set hostnames and DHCP reservations so you'll [see hostnames in the query log instead of IP addresses](https://discourse.pi-hole.net/t/how-do-i-show-hostnames-instead-of-ip-addresses-in-the-dashboard/3530). You can still do this without using Pi-hole's DHCP server; it just takes a little more work. If you do plan to use Pi-hole's DHCP server, be sure to disable DHCP on your router first. + +

+ +

+ +## The FTL Engine: Our API + +A read-only API can be accessed at `admin/api.php` (the same output can be achieved on the CLI by running `pihole -c -j`). + +It returns the following JSON: +``` json +{ + "domains_being_blocked":111175, + "dns_queries_today":15669, + "ads_blocked_today":1752, + "ads_percentage_today":11.181314, + "unique_domains":1178, + "queries_forwarded":9177, + "queries_cached":4740, + "unique_clients":18 + } +``` + +More details on the API can be found [here](https://discourse.pi-hole.net/t/pi-hole-api/1863) and on [the repo itself](https://github.com/pi-hole/FTL). + +### Real-time Statistics, Courtesy Of The Time Cops + +Using [chronometer2](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/chronometer.sh), you can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-pi-holes-stats-over-ssh-or-on-an-lcd-using-chronometer/240) via `ssh` or on an LCD screen such as the [2.8" LCD screen from Adafruit](http://amzn.to/1P0q1Fj). + +Simply run `pihole -c` for some detailed information. +``` +|¯¯¯(¯)__|¯|_ ___|¯|___ Pi-hole: v3.2 +| ¯_/¯|__| ' \/ _ \ / -_) AdminLTE: v3.2 +|_| |_| |_||_\___/_\___| FTL: v2.10 + —————————————————————————————————————————————————————————— + Hostname: pihole (Raspberry Pi 1, Model B) + Uptime: 11 days, 12:55:01 + Task Load: 0.35 0.16 0.15 (Active: 5 of 33 tasks) + CPU usage: 48% (1 core @ 700 MHz, 47c) + RAM usage: 12% (Used: 54 MB of 434 MB) + HDD usage: 20% (Used: 1 GB of 7 GB) + LAN addr: 192.168.1.100 (Gateway: 192.168.1.1) + Pi-hole: Active (Blocking: 111175 sites) + Ads Today: 11% (1759 of 15812 queries) + Fwd DNS: 208.67.222.222 (Alt DNS: 3 others) + —————————————————————————————————————————————————————————— + Recently blocked: www.google-analytics.com + Top Advertiser: www.example.org + Top Domain: www.example.org + Top Client: somehost +``` + +

+ +

+ +

+ +

+ +# Get Help Or Connect With Us On The Web - [Users Forum](https://discourse.pi-hole.net/) - [FAQs](https://discourse.pi-hole.net/c/faqs) +- [Feature requests](https://discourse.pi-hole.net/c/feature-requests?order=votes) - [Wiki](https://github.com/pi-hole/pi-hole/wiki) +- [Facebook](https://www.facebook.com/ThePiHole/) - ![Twitter](https://assets.pi-hole.net/static/twitter.png) [Tweet @The_Pi_Hole](https://twitter.com/The_Pi_Hole) - ![Reddit](https://assets.pi-hole.net/static/reddit.png) [Reddit /r/pihole](https://www.reddit.com/r/pihole/) - ![YouTube](https://assets.pi-hole.net/static/youtube.png) [Pi-hole channel](https://www.youtube.com/channel/UCT5kq9w0wSjogzJb81C9U0w) - [![Join the chat at https://gitter.im/pi-hole/pi-hole](https://badges.gitter.im/pi-hole/pi-hole.svg)](https://gitter.im/pi-hole/pi-hole?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) -## Technical Details +# Technical Details -The Pi-hole™ is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement. - -### Gravity - -The [gravity.sh](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) does most of the magic. The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0) (if you decide to use the [mahakala list](https://github.com/pi-hole/pi-hole/commit/963eacfe0537a7abddf30441c754c67ca1e40965)). This script is controlled by the `pihole` command. Please run `pihole -h` to see what commands can be run via `pihole`. +To summarize into a short sentence, the Pi-hole is an **advertising-aware DNS/Web server**. And while quite outdated at this point, [this original blog post about Pi-hole](https://jacobsalmela.com/2015/06/16/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0/) goes into **great detail** about how it was setup and how it works. Syntactically, it's no longer accurate, but the same basic principles and logic still apply to Pi-hole's current state. - -#### Other Operating Systems - -The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a web server, it should work OK. If there are other platforms you'd like supported, let us know. - -### Web Interface - -The [Web interface](https://github.com/pi-hole/AdminLTE#pi-hole-admin-dashboard) will be installed automatically so you can view stats and change settings. You can find it at: - -`http://192.168.1.x/admin/index.php` or `http://pi.hole/admin` - -![Pi-hole Advanced Stats Dashboard](https://assets.pi-hole.net/static/dashboard212.png) - -### Whitelist and blacklist - -Domains can be whitelisted and blacklisted using either the web interface or the command line. See [the wiki page](https://github.com/pi-hole/pi-hole/wiki/Whitelisting-and-Blacklisting) for more details -

- -

- -### Settings - -The settings page lets you control and configure your Pi-hole™. You can do things like: - -- enable Pi-hole's built-in DHCP server -- exclude domains from the graphs -- configure upstream DNS servers -- and more! - -![Settings page](https://assets.pi-hole.net/static/settings212.png) - -#### Built-in DHCP Server - -Pi-hole™ ships with a built-in DHCP server. This allows you to let your network devices use Pi-hole™ as their DNS server if your router does not let you adjust the DHCP options. -

- -

- -## API - -A basic read-only API can be accessed at `/admin/api.php`. It returns the following JSON: - -``` json -{ - "domains_being_blocked": "136708", - "dns_queries_today": "18108", - "ads_blocked_today": "14648", - "ads_percentage_today": "80.89" -} -``` - -The same output can be achieved on the CLI by running `chronometer.sh -j` - -## Real-time Statistics - -You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-pi-holes-stats-over-ssh-or-on-an-lcd-using-chronometer/240) via `ssh` or on an [2.8" LCD screen](http://amzn.to/1P0q1Fj). This is accomplished via [`chronometer.sh`](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/chronometer.sh). ![Pi-hole LCD](http://i.imgur.com/nBEqycp.jpg) - -## Pi-hole™ Projects +# Pi-hole Projects - [An ad blocking Magic Mirror](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware) - [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py) @@ -158,7 +300,7 @@ You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my- - [Pi-hole Droid - open source Android client](https://github.com/friimaind/pi-hole-droid) - [Windows DNS Swapper](https://github.com/roots84/DNS-Swapper), see [#1400](https://github.com/pi-hole/pi-hole/issues/1400) -## Coverage +# Coverage - [Adafruit livestream install](https://www.youtube.com/watch?v=eg4u2j1HYlI) - [TekThing: 5 fun, easy projects for a Raspberry Pi](https://youtu.be/QwrKlyC2kdM?t=1m42s) diff --git a/advanced/Scripts/chronometer.sh b/advanced/Scripts/chronometer.sh index d9b7d05b..d9b01fc0 100755 --- a/advanced/Scripts/chronometer.sh +++ b/advanced/Scripts/chronometer.sh @@ -7,6 +7,7 @@ # # This file is copyright under the latest version of the EUPL. # Please see LICENSE file for your rights under this license. +LC_NUMERIC=C # Retrieve stats from FTL engine pihole-FTL() { @@ -32,43 +33,74 @@ pihole-FTL() { exec 3<&- fi else - echo -e "${COL_LIGHT_RED}FTL offline${COL_NC}" + echo "0" fi } -# Print spaces to align right-side content +# Print spaces to align right-side additional text printFunc() { - txt_len="${#2}" - - # Reduce string length when using colour code - [ "${2:0:1}" == "" ] && txt_len=$((txt_len-7)) - - if [[ "$3" == "last" ]]; then - # Prevent final line from printing trailing newline - scr_size=( $(stty size 2>/dev/null || echo 24 80) ) - scr_width="${scr_size[1]}" - - title_len="${#1}" - spc_num=$(( (scr_width - title_len) - txt_len )) - [[ "$spc_num" -lt 0 ]] && spc_num="0" - spc=$(printf "%${spc_num}s") - - printf "%s%s$spc" "$1" "$2" - else - # Determine number of spaces for padding - spc_num=$(( 20 - txt_len )) - [[ "$spc_num" -lt 0 ]] && spc_num="0" - spc=$(printf "%${spc_num}s") + local text_last - # Print string (Max 20 characters, prevents overflow) - printf "%s%s$spc" "$1" "${2:0:20}" + title="$1" + title_len="${#title}" + + text_main="$2" + text_main_nocol="$text_main" + if [[ "${text_main:0:1}" == "" ]]; then + text_main_nocol=$(sed 's/\[[0-9;]\{1,5\}m//g' <<< "$text_main") + fi + text_main_len="${#text_main_nocol}" + + text_addn="$3" + if [[ "$text_addn" == "last" ]]; then + text_addn="" + text_last="true" + fi + + # If there is additional text, define max length of text_main + if [[ -n "$text_addn" ]]; then + case "$scr_cols" in + [0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-4]) text_main_max_len="9";; + 4[5-9]) text_main_max_len="14";; + *) text_main_max_len="19";; + esac + fi + + [[ -z "$text_addn" ]] && text_main_max_len="$(( scr_cols - title_len ))" + + # Remove excess characters from main text + if [[ "$text_main_len" -gt "$text_main_max_len" ]]; then + # Trim text without colours + text_main_trim="${text_main_nocol:0:$text_main_max_len}" + # Replace with trimmed text + text_main="${text_main/$text_main_nocol/$text_main_trim}" + fi + + # Determine amount of spaces for each line + if [[ -n "$text_last" ]]; then + # Move cursor to end of screen + spc_num=$(( scr_cols - ( title_len + text_main_len ) )) + else + spc_num=$(( text_main_max_len - text_main_len )) + fi + + [[ "$spc_num" -le 0 ]] && spc_num="0" + spc=$(printf "%${spc_num}s") + #spc="${spc// /.}" # Debug: Visualise spaces + + printf "%s%s$spc" "$title" "$text_main" + + if [[ -n "$text_addn" ]]; then + printf "%s(%s)%s\n" "$COL_NC$COL_DARK_GRAY" "$text_addn" "$COL_NC" + else + # Do not print trailing newline on final line + [[ -z "$text_last" ]] && printf "%s\n" "$COL_NC" fi } # Perform on first Chrono run (not for JSON formatted string) get_init_stats() { - LC_NUMERIC=C - calcFunc(){ awk "BEGIN {print $*}"; } + calcFunc(){ awk "BEGIN {print $*}" 2> /dev/null; } # Convert bytes to human-readable format hrBytes() { @@ -90,33 +122,53 @@ get_init_stats() { # Convert seconds to human-readable format hrSecs() { - day=$(( $1/60/60/24 )); hrs=$(( $1/3600%24 )); mins=$(( ($1%3600)/60 )); secs=$(( $1%60 )) + day=$(( $1/60/60/24 )); hrs=$(( $1/3600%24 )) + mins=$(( ($1%3600)/60 )); secs=$(( $1%60 )) [[ "$day" -ge "2" ]] && plu="s" [[ "$day" -ge "1" ]] && days="$day day${plu}, " || days="" printf "%s%02d:%02d:%02d\n" "$days" "$hrs" "$mins" "$secs" - } + } # Set Colour Codes coltable="/opt/pihole/COL_TABLE" if [[ -f "${coltable}" ]]; then source ${coltable} else - COL_NC='' - COL_DARK_GRAY='' - COL_LIGHT_GREEN='' - COL_LIGHT_BLUE='' - COL_LIGHT_RED='' - COL_YELLOW='' - COL_LIGHT_RED='' - COL_URG_RED='' + COL_NC="" + COL_DARK_GRAY="" + COL_LIGHT_GREEN="" + COL_LIGHT_BLUE="" + COL_LIGHT_RED="" + COL_YELLOW="" + COL_LIGHT_RED="" + COL_URG_RED="" fi - # Get RPi model number, or OS distro info + # Get RPi throttle state (RPi 3B only) & model number, or OS distro info if command -v vcgencmd &> /dev/null; then - sys_rev=$(awk '/Revision/ {print $3}' < /proc/cpuinfo) - case "$sys_rev" in + local sys_throttle_raw + local sys_rev_raw + + sys_throttle_raw=$(vgt=$(sudo vcgencmd get_throttled); echo "${vgt##*x}") + + # Active Throttle Notice: http://bit.ly/2gnunOo + if [[ "$sys_throttle_raw" != "0" ]]; then + case "$sys_throttle_raw" in + *0001) thr_type="${COL_YELLOW}Under Voltage";; + *0002) thr_type="${COL_LIGHT_BLUE}Arm Freq Cap";; + *0003) thr_type="${COL_YELLOW}UV${COL_DARK_GRAY},${COL_NC} ${COL_LIGHT_BLUE}AFC";; + *0004) thr_type="${COL_LIGHT_RED}Throttled";; + *0005) thr_type="${COL_YELLOW}UV${COL_DARK_GRAY},${COL_NC} ${COL_LIGHT_RED}TT";; + *0006) thr_type="${COL_LIGHT_BLUE}AFC${COL_DARK_GRAY},${COL_NC} ${COL_LIGHT_RED}TT";; + *0007) thr_type="${COL_YELLOW}UV${COL_DARK_GRAY},${COL_NC} ${COL_LIGHT_BLUE}AFC${COL_DARK_GRAY},${COL_NC} ${COL_LIGHT_RED}TT";; + esac + [[ -n "$thr_type" ]] && sys_throttle="$thr_type${COL_DARK_GRAY}" + fi + + sys_rev_raw=$(awk '/Revision/ {print $3}' < /proc/cpuinfo) + case "$sys_rev_raw" in 000[2-6]) sys_model=" 1, Model B";; # 256MB - 000[7-9]) sys_model=" 1, Model A" ;; # 256MB + 000[7-9]) sys_model=" 1, Model A";; # 256MB 000d|000e|000f) sys_model=" 1, Model B";; # 512MB 0010|0013) sys_model=" 1, Model B+";; # 512MB 0012|0015) sys_model=" 1, Model A+";; # 256MB @@ -126,7 +178,7 @@ get_init_stats() { 90009[2-3]|920093) sys_model=" Zero";; # 512MB 9000c1) sys_model=" Zero W";; # 512MB a02082|a[2-3]2082) sys_model=" 3, Model B";; # 1GB - *) sys_model="" ;; + *) sys_model="";; esac sys_type="Raspberry Pi$sys_model" else @@ -137,7 +189,6 @@ get_init_stats() { # Get core count sys_cores=$(grep -c "^processor" /proc/cpuinfo) - [[ "$sys_cores" -ne 1 ]] && sys_cores_plu="cores" || sys_cores_plu="core" # Test existence of clock speed file for ARM CPU if [[ -f "/sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq" ]]; then @@ -168,80 +219,95 @@ get_sys_stats() { # Update every 12 refreshes (Def: every 60s) count=$((count+1)) if [[ "$count" == "1" ]] || (( "$count" % 12 == 0 )); then + # Do not source setupVars if file does not exist [[ -n "$setupVars" ]] && source "$setupVars" - - - ph_ver_raw=($(pihole -v -c 2> /dev/null | sed -n 's/^.* v/v/p')) + + mapfile -t ph_ver_raw < <(pihole -v -c 2> /dev/null | sed -n 's/^.* v/v/p') if [[ -n "${ph_ver_raw[0]}" ]]; then ph_core_ver="${ph_ver_raw[0]}" ph_lte_ver="${ph_ver_raw[1]}" ph_ftl_ver="${ph_ver_raw[2]}" else - ph_core_ver="${COL_LIGHT_RED}API unavailable${COL_NC}" + ph_core_ver="-1" fi - + sys_name=$(hostname) - + [[ -n "$TEMPERATUREUNIT" ]] && temp_unit="$TEMPERATUREUNIT" || temp_unit="c" - + # Get storage stats for partition mounted on / - disk_raw=($(df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }')) + read -r -a disk_raw <<< "$(df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }')" disk_used="${disk_raw[0]}" disk_total="${disk_raw[1]}" disk_perc="${disk_raw[2]}" - + net_gateway=$(route -n | awk '$4 == "UG" {print $2;exit}') - + # Get DHCP stats, if feature is enabled if [[ "$DHCP_ACTIVE" == "true" ]]; then - ph_dhcp_eip="${DHCP_END##*.}" ph_dhcp_max=$(( ${DHCP_END##*.} - ${DHCP_START##*.} + 1 )) fi - - # Get alt DNS server, or print total count of alt DNS servers - if [[ -z "${PIHOLE_DNS_3}" ]]; then - ph_alts="${PIHOLE_DNS_2}" - else - dns_count="0" - [[ -n "${PIHOLE_DNS_2}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_3}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_4}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_5}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_6}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_7}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_8}" ]] && dns_count=$((dns_count+1)) - [[ -n "${PIHOLE_DNS_9}" ]] && dns_count="$dns_count+" - ph_alts="${dns_count} others" - fi + + # Get DNS server count + dns_count="0" + [[ -n "${PIHOLE_DNS_1}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_2}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_3}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_4}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_5}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_6}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_7}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_8}" ]] && dns_count=$((dns_count+1)) + [[ -n "${PIHOLE_DNS_9}" ]] && dns_count="$dns_count+" fi - + + # Get screen size + read -r -a scr_size <<< "$(stty size 2>/dev/null || echo 24 80)" + scr_lines="${scr_size[0]}" + scr_cols="${scr_size[1]}" + + # Determine Chronometer size behaviour + if [[ "$scr_cols" -ge 58 ]]; then + chrono_width="large" + elif [[ "$scr_cols" -gt 40 ]]; then + chrono_width="medium" + else + chrono_width="small" + fi + + # Determine max length of divider string + scr_line_len=$(( scr_cols - 2 )) + [[ "$scr_line_len" -ge 58 ]] && scr_line_len="58" + scr_line_str=$(printf "%${scr_line_len}s") + scr_line_str="${scr_line_str// /—}" + sys_uptime=$(hrSecs "$(cut -d. -f1 /proc/uptime)") sys_loadavg=$(cut -d " " -f1,2,3 /proc/loadavg) - - # Get CPU usage, only counting processes over 1% CPU as active + + # Get CPU usage, only counting processes over 1% as active cpu_raw=$(ps -eo pcpu,rss --no-headers | grep -E -v " 0") cpu_tasks=$(wc -l <<< "$cpu_raw") cpu_taskact=$(sed -r "/(^ 0.)/d" <<< "$cpu_raw" | wc -l) cpu_perc=$(awk '{sum+=$1} END {printf "%.0f\n", sum/'"$sys_cores"'}' <<< "$cpu_raw") - + # Get CPU clock speed if [[ -n "$scaling_freq_file" ]]; then cpu_mhz=$(( $(< /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq) / 1000 )) else - cpu_mhz=$(lscpu | awk -F "[ .]+" '/MHz/ {print $4;exit}') + cpu_mhz=$(lscpu | awk -F ":" '/MHz/ {print $2;exit}') + cpu_mhz=$(printf "%.0f" "${cpu_mhz//[[:space:]]/}") fi - - # Determine correct string format for CPU clock speed + + # Determine whether to display CPU clock speed as MHz or GHz if [[ -n "$cpu_mhz" ]]; then - [[ "$cpu_mhz" -le "999" ]] && cpu_freq="$cpu_mhz MHz" || cpu_freq="$(calcFunc "$cpu_mhz"/1000) Ghz" - [[ -n "$cpu_freq" ]] && cpu_freq_str=" @ $cpu_freq" || cpu_freq_str="" + [[ "$cpu_mhz" -le "999" ]] && cpu_freq="$cpu_mhz MHz" || cpu_freq="$(calcFunc "$cpu_mhz"/1000) GHz" fi - + # Determine colour for temperature if [[ -n "$temp_file" ]]; then if [[ "$temp_unit" == "C" ]]; then - cpu_temp=$(printf "%'.0fc\n" "$(calcFunc "$(< $temp_file) / 1000")") - + cpu_temp=$(printf "%.0fc\n" "$(calcFunc "$(< $temp_file) / 1000")") + case "${cpu_temp::-1}" in -*|[0-9]|[1-3][0-9]) cpu_col="$COL_LIGHT_BLUE";; 4[0-9]) cpu_col="";; @@ -249,13 +315,13 @@ get_sys_stats() { 6[0-9]) cpu_col="$COL_LIGHT_RED";; *) cpu_col="$COL_URG_RED";; esac - + # $COL_NC$COL_DARK_GRAY is needed for $COL_URG_RED - cpu_temp_str=", $cpu_col$cpu_temp$COL_NC$COL_DARK_GRAY" - + cpu_temp_str=" @ $cpu_col$cpu_temp$COL_NC$COL_DARK_GRAY" + elif [[ "$temp_unit" == "F" ]]; then - cpu_temp=$(printf "%'.0ff\n" "$(calcFunc "($(< $temp_file) / 1000) * 9 / 5 + 32")") - + cpu_temp=$(printf "%.0ff\n" "$(calcFunc "($(< $temp_file) / 1000) * 9 / 5 + 32")") + case "${cpu_temp::-1}" in -*|[0-9]|[0-9][0-9]) cpu_col="$COL_LIGHT_BLUE";; 1[0-1][0-9]) cpu_col="";; @@ -263,132 +329,199 @@ get_sys_stats() { 1[4-5][0-9]) cpu_col="$COL_LIGHT_RED";; *) cpu_col="$COL_URG_RED";; esac - - cpu_temp_str=", $cpu_col$cpu_temp$COL_NC$COL_DARK_GRAY" - + + cpu_temp_str=" @ $cpu_col$cpu_temp$COL_NC$COL_DARK_GRAY" + else - cpu_temp_str=$(printf ", %'.0fk\n" "$(calcFunc "($(< $temp_file) / 1000) + 273.15")") + cpu_temp_str=$(printf " @ %.0fk\n" "$(calcFunc "($(< $temp_file) / 1000) + 273.15")") fi else cpu_temp_str="" fi - - ram_raw=($(awk '/MemTotal:/{total=$2} /MemFree:/{free=$2} /Buffers:/{buffers=$2} /^Cached:/{cached=$2} END {printf "%.0f %.0f %.0f", (total-free-buffers-cached)*100/total, (total-free-buffers-cached)*1024, total*1024}' /proc/meminfo)) + + read -r -a ram_raw <<< "$(awk '/MemTotal:/{total=$2} /MemFree:/{free=$2} /Buffers:/{buffers=$2} /^Cached:/{cached=$2} END {printf "%.0f %.0f %.0f", (total-free-buffers-cached)*100/total, (total-free-buffers-cached)*1024, total*1024}' /proc/meminfo)" ram_perc="${ram_raw[0]}" ram_used="${ram_raw[1]}" ram_total="${ram_raw[2]}" - + if [[ "$(pihole status web 2> /dev/null)" == "1" ]]; then ph_status="${COL_LIGHT_GREEN}Active" else - ph_status="${COL_LIGHT_RED}Inactive" + ph_status="${COL_LIGHT_RED}Offline" fi - + if [[ "$DHCP_ACTIVE" == "true" ]]; then - ph_dhcp_num=$(wc -l 2> /dev/null < "/etc/pihole/dhcp.leases") + local ph_dhcp_range + + ph_dhcp_range=$(seq -s "|" -f "${DHCP_START%.*}.%g" "${DHCP_START##*.}" "${DHCP_END##*.}") + + # Count dynamic leases from available range, and not static leases + ph_dhcp_num=$(grep -cE "$ph_dhcp_range" "/etc/pihole/dhcp.leases") + ph_dhcp_percent=$(( ph_dhcp_num * 100 / ph_dhcp_max )) fi } get_ftl_stats() { local stats_raw - - stats_raw=($(pihole-FTL "stats")) - domains_being_blocked_raw="${stats_raw[1]}" - dns_queries_today_raw="${stats_raw[3]}" - ads_blocked_today_raw="${stats_raw[5]}" - ads_percentage_today_raw="${stats_raw[7]}" + + mapfile -t stats_raw < <(pihole-FTL "stats") + domains_being_blocked_raw="${stats_raw[1]#* }" + dns_queries_today_raw="${stats_raw[3]#* }" + ads_blocked_today_raw="${stats_raw[5]#* }" + ads_percentage_today_raw="${stats_raw[7]#* }" + queries_forwarded_raw="${stats_raw[11]#* }" + queries_cached_raw="${stats_raw[13]#* }" # Only retrieve these stats when not called from jsonFunc if [[ -z "$1" ]]; then - local recent_blocked_raw local top_ad_raw local top_domain_raw local top_client_raw - - domains_being_blocked=$(printf "%'.0f\n" "${domains_being_blocked_raw}") - dns_queries_today=$(printf "%'.0f\n" "${dns_queries_today_raw}") - ads_blocked_today=$(printf "%'.0f\n" "${ads_blocked_today_raw}") + + domains_being_blocked=$(printf "%.0f\n" "${domains_being_blocked_raw}") + dns_queries_today=$(printf "%.0f\n" "${dns_queries_today_raw}") + ads_blocked_today=$(printf "%.0f\n" "${ads_blocked_today_raw}") ads_percentage_today=$(printf "%'.0f\n" "${ads_percentage_today_raw}") - - recent_blocked_raw=$(pihole-FTL recentBlocked) - top_ad_raw=($(pihole-FTL "top-ads (1)")) - top_domain_raw=($(pihole-FTL "top-domains (1)")) - top_client_raw=($(pihole-FTL "top-clients (1)")) - - # Limit strings to 40 characters to prevent overflow - recent_blocked="${recent_blocked_raw:0:40}" - top_ad="${top_ad_raw[2]:0:40}" - top_domain="${top_domain_raw[2]:0:40}" - [[ "${top_client_raw[3]}" ]] && top_client="${top_client_raw[3]:0:40}" || top_client="${top_client_raw[2]:0:40}" + queries_cached_percentage=$(printf "%.0f\n" "$(calcFunc "$queries_cached_raw * 100 / ( $queries_forwarded_raw + $queries_cached_raw )")") + recent_blocked=$(pihole-FTL recentBlocked) + read -r -a top_ad_raw <<< "$(pihole-FTL "top-ads (1)")" + read -r -a top_domain_raw <<< "$(pihole-FTL "top-domains (1)")" + read -r -a top_client_raw <<< "$(pihole-FTL "top-clients (1)")" + + top_ad="${top_ad_raw[2]}" + top_domain="${top_domain_raw[2]}" + if [[ "${top_client_raw[3]}" ]]; then + top_client="${top_client_raw[3]}" + else + top_client="${top_client_raw[2]}" + fi fi } +get_strings() { + # Expand or contract strings depending on screen size + if [[ "$chrono_width" == "large" ]]; then + phc_str=" ${COL_DARK_GRAY}Pi-hole" + lte_str=" ${COL_DARK_GRAY}Admin" + ftl_str=" ${COL_DARK_GRAY}FTL" + api_str="${COL_LIGHT_RED}API Offline" + + host_info="$sys_type" + sys_info="$sys_throttle" + sys_info2="Active: $cpu_taskact of $cpu_tasks tasks" + used_str="Used: " + leased_str="Leased: " + domains_being_blocked=$(printf "%'.0f" "$domains_being_blocked") + ph_info="Blocking: $domains_being_blocked sites" + total_str="Total: " + else + phc_str=" ${COL_DARK_GRAY}PH" + lte_str=" ${COL_DARK_GRAY}Web" + ftl_str=" ${COL_DARK_GRAY}FTL" + api_str="${COL_LIGHT_RED}API Down" + ph_info="$domains_being_blocked blocked" + fi + + [[ "$sys_cores" -ne 1 ]] && sys_cores_txt="${sys_cores}x " + cpu_info="$sys_cores_txt$cpu_freq$cpu_temp_str" + ram_info="$used_str$(hrBytes "$ram_used") of $(hrBytes "$ram_total")" + disk_info="$used_str$(hrBytes "$disk_used") of $(hrBytes "$disk_total")" + + lan_info="Gateway: $net_gateway" + dhcp_info="$leased_str$ph_dhcp_num of $ph_dhcp_max" + + ads_info="$total_str$ads_blocked_today of $dns_queries_today" + dns_info="$dns_count DNS servers" + + [[ "$recent_blocked" == "0" ]] && recent_blocked="${COL_LIGHT_RED}FTL offline${COL_NC}" +} + chronoFunc() { get_init_stats - + for (( ; ; )); do get_sys_stats get_ftl_stats - - # Do not print LTE/FTL strings if API is unavailable - ph_core_str=" ${COL_DARK_GRAY}Pi-hole: $ph_core_ver${COL_NC}" - if [[ -n "$ph_lte_ver" ]]; then - ph_lte_str=" ${COL_DARK_GRAY}AdminLTE: $ph_lte_ver${COL_NC}" - ph_ftl_str=" ${COL_DARK_GRAY}FTL: $ph_ftl_ver${COL_NC}" - fi - - clear - - echo -e "|¯¯¯(¯)__|¯|_ ___|¯|___$ph_core_str -| ¯_/¯|__| ' \/ _ \ / -_)$ph_lte_str -|_| |_| |_||_\___/_\___|$ph_ftl_str - ${COL_DARK_GRAY}——————————————————————————————————————————————————————————${COL_NC}" + get_strings - printFunc " Hostname: " "$sys_name" - [ -n "$sys_type" ] && printf "%s(%s)%s\n" "$COL_DARK_GRAY" "$sys_type" "$COL_NC" || printf "\n" - - printf "%s\n" " Uptime: $sys_uptime" - - printFunc " Task Load: " "$sys_loadavg" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Active: $cpu_taskact of $cpu_tasks tasks" "$COL_NC" - - printFunc " CPU usage: " "$cpu_perc%" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "$sys_cores $sys_cores_plu$cpu_freq_str$cpu_temp_str" "$COL_NC" - - printFunc " RAM usage: " "$ram_perc%" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Used: $(hrBytes "$ram_used") of $(hrBytes "$ram_total")" "$COL_NC" - - printFunc " HDD usage: " "$disk_perc" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Used: $(hrBytes "$disk_used") of $(hrBytes "$disk_total")" "$COL_NC" - - printFunc " LAN addr: " "${IPV4_ADDRESS/\/*/}" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Gateway: $net_gateway" "$COL_NC" - - if [[ "$DHCP_ACTIVE" == "true" ]]; then - printFunc " DHCP: " "$DHCP_START to $ph_dhcp_eip" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Leased: $ph_dhcp_num of $ph_dhcp_max" "$COL_NC" + # Strip excess development version numbers + if [[ "$ph_core_ver" != "-1" ]]; then + phc_ver_str="$phc_str: ${ph_core_ver%-*}${COL_NC}" + lte_ver_str="$lte_str: ${ph_lte_ver%-*}${COL_NC}" + ftl_ver_str="$ftl_str: ${ph_ftl_ver%-*}${COL_NC}" + else + phc_ver_str="$phc_str: $api_str${COL_NC}" fi - - printFunc " Pi-hole: " "$ph_status" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Blocking: $domains_being_blocked sites" "$COL_NC" - - printFunc " Ads Today: " "$ads_percentage_today%" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "$ads_blocked_today of $dns_queries_today queries" "$COL_NC" - - printFunc " Fwd DNS: " "$PIHOLE_DNS_1" - printf "%s(%s)%s\n" "$COL_DARK_GRAY" "Alt DNS: $ph_alts" "$COL_NC" - - echo -e " ${COL_DARK_GRAY}——————————————————————————————————————————————————————————${COL_NC}" - echo " Recently blocked: $recent_blocked" - echo " Top Advertiser: $top_ad" - echo " Top Domain: $top_domain" - printFunc " Top Client: " "$top_client" "last" - - if [[ "$1" == "exit" ]]; then + + # Get refresh number + if [[ "$*" == *"-r"* ]]; then + num="$*" + num="${num/*-r /}" + num="${num/ */}" + num_str="Refresh set for every $num seconds" + else + num_str="" + fi + + clear + + # Remove exit message heading on third refresh + if [[ "$count" -le 2 ]] && [[ "$*" != *"-e"* ]]; then + echo -e " ${COL_LIGHT_GREEN}Pi-hole Chronometer${COL_NC} + $num_str + ${COL_LIGHT_RED}Press Ctrl-C to exit${COL_NC} + ${COL_DARK_GRAY}$scr_line_str${COL_NC}" + else + echo -e "|¯¯¯(¯)_|¯|_ ___|¯|___$phc_ver_str +| ¯_/¯|_| ' \/ _ \ / -_)$lte_ver_str +|_| |_| |_||_\___/_\___|$ftl_ver_str + ${COL_DARK_GRAY}$scr_line_str${COL_NC}" + fi + + printFunc " Hostname: " "$sys_name" "$host_info" + printFunc " Uptime: " "$sys_uptime" "$sys_info" + printFunc " Task Load: " "$sys_loadavg" "$sys_info2" + printFunc " CPU usage: " "$cpu_perc%" "$cpu_info" + printFunc " RAM usage: " "$ram_perc%" "$ram_info" + printFunc " HDD usage: " "$disk_perc" "$disk_info" + + if [[ "$scr_lines" -gt 17 ]] && [[ "$chrono_width" != "small" ]]; then + printFunc " LAN addr: " "${IPV4_ADDRESS/\/*/}" "$lan_info" + fi + + if [[ "$DHCP_ACTIVE" == "true" ]]; then + printFunc "DHCP usage: " "$ph_dhcp_percent%" "$dhcp_info" + fi + + printFunc " Pi-hole: " "$ph_status" "$ph_info" + printFunc " Ads Today: " "$ads_percentage_today%" "$ads_info" + printFunc "Local Qrys: " "$queries_cached_percentage%" "$dns_info" + + printFunc " Blocked: " "$recent_blocked" + printFunc "Top Advert: " "$top_ad" + + # Provide more stats on screens with more lines + if [[ "$scr_lines" -eq 17 ]]; then + if [[ "$DHCP_ACTIVE" == "true" ]]; then + printFunc "Top Domain: " "$top_domain" "last" + else + print_client="true" + fi + else + print_client="true" + fi + + if [[ -n "$print_client" ]]; then + printFunc "Top Domain: " "$top_domain" + printFunc "Top Client: " "$top_client" "last" + fi + + # Handle exit/refresh options + if [[ "$*" == *"-e"* ]]; then exit 0 else - if [[ -n "$1" ]]; then - sleep "${1}" + if [[ "$*" == *"-r"* ]]; then + sleep "$num" else sleep 5 fi @@ -409,14 +542,14 @@ helpFunc() { echo "Usage: pihole -c [options] Example: 'pihole -c -j' Calculates stats and displays to an LCD - + Options: -j, --json Output stats as JSON formatted string -r, --refresh Set update frequency (in seconds) -e, --exit Output stats and exit witout refreshing -h, --help Display this help text" fi - + exit 0 } @@ -428,8 +561,8 @@ for var in "$@"; do case "$var" in "-j" | "--json" ) jsonFunc;; "-h" | "--help" ) helpFunc;; - "-r" | "--refresh" ) chronoFunc "$2";; - "-e" | "--exit" ) chronoFunc "exit";; + "-r" | "--refresh" ) chronoFunc "$@";; + "-e" | "--exit" ) chronoFunc "$@";; * ) helpFunc "?";; esac done diff --git a/advanced/Scripts/list.sh b/advanced/Scripts/list.sh index e7151cf6..86589083 100755 --- a/advanced/Scripts/list.sh +++ b/advanced/Scripts/list.sh @@ -62,11 +62,14 @@ EscapeRegexp() { } HandleOther() { - # First, convert everything to lowercase - domain=$(sed -e "y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/" <<< "$1") + # Convert to lowercase + domain="${1,,}" # Check validity of domain - validDomain=$(echo "${domain}" | perl -lne 'print if /(?!.*[^a-z0-9-\.].*)^((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9-]+\.)*[a-z]{2,63}/') + validDomain=$(perl -lne 'print if /^((-|_)*[a-z\d]((-|_)*[a-z\d])*(-|_)*)(\.(-|_)*([a-z\d]((-|_)*[a-z\d])*))*$/' <<< "${domain}") # Valid chars check + validDomain=$(perl -lne 'print if /^.{1,253}$/' <<< "${validDomain}") # Overall length check + validDomain=$(perl -lne 'print if /^[^\.]{1,63}(\.[^\.]{1,63})*$/' <<< "${validDomain}") # Length of each label + if [[ -z "${validDomain}" ]]; then echo -e " ${CROSS} $1 is not a valid argument or domain name!" else @@ -98,7 +101,7 @@ PoplistFile() { AddDomain() { list="$2" domain=$(EscapeRegexp "$1") - + [[ "${list}" == "${whitelist}" ]] && listname="whitelist" [[ "${list}" == "${blacklist}" ]] && listname="blacklist" [[ "${list}" == "${wildcardlist}" ]] && listname="wildcard blacklist" @@ -151,7 +154,7 @@ AddDomain() { RemoveDomain() { list="$2" domain=$(EscapeRegexp "$1") - + [[ "${list}" == "${whitelist}" ]] && listname="whitelist" [[ "${list}" == "${blacklist}" ]] && listname="blacklist" [[ "${list}" == "${wildcardlist}" ]] && listname="wildcard blacklist" diff --git a/advanced/Scripts/piholeCheckout.sh b/advanced/Scripts/piholeCheckout.sh index 657bde83..f9add489 100644 --- a/advanced/Scripts/piholeCheckout.sh +++ b/advanced/Scripts/piholeCheckout.sh @@ -3,7 +3,7 @@ # (c) 2017 Pi-hole, LLC (https://pi-hole.net) # Network-wide ad blocking via your own hardware. # -# Switch Pi-hole subsystems to a different Github branch +# Switch Pi-hole subsystems to a different Github branch. # # This file is copyright under the latest version of the EUPL. # Please see LICENSE file for your rights under this license. @@ -154,7 +154,7 @@ fetch_checkout_pull_branch() { cd "${directory}" || return 1 git remote set-branches origin "${branch}" || return 1 git stash --all --quiet &> /dev/null || true - git clean --force -d || true + git clean --quiet --force -d || true git fetch --quiet || return 1 checkout_pull_branch "${directory}" "${branch}" || return 1 } @@ -171,7 +171,7 @@ checkout_pull_branch() { oldbranch="$(git symbolic-ref HEAD)" - git checkout "${branch}" || return 1 + git checkout "${branch}" --quiet || return 1 if [[ "$(git diff "${oldbranch}" | grep -c "^")" -gt "0" ]]; then update="true" @@ -180,7 +180,7 @@ checkout_pull_branch() { git_pull=$(git pull || return 1) if [[ "$git_pull" == *"up-to-date"* ]]; then - echo -e "\n ${INFO} $(git pull)" + echo -e " ${INFO} $(git pull)" else echo -e "$git_pull\n" fi @@ -239,19 +239,20 @@ checkout() { if [[ "${1}" == "dev" ]] ; then # Shortcut to check out development branches echo -e " ${INFO} Shortcut \"dev\" detected - checking out development / devel branches..." - echo -e " ${INFO} Pi-hole core" + echo "" + echo -e " ${INFO} Pi-hole Core" fetch_checkout_pull_branch "${PI_HOLE_FILES_DIR}" "development" || { echo " ${CROSS} Unable to pull Core developement branch"; exit 1; } if [[ ${INSTALL_WEB} == "true" ]]; then + echo "" echo -e " ${INFO} Web interface" fetch_checkout_pull_branch "${webInterfaceDir}" "devel" || { echo " ${CROSS} Unable to pull Web development branch"; exit 1; } fi - echo -e " ${TICK} Pi-hole core" + #echo -e " ${TICK} Pi-hole Core" get_binary_name local path path="development/${binary}" FTLinstall "${binary}" "${path}" - elif [[ "${1}" == "master" ]] ; then # Shortcut to check out master branches echo -e " ${INFO} Shortcut \"master\" detected - checking out master branches..." @@ -261,13 +262,11 @@ checkout() { echo -e " ${INFO} Web interface" fetch_checkout_pull_branch "${webInterfaceDir}" "master" || { echo " ${CROSS} Unable to pull Web master branch"; exit 1; } fi - echo -e " ${TICK} Web interface" - - get_binary_name + #echo -e " ${TICK} Web Interface" + get_binary_name local path path="master/${binary}" FTLinstall "${binary}" "${path}" - elif [[ "${1}" == "core" ]] ; then str="Fetching branches from ${piholeGitUrl}" echo -ne " ${INFO} $str" diff --git a/advanced/Scripts/piholeDebug.sh b/advanced/Scripts/piholeDebug.sh index 8020cc80..60b04b73 100755 --- a/advanced/Scripts/piholeDebug.sh +++ b/advanced/Scripts/piholeDebug.sh @@ -9,534 +9,1124 @@ # Please see LICENSE file for your rights under this license. - +# -e option instructs bash to immediately exit if any command [1] has a non-zero exit status +# -u a reference to any variable you haven't previously defined +# with the exceptions of $* and $@ - is an error, and causes the program to immediately exit +# -o pipefail prevents errors in a pipeline from being masked. If any command in a pipeline fails, +# that return code will be used as the return code of the whole pipeline. By default, the +# pipeline's return code is that of the last command - even if it succeeds set -o pipefail +#IFS=$'\n\t' ######## GLOBAL VARS ######## -VARSFILE="/etc/pihole/setupVars.conf" -DEBUG_LOG="/var/log/pihole_debug.log" -DNSMASQFILE="/etc/dnsmasq.conf" -DNSMASQCONFDIR="/etc/dnsmasq.d/*" -LIGHTTPDFILE="/etc/lighttpd/lighttpd.conf" -LIGHTTPDERRFILE="/var/log/lighttpd/error.log" -GRAVITYFILE="/etc/pihole/gravity.list" -WHITELISTFILE="/etc/pihole/whitelist.txt" -BLACKLISTFILE="/etc/pihole/blacklist.txt" -ADLISTFILE="/etc/pihole/adlists.list" -PIHOLELOG="/var/log/pihole.log" -PIHOLEGITDIR="/etc/.pihole/" -ADMINGITDIR="/var/www/html/admin/" -WHITELISTMATCHES="/tmp/whitelistmatches.list" -readonly FTLLOG="/var/log/pihole-FTL.log" +# These variables would normally be next to the other files +# but we need them to be first in order to get the colors needed for the script output +PIHOLE_SCRIPTS_DIRECTORY="/opt/pihole" +PIHOLE_COLTABLE_FILE="${PIHOLE_SCRIPTS_DIRECTORY}/COL_TABLE" -TIMEOUT=60 -# Header info and introduction -cat << EOM -::: Beginning Pi-hole debug at $(date)! -::: -::: This process collects information from your Pi-hole, and optionally uploads -::: it to a unique and random directory on tricorder.pi-hole.net. -::: -::: NOTE: All log files auto-delete after 48 hours and ONLY the Pi-hole developers -::: can access your data via the given token. We have taken these extra steps to -::: secure your data and will work to further reduce any personal information gathered. -::: -::: Please read and note any issues, and follow any directions advised during this process. -EOM +# These provide the colors we need for making the log more readable +if [[ -f ${PIHOLE_COLTABLE_FILE} ]]; then + source ${PIHOLE_COLTABLE_FILE} +else + COL_NC='\e[0m' # No Color + COL_YELLOW='\e[1;33m' + COL_LIGHT_PURPLE='\e[1;35m' + COL_CYAN='\e[0;36m' + TICK="[${COL_LIGHT_GREEN}✓${COL_NC}]" + CROSS="[${COL_LIGHT_RED}✗${COL_NC}]" + INFO="[i]" + DONE="${COL_LIGHT_GREEN} done!${COL_NC}" + OVER="\r\033[K" +fi -source ${VARSFILE} +OBFUSCATED_PLACEHOLDER="" -### Private functions exist here ### -log_write() { - echo "${@}" >&3 +# FAQ URLs for use in showing the debug log +FAQ_UPDATE_PI_HOLE="${COL_CYAN}https://discourse.pi-hole.net/t/how-do-i-update-pi-hole/249${COL_NC}" +FAQ_CHECKOUT_COMMAND="${COL_CYAN}https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#checkout${COL_NC}" +FAQ_HARDWARE_REQUIREMENTS="${COL_CYAN}https://discourse.pi-hole.net/t/hardware-software-requirements/273${COL_NC}" +FAQ_HARDWARE_REQUIREMENTS_PORTS="${COL_CYAN}https://discourse.pi-hole.net/t/hardware-software-requirements/273#ports${COL_NC}" +FAQ_GATEWAY="${COL_CYAN}https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546${COL_NC}" +FAQ_ULA="${COL_CYAN}https://discourse.pi-hole.net/t/use-ipv6-ula-addresses-for-pi-hole/2127${COL_NC}" +FAQ_FTL_COMPATIBILITY="${COL_CYAN}https://github.com/pi-hole/FTL#compatibility-list${COL_NC}" +FAQ_BAD_ADDRESS="${COL_CYAN}https://discourse.pi-hole.net/t/why-do-i-see-bad-address-at-in-pihole-log/3972${COL_NC}" + +# Other URLs we may use +FORUMS_URL="${COL_CYAN}https://discourse.pi-hole.net${COL_NC}" +TRICORDER_CONTEST="${COL_CYAN}https://pi-hole.net/2016/11/07/crack-our-medical-tricorder-win-a-raspberry-pi-3/${COL_NC}" + +# Port numbers used for uploading the debug log +TRICORDER_NC_PORT_NUMBER=9999 +TRICORDER_SSL_PORT_NUMBER=9998 + +# Directories required by Pi-hole +# https://discourse.pi-hole.net/t/what-files-does-pi-hole-use/1684 +CORE_GIT_DIRECTORY="/etc/.pihole" +CRON_D_DIRECTORY="/etc/cron.d" +DNSMASQ_D_DIRECTORY="/etc/dnsmasq.d" +PIHOLE_DIRECTORY="/etc/pihole" +PIHOLE_SCRIPTS_DIRECTORY="/opt/pihole" +BIN_DIRECTORY="/usr/local/bin" +RUN_DIRECTORY="/run" +LOG_DIRECTORY="/var/log" +WEB_SERVER_LOG_DIRECTORY="${LOG_DIRECTORY}/lighttpd" +WEB_SERVER_CONFIG_DIRECTORY="/etc/lighttpd" +HTML_DIRECTORY="/var/www/html" +WEB_GIT_DIRECTORY="${HTML_DIRECTORY}/admin" +BLOCK_PAGE_DIRECTORY="${HTML_DIRECTORY}/pihole" + +# Files required by Pi-hole +# https://discourse.pi-hole.net/t/what-files-does-pi-hole-use/1684 +PIHOLE_CRON_FILE="${CRON_D_DIRECTORY}/pihole" + +PIHOLE_DNS_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/01-pihole.conf" +PIHOLE_DHCP_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/02-pihole-dhcp.conf" +PIHOLE_WILDCARD_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/03-wildcard.conf" + +WEB_SERVER_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/lighttpd.conf" +WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf" + +PIHOLE_DEFAULT_AD_LISTS="${PIHOLE_DIRECTORY}/adlists.default" +PIHOLE_USER_DEFINED_AD_LISTS="${PIHOLE_DIRECTORY}/adlists.list" +PIHOLE_BLACKLIST_FILE="${PIHOLE_DIRECTORY}/blacklist.txt" +PIHOLE_BLOCKLIST_FILE="${PIHOLE_DIRECTORY}/gravity.list" +PIHOLE_INSTALL_LOG_FILE="${PIHOLE_DIRECTORY}/install.log" +PIHOLE_RAW_BLOCKLIST_FILES=${PIHOLE_DIRECTORY}/list.* +PIHOLE_LOCAL_HOSTS_FILE="${PIHOLE_DIRECTORY}/local.list" +PIHOLE_LOGROTATE_FILE="${PIHOLE_DIRECTORY}/logrotate" +PIHOLE_SETUP_VARS_FILE="${PIHOLE_DIRECTORY}/setupVars.conf" +PIHOLE_WHITELIST_FILE="${PIHOLE_DIRECTORY}/whitelist.txt" + +PIHOLE_COMMAND="${BIN_DIRECTORY}/pihole" +PIHOLE_COLTABLE_FILE="${BIN_DIRECTORY}/COL_TABLE" + +FTL_PID="${RUN_DIRECTORY}/pihole-FTL.pid" +FTL_PORT="${RUN_DIRECTORY}/pihole-FTL.port" + +PIHOLE_LOG="${LOG_DIRECTORY}/pihole.log" +PIHOLE_LOG_GZIPS=${LOG_DIRECTORY}/pihole.log.[0-9].* +PIHOLE_DEBUG_LOG="${LOG_DIRECTORY}/pihole_debug.log" +PIHOLE_DEBUG_LOG_SANITIZED="${LOG_DIRECTORY}/pihole_debug-sanitized.log" +PIHOLE_FTL_LOG="${LOG_DIRECTORY}/pihole-FTL.log" + +PIHOLE_WEB_SERVER_ACCESS_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/access.log" +PIHOLE_WEB_SERVER_ERROR_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/error.log" + +# An array of operating system "pretty names" that we officialy support +# We can loop through the array at any time to see if it matches a value +SUPPORTED_OS=("Raspbian" "Ubuntu" "Fedora" "Debian" "CentOS") + +# Store Pi-hole's processes in an array for easy use and parsing +PIHOLE_PROCESSES=( "dnsmasq" "lighttpd" "pihole-FTL" ) + +# Store the required directories in an array so it can be parsed through +REQUIRED_DIRECTORIES=(${CORE_GIT_DIRECTORY} +${CRON_D_DIRECTORY} +${DNSMASQ_D_DIRECTORY} +${PIHOLE_DIRECTORY} +${PIHOLE_SCRIPTS_DIRECTORY} +${BIN_DIRECTORY} +${RUN_DIRECTORY} +${LOG_DIRECTORY} +${WEB_SERVER_LOG_DIRECTORY} +${WEB_SERVER_CONFIG_DIRECTORY} +${HTML_DIRECTORY} +${WEB_GIT_DIRECTORY} +${BLOCK_PAGE_DIRECTORY}) + +# Store the required directories in an array so it can be parsed through +mapfile -t array <<< "$var" +REQUIRED_FILES=(${PIHOLE_CRON_FILE} +${PIHOLE_DNS_CONFIG_FILE} +${PIHOLE_DHCP_CONFIG_FILE} +${PIHOLE_WILDCARD_CONFIG_FILE} +${WEB_SERVER_CONFIG_FILE} +${PIHOLE_DEFAULT_AD_LISTS} +${PIHOLE_USER_DEFINED_AD_LISTS} +${PIHOLE_BLACKLIST_FILE} +${PIHOLE_BLOCKLIST_FILE} +${PIHOLE_INSTALL_LOG_FILE} +${PIHOLE_RAW_BLOCKLIST_FILES} +${PIHOLE_LOCAL_HOSTS_FILE} +${PIHOLE_LOGROTATE_FILE} +${PIHOLE_SETUP_VARS_FILE} +${PIHOLE_WHITELIST_FILE} +${PIHOLE_COMMAND} +${PIHOLE_COLTABLE_FILE} +${FTL_PID} +${FTL_PORT} +${PIHOLE_LOG} +${PIHOLE_LOG_GZIPS} +${PIHOLE_DEBUG_LOG} +${PIHOLE_FTL_LOG} +${PIHOLE_WEB_SERVER_ACCESS_LOG_FILE} +${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}) + +DISCLAIMER="This process collects information from your Pi-hole, and optionally uploads it to a unique and random directory on tricorder.pi-hole.net. + +The intent of this script is to allow users to self-diagnose their installations. This is accomplished by running tests against our software and providing the user with links to FAQ articles when a problem is detected. Since we are a small team and Pi-hole has been growing steadily, it is our hope that this will help us spend more time on development. + +NOTE: All log files auto-delete after 48 hours and ONLY the Pi-hole developers can access your data via the given token. We have taken these extra steps to secure your data and will work to further reduce any personal information gathered. +" + +show_disclaimer(){ + log_write "${DISCLAIMER}" } -log_echo() { - case ${1} in - -n) - echo -n "::: ${2}" - log_write "${2}" - ;; - -r) - echo "::: ${2}" - log_write "${2}" - ;; - -l) - echo "${2}" - log_write "${2}" - ;; - *) - echo "::: ${1}" - log_write "${1}" +source_setup_variables() { + # Display the current test that is running + log_write "\n${COL_LIGHT_PURPLE}*** [ INITIALIZING ]${COL_NC} Sourcing setup variables" + # If the variable file exists, + if ls "${PIHOLE_SETUP_VARS_FILE}" 1> /dev/null 2>&1; then + log_write "${INFO} Sourcing ${PIHOLE_SETUP_VARS_FILE}..."; + # source it + source ${PIHOLE_SETUP_VARS_FILE} + else + # If it can't, show an error + log_write "${PIHOLE_SETUP_VARS_FILE} ${COL_LIGHT_RED}does not exist or cannot be read.${COL_NC}" + fi +} + +make_temporary_log() { + # Create a random temporary file for the log + TEMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX) + # Open handle 3 for templog + # https://stackoverflow.com/questions/18460186/writing-outputs-to-log-file-and-console + exec 3>"$TEMPLOG" + # Delete templog, but allow for addressing via file handle + # This lets us write to the log without having a temporary file on the drive, which + # is meant to be a security measure so there is not a lingering file on the drive during the debug process + rm "$TEMPLOG" +} + +log_write() { + # echo arguments to both the log and the console + echo -e "${@}" | tee -a /proc/$$/fd/3 +} + +copy_to_debug_log() { + # Copy the contents of file descriptor 3 into the debug log + cat /proc/$$/fd/3 > "${PIHOLE_DEBUG_LOG}" + # Since we use color codes such as '\e[1;33m', they should be removed before being + # uploaded to our server, since it can't properly display in color + # This is accomplished by use sed to remove characters matching that patter + # The entire file is then copied over to a sanitized version of the log + sed 's/\[[0-9;]\{1,5\}m//g' > "${PIHOLE_DEBUG_LOG_SANITIZED}" <<< cat "${PIHOLE_DEBUG_LOG}" +} + +initiate_debug() { + # Clear the screen so the debug log is readable + clear + show_disclaimer + # Display that the debug process is beginning + log_write "${COL_LIGHT_PURPLE}*** [ INITIALIZING ]${COL_NC}" + # Timestamp the start of the log + log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initiated." +} + +# This is a function for visually displaying the curent test that is being run. +# Accepts one variable: the name of what is being diagnosed +# Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗] +echo_current_diagnostic() { + # Colors are used for visually distinguishing each test in the output + # These colors do not show in the GUI, but the formatting will + log_write "\n${COL_LIGHT_PURPLE}*** [ DIAGNOSING ]:${COL_NC} ${1}" +} + +compare_local_version_to_git_version() { + # The git directory to check + local git_dir="${1}" + # The named component of the project (Core or Web) + local pihole_component="${2}" + # If we are checking the Core versions, + if [[ "${pihole_component}" == "Core" ]]; then + # We need to search for "Pi-hole" when using pihole -v + local search_term="Pi-hole" + elif [[ "${pihole_component}" == "Web" ]]; then + # We need to search for "AdminLTE" so store it in a variable as well + local search_term="AdminLTE" + fi + # Display what we are checking + echo_current_diagnostic "${pihole_component} version" + # Store the error message in a variable in case we want to change and/or reuse it + local error_msg="git status failed" + # If the pihole git directory exists, + if [[ -d "${git_dir}" ]]; then + # move into it + cd "${git_dir}" || \ + # If not, show an error + log_write "${COL_LIGHT_RED}Could not cd into ${git_dir}$COL_NC" + if git status &> /dev/null; then + # The current version the user is on + local remote_version + remote_version=$(git describe --tags --abbrev=0); + # What branch they are on + local remote_branch + remote_branch=$(git rev-parse --abbrev-ref HEAD); + # The commit they are on + local remote_commit + remote_commit=$(git describe --long --dirty --tags --always) + # echo this information out to the user in a nice format + # If the current version matches what pihole -v produces, the user is up-to-date + if [[ "${remote_version}" == "$(pihole -v | awk '/${search_term}/ {print $6}' | cut -d ')' -f1)" ]]; then + log_write "${TICK} ${pihole_component}: ${COL_LIGHT_GREEN}${remote_version}${COL_NC}" + # If not, + else + # echo the current version in yellow, signifying it's something to take a look at, but not a critical error + # Also add a URL to an FAQ + log_write "${INFO} ${pihole_component}: ${COL_YELLOW}${remote_version:-Untagged}${COL_NC} (${FAQ_UPDATE_PI_HOLE})" + fi + + # If the repo is on the master branch, they are on the stable codebase + if [[ "${remote_branch}" == "master" ]]; then + # so the color of the text is green + log_write "${INFO} Branch: ${COL_LIGHT_GREEN}${remote_branch}${COL_NC}" + # If it is any other branch, they are in a developement branch + else + # So show that in yellow, signifying it's something to take a look at, but not a critical error + log_write "${INFO} Branch: ${COL_YELLOW}${remote_branch:-Detached}${COL_NC} (${FAQ_CHECKOUT_COMMAND})" + fi + # echo the current commit + log_write "${INFO} Commit: ${remote_commit}" + # If git status failed, + else + # Return an error message + log_write "${error_msg}" + # and exit with a non zero code + return 1 + fi + else + : + fi +} + +check_ftl_version() { + local ftl_name="FTL" + echo_current_diagnostic "${ftl_name} version" + # Use the built in command to check FTL's version + FTL_VERSION=$(pihole-FTL version) + # Compare the current FTL version to the remote version + if [[ "${FTL_VERSION}" == "$(pihole -v | awk '/FTL/ {print $6}' | cut -d ')' -f1)" ]]; then + # If they are the same, FTL is up-to-date + log_write "${TICK} ${ftl_name}: ${COL_LIGHT_GREEN}${FTL_VERSION}${COL_NC}" + else + # If not, show it in yellow, signifying there is an update + log_write "${TICK} ${ftl_name}: ${COL_YELLOW}${FTL_VERSION}${COL_NC} (${FAQ_UPDATE_PI_HOLE})" + fi +} + +# Checks the core version of the Pi-hole codebase +check_component_versions() { + # Check the Web version, branch, and commit + compare_local_version_to_git_version "${CORE_GIT_DIRECTORY}" "Core" + # Check the Web version, branch, and commit + compare_local_version_to_git_version "${WEB_GIT_DIRECTORY}" "Web" + # Check the FTL version + check_ftl_version +} + + +get_program_version() { + local program_name="${1}" + # Create a loval variable so this function can be safely reused + local program_version + echo_current_diagnostic "${program_name} version" + # Evalutate the program we are checking, if it is any of the ones below, show the version + case "${program_name}" in + "lighttpd") program_version="$(${program_name} -v |& head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)" + ;; + "dnsmasq") program_version="$(${program_name} -v |& head -n1 | awk '{print $3}')" + ;; + "php") program_version="$(${program_name} -v |& head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)" + ;; + # If a match is not found, show an error + *) echo "Unrecognized program"; + esac + # If the program does not have a version (the variable is empty) + if [[ -z "${program_version}" ]]; then + # Display and error + log_write "${CROSS} ${COL_LIGHT_RED}${program_name} version could not be detected.${COL_NC}" + else + # Otherwise, display the version + log_write "${INFO} ${program_version}" + fi +} + +# These are the most critical dependencies of Pi-hole, so we check for them +# and their versions, using the functions above. +check_critical_program_versions() { + # Use the function created earlier and bundle them into one function that checks all the version numbers + get_program_version "dnsmasq" + get_program_version "lighttpd" + get_program_version "php" +} + +is_os_supported() { + local os_to_check="${1}" + # Strip just the base name of the system using sed + the_os=$(echo ${os_to_check} | sed 's/ .*//') + # If the variable is one of our supported OSes, + case "${the_os}" in + # Print it in green + "Raspbian") log_write "${TICK} ${COL_LIGHT_GREEN}${os_to_check}${COL_NC}";; + "Ubuntu") log_write "${TICK} ${COL_LIGHT_GREEN}${os_to_check}${COL_NC}";; + "Fedora") log_write "${TICK} ${COL_LIGHT_GREEN}${os_to_check}${COL_NC}";; + "Debian") log_write "${TICK} ${COL_LIGHT_GREEN}${os_to_check}${COL_NC}";; + "CentOS") log_write "${TICK} ${COL_LIGHT_GREEN}${os_to_check}${COL_NC}";; + # If not, show it in red and link to our software requirements page + *) log_write "${CROSS} ${COL_LIGHT_RED}${os_to_check}${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS})"; esac } -header_write() { - log_echo "" - log_echo "---= ${1}" - log_write "" -} +get_distro_attributes() { + # Put the current Internal Field Separator into another variable so it can be restored later + OLD_IFS="$IFS" + # Store the distro info in an array and make it global since the OS won't change, + # but we'll keep it within the function for better unit testing + IFS=$'\r\n' command eval 'distro_info=( $(cat /etc/*release) )' -file_parse() { - while read -r line; do - if [ ! -z "${line}" ]; then - [[ "${line}" =~ ^#.*$ || ! "${line}" || "${line}" == "WEBPASSWORD="* ]] && continue - log_write "${line}" - fi - done < "${1}" - log_write "" -} - -block_parse() { - log_write "${1}" -} - -lsof_parse() { - local user - local process - - user=$(echo ${1} | cut -f 3 -d ' ' | cut -c 2-) - process=$(echo ${1} | cut -f 2 -d ' ' | cut -c 2-) - [[ ${2} -eq ${process} ]] \ - && echo "::: Correctly configured." \ - || log_echo "::: Failure: Incorrectly configured daemon." - - log_write "Found user ${user} with process ${process}" -} - - -version_check() { - header_write "Detecting Installed Package Versions:" - - local error_found - local pi_hole_ver - local pi_hole_branch - local pi_hole_commit - local admin_ver - local admin_branch - local admin_commit - local light_ver - local php_ver - local status - error_found=0 - - cd "${PIHOLEGITDIR}" &> /dev/null || \ - { status="Pi-hole git directory not found."; error_found=1; } - if git status &> /dev/null; then - pi_hole_ver=$(git describe --tags --abbrev=0) - pi_hole_branch=$(git rev-parse --abbrev-ref HEAD) - pi_hole_commit=$(git describe --long --dirty --tags --always) - log_echo -r "Pi-hole: ${pi_hole_ver:-Untagged} (${pi_hole_branch:-Detached}:${pi_hole_commit})" - else - status=${status:-"Pi-hole repository damaged."} - error_found=1 - fi - if [[ "${status}" ]]; then - log_echo "${status}" - unset status - fi - - cd "${ADMINGITDIR}" || \ - { status="Pi-hole Dashboard git directory not found."; error_found=1; } - if git status &> /dev/null; then - admin_ver=$(git describe --tags --abbrev=0) - admin_branch=$(git rev-parse --abbrev-ref HEAD) - admin_commit=$(git describe --long --dirty --tags --always) - log_echo -r "Pi-hole Dashboard: ${admin_ver:-Untagged} (${admin_branch:-Detached}:${admin_commit})" - else - status=${status:-"Pi-hole Dashboard repository damaged."} - error_found=1 - fi - if [[ "${status}" ]]; then - log_echo "${status}" - unset status - fi - - if light_ver=$(lighttpd -v |& head -n1 | cut -d " " -f1); then - log_echo -r "${light_ver}" - else - log_echo "lighttpd not installed." - error_found=1 - fi - if php_ver=$(php -v |& head -n1); then - log_echo -r "${php_ver}" - else - log_echo "PHP not installed." - error_found=1 - fi - - return "${error_found}" -} - -dir_check() { - header_write "Detecting contents of ${1}:" - for file in $1*; do - header_write "File ${file} found" - echo -n "::: Parsing..." - file_parse "${file}" - echo "done" + # Set a named variable for better readability + local distro_attribute + # For each line found in an /etc/*release file, + for distro_attribute in "${distro_info[@]}"; do + # store the key in a variable + local pretty_name_key=$(echo "${distro_attribute}" | grep "PRETTY_NAME" | cut -d '=' -f1) + # we need just the OS PRETTY_NAME, + if [[ "${pretty_name_key}" == "PRETTY_NAME" ]]; then + # so save in in a variable when we find it + PRETTY_NAME_VALUE=$(echo "${distro_attribute}" | grep "PRETTY_NAME" | cut -d '=' -f2- | tr -d '"') + # then pass it as an argument that checks if the OS is supported + is_os_supported "${PRETTY_NAME_VALUE}" + else + # Since we only need the pretty name, we can just skip over anything that is not a match + : + fi done - echo ":::" + # Set the IFS back to what it was + IFS="$OLD_IFS" } -files_check() { - #Check non-zero length existence of ${1} - header_write "Detecting existence of ${1}:" - local search_file="${1}" - if [[ -s ${search_file} ]]; then - echo -n "::: File exists, parsing..." - file_parse "${search_file}" - echo "done" - return 0 +diagnose_operating_system() { + # error message in a variable so we can easily modify it later (or re-use it) + local error_msg="Distribution unknown -- most likely you are on an unsupported platform and may run into issues." + # Display the current test that is running + echo_current_diagnostic "Operating system" + + # If there is a /etc/*release file, it's probably a supported operating system, so we can + if ls /etc/*release 1> /dev/null 2>&1; then + # display the attributes to the user from the function made earlier + get_distro_attributes else - log_echo "${1} not found!" - return 1 + # If it doesn't exist, it's not a system we currently support and link to FAQ + log_write "${CROSS} ${COL_LIGHT_RED}${error_msg}${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS})" fi - echo ":::" -} - -source_file() { - local file_found=$(files_check "${1}") \ - && (source "${1}" &> /dev/null && echo "${file_found} and was successfully sourced") \ - || log_echo -l "${file_found} and could not be sourced" -} - -distro_check() { - local soft_fail - header_write "Detecting installed OS Distribution" - soft_fail=0 - local distro="$(cat /etc/*release)" && block_parse "${distro}" || (log_echo "Distribution details not found." && soft_fail=1) - return "${soft_fail}" } processor_check() { - header_write "Checking processor variety" - log_write $(uname -m) && return 0 || return 1 -} - -ipv6_check() { - # Check if system is IPv6 enabled, for use in other functions - if [[ $IPV6_ADDRESS ]]; then - ls /proc/net/if_inet6 &>/dev/null - return 0 + echo_current_diagnostic "Processor" + # Store the processor type in a variable + PROCESSOR=$(uname -m) + # If it does not contain a value, + if [[ -z "${PROCESSOR}" ]]; then + # we couldn't detect it, so show an error + PROCESSOR=$(lscpu | awk '/Architecture/ {print $2}') + log_write "${CROSS} ${COL_LIGHT_RED}${PROCESSOR}${COL_NC} has not been tested with FTL, but may still work: (${FAQ_FTL_COMPATIBILITY})" else - return 1 + # Check if the architecture is currently supported for FTL + case "${PROCESSOR}" in + "amd64") "${TICK} ${COL_LIGHT_GREEN}${PROCESSOR}${COL_NC}" + ;; + "armv6l") "${TICK} ${COL_LIGHT_GREEN}${PROCESSOR}${COL_NC}" + ;; + "armv6") "${TICK} ${COL_LIGHT_GREEN}${PROCESSOR}${COL_NC}" + ;; + "armv7l") "${TICK} ${COL_LIGHT_GREEN}${PROCESSOR}${COL_NC}" + ;; + "aarch64") "${TICK} ${COL_LIGHT_GREEN}${PROCESSOR}${COL_NC}" + ;; + # Otherwise, show the processor type + *) log_write "${INFO} ${PROCESSOR}"; + esac fi } -ip_check() { - local protocol=${1} - local gravity=${2} - header_write "Checking IPv${protocol} Stack" - - local ip_addr_list="$(ip -${protocol} addr show dev ${PIHOLE_INTERFACE} | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')" - if [[ -n ${ip_addr_list} ]]; then - log_write "IPv${protocol} on ${PIHOLE_INTERFACE}" - log_write "Gravity configured for: ${2:-NOT CONFIGURED}" - log_write "----" - log_write "${ip_addr_list}" - echo "::: IPv${protocol} addresses located on ${PIHOLE_INTERFACE}" - ip_ping_check ${protocol} - return $(( 0 + $? )) +parse_setup_vars() { + echo_current_diagnostic "Setup variables" + # If the file exists, + if [[ -r "${PIHOLE_SETUP_VARS_FILE}" ]]; then + # parse it + parse_file "${PIHOLE_SETUP_VARS_FILE}" else - log_echo "No IPv${protocol} found on ${PIHOLE_INTERFACE}" - return 1 + # If not, show an error + log_write "${CROSS} ${COL_LIGHT_RED}Could not read ${PIHOLE_SETUP_VARS_FILE}.${COL_NC}" fi } -ip_ping_check() { - local protocol=${1} - local cmd - - if [[ ${protocol} == "6" ]]; then - cmd="ping6" - g_addr="2001:4860:4860::8888" - else - cmd="ping" - g_addr="8.8.8.8" - fi - - local ip_def_gateway=$(ip -${protocol} route | grep default | cut -d ' ' -f 3) - if [[ -n ${ip_def_gateway} ]]; then - echo -n "::: Pinging default IPv${protocol} gateway: " - if ! ping_gateway="$(${cmd} -q -W 3 -c 3 -n ${ip_def_gateway} -I ${PIHOLE_INTERFACE} | tail -n 3)"; then - log_echo "Gateway did not respond." - return 1 - else - log_echo "Gateway responded." - log_write "${ping_gateway}" - fi - echo -n "::: Pinging Internet via IPv${protocol}: " - if ! ping_inet="$(${cmd} -q -W 3 -c 3 -n ${g_addr} -I ${PIHOLE_INTERFACE} | tail -n 3)"; then - log_echo "Query did not respond." - return 1 - else - log_echo "Query responded." - log_write "${ping_inet}" - fi - else - log_echo " No gateway detected." - fi - return 0 -} - -port_check() { - local lsof_value - - lsof_value=$(lsof -i ${1}:${2} -FcL | tr '\n' ' ') \ - && lsof_parse "${lsof_value}" "${3}" \ - || log_echo "Failure: IPv${1} Port not in use" -} - -daemon_check() { - # Check for daemon ${1} on port ${2} - header_write "Daemon Process Information" - - echo "::: Checking ${2} port for ${1} listener." - - if [[ ${IPV6_READY} ]]; then - port_check 6 "${2}" "${1}" - fi - lsof_value=$(lsof -i 4:${2} -FcL | tr '\n' ' ') \ - port_check 4 "${2}" "${1}" -} - -testResolver() { +does_ip_match_setup_vars() { + # Check for IPv4 or 6 local protocol="${1}" - header_write "Resolver Functions Check (IPv${protocol})" - local IP="${2}" - local g_addr - local l_addr - local url - local testurl - local localdig - local piholedig - local remotedig - - if [[ ${protocol} == "6" ]]; then - g_addr="2001:4860:4860::8888" - l_addr="::1" - r_type="AAAA" - else - g_addr="8.8.8.8" - l_addr="127.0.0.1" - r_type="A" - fi - - # Find a blocked url that has not been whitelisted. - url=$(shuf -n 1 "${GRAVITYFILE}" | awk -F ' ' '{ print $2 }') - - testurl="${url:-doubleclick.com}" - - - log_write "Resolution of ${testurl} from Pi-hole (${l_addr}):" - if localdig=$(dig -"${protocol}" "${testurl}" @${l_addr} +short "${r_type}"); then - log_write "${localdig}" - else - log_write "Failed to resolve ${testurl} on Pi-hole (${l_addr})" - fi - log_write "" - - log_write "Resolution of ${testurl} from Pi-hole (${IP}):" - if piholedig=$(dig -"${protocol}" "${testurl}" @"${IP}" +short "${r_type}"); then - log_write "${piholedig}" - else - log_write "Failed to resolve ${testurl} on Pi-hole (${IP})" - fi - log_write "" - - - log_write "Resolution of ${testurl} from ${g_addr}:" - if remotedig=$(dig -"${protocol}" "${testurl}" @${g_addr} +short "${r_type}"); then - log_write "${remotedig:-NXDOMAIN}" - else - log_write "Failed to resolve ${testurl} on upstream server ${g_addr}" - fi - log_write "" -} - -testChaos(){ - # Check Pi-hole specific records - - log_write "Pi-hole dnsmasq specific records lookups" - log_write "Cache Size:" - log_write $(dig +short chaos txt cachesize.bind) - log_write "Upstream Servers:" - log_write $(dig +short chaos txt servers.bind) - log_write "" - -} -checkProcesses() { - header_write "Processes Check" - - echo "::: Logging status of lighttpd, dnsmasq and pihole-FTL..." - PROCESSES=( lighttpd dnsmasq pihole-FTL ) - for i in "${PROCESSES[@]}"; do - log_write "Status for ${i} daemon:" - log_write $(systemctl is-active "${i}") - done - log_write "" -} - -debugLighttpd() { - echo "::: Checking for necessary lighttpd files." - files_check "${LIGHTTPDFILE}" - files_check "${LIGHTTPDERRFILE}" - echo ":::" -} - -countdown() { - local tuvix - tuvix=${TIMEOUT} - printf "::: Logging will automatically teminate in %s seconds\n" "${TIMEOUT}" - while [ $tuvix -ge 1 ] - do - printf ":::\t%s seconds left. " "${tuvix}" - if [[ -z "${WEBCALL}" ]]; then - printf "\r" + # IP address to check for + local ip_address="${2}" + # See what IP is in the setupVars.conf file + local setup_vars_ip=$(cat ${PIHOLE_SETUP_VARS_FILE} | grep IPV${protocol}_ADDRESS | cut -d '=' -f2) + # If it's an IPv6 address + if [[ "${protocol}" == "6" ]]; then + # Strip off the / (CIDR notation) + if [[ "${ip_address%/*}" == "${setup_vars_ip%/*}" ]]; then + # if it matches, show it in green + log_write " ${COL_LIGHT_GREEN}${ip_address%/*}${COL_NC} matches the IP found in ${PIHOLE_SETUP_VARS_FILE}" else - printf "\n" + # otherwise show it in red with an FAQ URL + log_write " ${COL_LIGHT_RED}${ip_address%/*}${COL_NC} does not match the IP found in ${PIHOLE_SETUP_VARS_FILE} (${FAQ_ULA})" fi - sleep 5 - tuvix=$(( tuvix - 5 )) + + else + # if the protocol isn't 6, it's 4 so no need to strip the CIDR notation + # since it exists in the setupVars.conf that way + if [[ "${ip_address}" == "${setup_vars_ip}" ]]; then + # show in green if it matches + log_write " ${COL_LIGHT_GREEN}${ip_address}${COL_NC} matches the IP found in ${PIHOLE_SETUP_VARS_FILE}" + else + # otherwise show it in red + log_write " ${COL_LIGHT_RED}${ip_address}${COL_NC} does not match the IP found in ${PIHOLE_SETUP_VARS_FILE} (${FAQ_ULA})" + fi + fi +} + +detect_ip_addresses() { + # First argument should be a 4 or a 6 + local protocol=${1} + # Use ip to show the addresses for the chosen protocol + # Store the values in an arry so they can be looped through + # Get the lines that are in the file(s) and store them in an array for parsing later + declare -a ip_addr_list=( $(ip -${protocol} addr show dev ${PIHOLE_INTERFACE} | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }') ) + + # If there is something in the IP address list, + if [[ -n ${ip_addr_list} ]]; then + # Local iterator + local i + # Display the protocol and interface + log_write "${TICK} IPv${protocol} address(es) bound to the ${PIHOLE_INTERFACE} interface:" + # Since there may be more than one IP address, store them in an array + for i in "${!ip_addr_list[@]}"; do + # For each one in the list, print it out + does_ip_match_setup_vars "${protocol}" "${ip_addr_list[$i]}" + done + # Print a blank line just for formatting + log_write "" + else + # If there are no IPs detected, explain that the protocol is not configured + log_write "${CROSS} ${COL_LIGHT_RED}No IPv${protocol} address(es) found on the ${PIHOLE_INTERFACE}${COL_NC} interace.\n" + return 1 + fi + # If the protocol is v6 + if [[ "${protocol}" == "6" ]]; then + # let the user know that as long as there is one green address, things should be ok + log_write " ^ Please note that you may have more than one IP address listed." + log_write " As long as one of them is green, and it matches what is in ${PIHOLE_SETUP_VARS_FILE}, there is no need for concern.\n" + log_write " The link to the FAQ is for an issue that sometimes occurs when the IPv6 address changes, which is why we check for it.\n" + fi +} + +ping_ipv4_or_ipv6() { + # Give the first argument a readable name (a 4 or a six should be the argument) + local protocol="${1}" + # If the protocol is 6, + if [[ ${protocol} == "6" ]]; then + # use ping6 + cmd="ping6" + # and Google's public IPv6 address + public_address="2001:4860:4860::8888" + else + # Otherwise, just use ping + cmd="ping" + # and Google's public IPv4 address + public_address="8.8.8.8" + fi +} + +ping_gateway() { + local protocol="${1}" + ping_ipv4_or_ipv6 "${protocol}" + # Check if we are using IPv4 or IPv6 + # Find the default gateway using IPv4 or IPv6 + local gateway + gateway="$(ip -${protocol} route | grep default | cut -d ' ' -f 3)" + + # If the gateway variable has a value (meaning a gateway was found), + if [[ -n "${gateway}" ]]; then + log_write "${INFO} Default IPv${protocol} gateway: ${gateway}" + # Let the user know we will ping the gateway for a response + log_write " * Pinging ${gateway}..." + # Try to quietly ping the gateway 3 times, with a timeout of 3 seconds, using numeric output only, + # on the pihole interface, and tail the last three lines of the output + # If pinging the gateway is not successful, + if ! ${cmd} -c 3 -W 2 -n ${gateway} -I ${PIHOLE_INTERFACE} >/dev/null; then + # let the user know + log_write "${CROSS} ${COL_LIGHT_RED}Gateway did not respond.${COL_NC} ($FAQ_GATEWAY)\n" + # and return an error code + return 1 + # Otherwise, + else + # show a success + log_write "${TICK} ${COL_LIGHT_GREEN}Gateway responded.${COL_NC}" + # and return a success code + return 0 + fi + fi +} + +ping_internet() { + local protocol="${1}" + # Ping a public address using the protocol passed as an argument + ping_ipv4_or_ipv6 "${protocol}" + log_write "* Checking Internet connectivity via IPv${protocol}..." + # Try to ping the address 3 times + if ! ${cmd} -W 2 -c 3 -n ${public_address} -I ${PIHOLE_INTERFACE} >/dev/null; then + # if it's unsuccessful, show an error + log_write "${CROSS} ${COL_LIGHT_RED}Cannot reach the Internet.${COL_NC}\n" + return 1 + else + # Otherwise, show success + log_write "${TICK} ${COL_LIGHT_GREEN}Query responded.${COL_NC}\n" + return 0 + fi +} + +compare_port_to_service_assigned() { + local service_name="${1}" + # The programs we use may change at some point, so they are in a varible here + local resolver="dnsmasq" + local web_server="lighttpd" + local ftl="pihole-FT" + if [[ "${service_name}" == "${resolver}" ]] || [[ "${service_name}" == "${web_server}" ]] || [[ "${service_name}" == "${ftl}" ]]; then + # if port 53 is dnsmasq, show it in green as it's standard + log_write "[${COL_LIGHT_GREEN}${port_number}${COL_NC}] is in use by ${COL_LIGHT_GREEN}${service_name}${COL_NC}" + # Otherwise, + else + # Show the service name in red since it's non-standard + log_write "[${COL_LIGHT_RED}${port_number}${COL_NC}] is in use by ${COL_LIGHT_RED}${service_name}${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_PORTS})" + fi +} + +check_required_ports() { + echo_current_diagnostic "Ports in use" + # Since Pi-hole needs 53, 80, and 4711, check what they are being used by + # so we can detect any issues + local resolver="dnsmasq" + local web_server="lighttpd" + local ftl="pihole-FT" + # Create an array for these ports in use + ports_in_use=() + # Sort the addresses and remove duplicates + while IFS= read -r line; do + ports_in_use+=( "$line" ) + done < <( lsof -i -P -n | awk -F' ' '/LISTEN/ {print $9, $1}' | sort -n | uniq | cut -d':' -f2 ) + + # Now that we have the values stored, + for i in "${!ports_in_use[@]}"; do + # loop through them and assign some local variables + local port_number + port_number="$(echo "${ports_in_use[$i]}" | awk '{print $1}')" + local service_name + service_name=$(echo "${ports_in_use[$i]}" | awk '{print $2}') + # Use a case statement to determine if the right services are using the right ports + case "${port_number}" in + 53) compare_port_to_service_assigned "${resolver}" + ;; + 80) compare_port_to_service_assigned "${web_server}" + ;; + 4711) compare_port_to_service_assigned "${ftl}" + ;; + # If it's not a default port that Pi-hole needs, just print it out for the user to see + *) log_write "[${port_number}] is in use by ${service_name}"; + esac done } -# Continuously append the pihole.log file to the pihole_debug.log file -dumpPiHoleLog() { - trap '{ echo -e "\n::: Finishing debug write from interrupt... Quitting!" ; exit 1; }' INT - echo "::: " - echo "::: --= User Action Required =--" - echo -e "::: Try loading a site that you are having trouble with now from a client web browser.. \n:::\t(Press CTRL+C to finish logging.)" - header_write "pihole.log" - if [ -e "${PIHOLELOG}" ]; then - # Dummy process to use for flagging down tail to terminate - countdown & - tail -n0 -f --pid=$! "${PIHOLELOG}" >&4 - else - log_write "No pihole.log file found!" - printf ":::\tNo pihole.log file found!\n" - fi +check_networking() { + # Runs through several of the functions made earlier; we just clump them + # together since they are all related to the networking aspect of things + echo_current_diagnostic "Networking" + detect_ip_addresses "4" + detect_ip_addresses "6" + ping_gateway "4" + ping_gateway "6" + check_required_ports } -# Anything to be done after capturing of pihole.log terminates -finalWork() { - local tricorder - echo "::: Finshed debugging!" +check_x_headers() { + # The X-Headers allow us to determine from the command line if the Web + # lighttpd.conf has a directive to show "X-Pi-hole: A black hole for Internet advertisements." + # in the header of any Pi-holed domain + # Similarly, it will show "X-Pi-hole: The Pi-hole Web interface is working!" if you view the header returned + # when accessing the dashboard (i.e curl -I pi.hole/admin/) + # server is operating correctly + echo_current_diagnostic "Dashboard and block page" + # Use curl -I to get the header and parse out just the X-Pi-hole one + local block_page + block_page=$(curl -Is localhost | awk '/X-Pi-hole/' | tr -d '\r') + # Do it for the dashboard as well, as the header is different than above + local dashboard + dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r') + # Store what the X-Header shoud be in variables for comparision later + local block_page_working + block_page_working="X-Pi-hole: A black hole for Internet advertisements." + local dashboard_working + dashboard_working="X-Pi-hole: The Pi-hole Web interface is working!" + local full_curl_output_block_page + full_curl_output_block_page="$(curl -Is localhost)" + local full_curl_output_dashboard + full_curl_output_dashboard="$(curl -Is localhost/admin/)" + # If the X-header found by curl matches what is should be, + if [[ $block_page == "$block_page_working" ]]; then + # display a success message + log_write "$TICK ${COL_LIGHT_GREEN}${block_page}${COL_NC}" + else + # Otherwise, show an error + log_write "$CROSS ${COL_LIGHT_RED}X-Header does not match or could not be retrieved.${COL_NC}" + log_write "${COL_LIGHT_RED}${full_curl_output_block_page}${COL_NC}" + fi - # Ensure the file exists, create if not, clear if exists. - truncate --size=0 "${DEBUG_LOG}" - chmod 644 ${DEBUG_LOG} - chown "$USER":pihole ${DEBUG_LOG} - # copy working temp file to final log location - cat /proc/$$/fd/3 >> "${DEBUG_LOG}" - # Straight dump of tailing the logs, can sanitize later if needed. - cat /proc/$$/fd/4 >> "${DEBUG_LOG}" + # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have, + if [[ $dashboard == "$dashboard_working" ]]; then + # then we can show a success + log_write "$TICK ${COL_LIGHT_GREEN}${dashboard}${COL_NC}" + else + # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way + log_write "$CROSS ${COL_LIGHT_RED}X-Header does not match or could not be retrieved.${COL_NC}" + log_write "${COL_LIGHT_RED}${full_curl_output_dashboard}${COL_NC}" + fi +} - echo "::: The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only." +dig_at() { + # We need to test if Pi-hole can properly resolve domain names + # as it is an essential piece of the software + + # Store the arguments as variables with names + local protocol="${1}" + local IP="${2}" + echo_current_diagnostic "Name resolution (IPv${protocol}) using a random blocked domain and a known ad-serving domain" + # Set more local variables + # We need to test name resolution locally, via Pi-hole, and via a public resolver + local local_dig + local pihole_dig + local remote_dig + # Use a static domain that we know has IPv4 and IPv6 to avoid false positives + # Sometimes the randomly chosen domains don't use IPv6, or something else is wrong with them + local remote_url="doubleclick.com" + + # If the protocol (4 or 6) is 6, + if [[ ${protocol} == "6" ]]; then + # Set the IPv6 variables and record type + local local_address="::1" + local pihole_address="${IPV6_ADDRESS%/*}" + local remote_address="2001:4860:4860::8888" + local record_type="AAAA" + # Othwerwise, it should be 4 + else + # so use the IPv4 values + local local_address="127.0.0.1" + local pihole_address="${IPV4_ADDRESS%/*}" + local remote_address="8.8.8.8" + local record_type="A" + fi + + # Find a random blocked url that has not been whitelisted. + # This helps emulate queries to different domains that a user might query + # It will also give extra assurance that Pi-hole is correctly resolving and blocking domains + local random_url=$(shuf -n 1 "${PIHOLE_BLOCKLIST_FILE}" | awk -F ' ' '{ print $2 }') + + # First, do a dig on localhost to see if Pi-hole can use itself to block a domain + if local_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @${local_address} +short "${record_type}"); then + # If it can, show sucess + log_write "${TICK} ${random_url} ${COL_LIGHT_GREEN}is ${local_dig}${COL_NC} via ${COL_CYAN}localhost$COL_NC (${local_address})" + else + # Otherwise, show a failure + log_write "${CROSS} ${COL_LIGHT_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_LIGHT_RED}localhost${COL_NC} (${local_address})" + fi + + # Next we need to check if Pi-hole can resolve a domain when the query is sent to it's IP address + # This better emulates how clients will interact with Pi-hole as opposed to above where Pi-hole is + # just asing itself locally + # The default timeouts and tries are reduced in case the DNS server isn't working, so the user isn't waiting for too long + + # If Pi-hole can dig itself from it's IP (not the loopback address) + if pihole_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @${pihole_address} +short "${record_type}"); then + # show a success + log_write "${TICK} ${random_url} ${COL_LIGHT_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})" + else + # Othewise, show a failure + log_write "${CROSS} ${COL_LIGHT_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_LIGHT_RED}Pi-hole${COL_NC} (${pihole_address})" + fi + + # Finally, we need to make sure legitimate queries can out to the Internet using an external, public DNS server + # We are using the static remote_url here instead of a random one because we know it works with IPv4 and IPv6 + if remote_dig=$(dig +tries=1 +time=2 -"${protocol}" "${remote_url}" @${remote_address} +short "${record_type}" | head -n1); then + # If successful, the real IP of the domain will be returned instead of Pi-hole's IP + log_write "${TICK} ${remote_url} ${COL_LIGHT_GREEN}is ${remote_dig}${COL_NC} via ${COL_CYAN}a remote, public DNS server${COL_NC} (${remote_address})" + else + # Otherwise, show an error + log_write "${CROSS} ${COL_LIGHT_RED}Failed to resolve${COL_NC} ${remote_url} via ${COL_LIGHT_RED}a remote, public DNS server${COL_NC} (${remote_address})" + fi +} + +process_status(){ + # Check to make sure Pi-hole's services are running and active + echo_current_diagnostic "Pi-hole processes" + # Local iterator + local i + # For each process, + for i in "${PIHOLE_PROCESSES[@]}"; do + # get its status via systemctl + local status_of_process=$(systemctl is-active "${i}") + # and print it out to the user + if [[ "${status_of_process}" == "active" ]]; then + # If it's active, show it in green + log_write "${TICK} ${COL_LIGHT_GREEN}${i}${COL_NC} daemon is ${COL_LIGHT_GREEN}${status_of_process}${COL_NC}" + else + # If it's not, show it in red + log_write "${CROSS} ${COL_LIGHT_RED}${i}${COL_NC} daemon is ${COL_LIGHT_RED}${status_of_process}${COL_NC}" + fi + done +} + +make_array_from_file() { + local filename="${1}" + # The second argument can put a limit on how many line should be read from the file + # Since some of the files are so large, this is helpful to limit the output + local limit=${2} + # A local iterator for testing if we are at the limit above + local i=0 + # Set the array to be empty so we can start fresh when the function is used + local file_content=() + # If the file is a directory + if [[ -d "${filename}" ]]; then + # do nothing since it cannot be parsed + : + else + # Otherwise, read the file line by line + while IFS= read -r line;do + # Othwerise, strip out comments and blank lines + new_line=$(echo "${line}" | sed -e 's/#.*$//' -e '/^$/d') + # If the line still has content (a non-zero value) + if [[ -n "${new_line}" ]]; then + # Put it into the array + file_content+=("${new_line}") + else + # Otherwise, it's a blank line or comment, so do nothing + : + fi + # Increment the iterator +1 + i=$((i+1)) + # but if the limit of lines we want to see is exceeded + if [[ -z ${limit} ]]; then + # do nothing + : + elif [[ $i -eq ${limit} ]]; then + break + fi + done < "${filename}" + # Now the we have made an array of the file's content + for each_line in "${file_content[@]}"; do + # Print each line + # At some point, we may want to check the file line-by-line, so that's the reason for an array + log_write " ${each_line}" + done + fi +} + +parse_file() { + # Set the first argument passed to this function as a named variable for better readability + local filename="${1}" + # Put the current Internal Field Separator into another variable so it can be restored later + OLD_IFS="$IFS" + # Get the lines that are in the file(s) and store them in an array for parsing later + IFS=$'\r\n' command eval 'file_info=( $(cat "${filename}") )' + + # Set a named variable for better readability + local file_lines + # For each line in the file, + for file_lines in "${file_info[@]}"; do + if [[ ! -z "${file_lines}" ]]; then + # don't include the Web password hash + [[ "${file_linesline}" =~ ^\#.*$ || ! "${file_lines}" || "${file_lines}" == "WEBPASSWORD="* ]] && continue + # otherwise, display the lines of the file + log_write " ${file_lines}" + fi + done + # Set the IFS back to what it was + IFS="$OLD_IFS" +} + +check_name_resolution() { + # Check name resoltion from localhost, Pi-hole's IP, and Google's name severs + # using the function we created earlier + dig_at 4 "${IPV4_ADDRESS%/*}" + # If IPv6 enabled, + if [[ "${IPV6_ADDRESS}" ]]; then + # check resolution + dig_at 6 "${IPV6_ADDRESS%/*}" + fi +} + +# This function can check a directory exists +# Pi-hole has files in several places, so we will reuse this function +dir_check() { + # Set the first argument passed to tihs function as a named variable for better readability + local directory="${1}" + # Display the current test that is running + echo_current_diagnostic "contents of ${COL_CYAN}${directory}${COL_NC}" + # For each file in the directory, + for filename in ${directory}; do + # check if exists first; if it does, + if ls "${filename}" 1> /dev/null 2>&1; then + # do nothing + : + else + # Otherwise, show an error + log_write "${COL_LIGHT_RED}${directory} does not exist.${COL_NC}" + fi + done +} + +list_files_in_dir() { + # Set the first argument passed to tihs function as a named variable for better readability + local dir_to_parse="${1}" + # Store the files found in an array + local files_found=( $(ls "${dir_to_parse}") ) + # For each file in the array, + for each_file in "${files_found[@]}"; do + if [[ -d "${dir_to_parse}/${each_file}" ]]; then + # If it's a directoy, do nothing + : + elif [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_BLOCKLIST_FILE}" ]] || \ + [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_DEBUG_LOG}" ]] || \ + [[ ${dir_to_parse}/${each_file} == ${PIHOLE_RAW_BLOCKLIST_FILES} ]] || \ + [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_INSTALL_LOG_FILE}" ]] || \ + [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_SETUP_VARS_FILE}" ]] || \ + [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_LOG}" ]] || \ + [[ "${dir_to_parse}/${each_file}" == "${PIHOLE_WEB_SERVER_ACCESS_LOG_FILE}" ]] || \ + [[ ${dir_to_parse}/${each_file} == ${PIHOLE_LOG_GZIPS} ]]; then + : + else + # Then, parse the file's content into an array so each line can be analyzed if need be + for i in "${!REQUIRED_FILES[@]}"; do + if [[ "${dir_to_parse}/${each_file}" == ${REQUIRED_FILES[$i]} ]]; then + # display the filename + log_write "\n${COL_LIGHT_GREEN}$(ls -ld ${dir_to_parse}/${each_file})${COL_NC}" + # Check if the file we want to view has a limit (because sometimes we just need a little bit of info from the file, not the entire thing) + case "${dir_to_parse}/${each_file}" in + # If it's Web server error log, just give the first 25 lines + "${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}") make_array_from_file "${dir_to_parse}/${each_file}" 25 + ;; + # Same for the FTL log + "${PIHOLE_FTL_LOG}") make_array_from_file "${dir_to_parse}/${each_file}" 25 + ;; + # parse the file into an array in case we ever need to analyze it line-by-line + *) make_array_from_file "${dir_to_parse}/${each_file}"; + esac + else + # Otherwise, do nothing since it's not a file needed for Pi-hole so we don't care about it + : + fi + done + fi + done +} + +show_content_of_files_in_dir() { + # Set a local variable for better readability + local directory="${1}" + # Check if the directory exists + dir_check "${directory}" + # if it does, list the files in it + list_files_in_dir "${directory}" +} + +show_content_of_pihole_files() { + # Show the content of the files in each of Pi-hole's folders + show_content_of_files_in_dir "${PIHOLE_DIRECTORY}" + show_content_of_files_in_dir "${DNSMASQ_D_DIRECTORY}" + show_content_of_files_in_dir "${WEB_SERVER_CONFIG_DIRECTORY}" + show_content_of_files_in_dir "${CRON_D_DIRECTORY}" + show_content_of_files_in_dir "${WEB_SERVER_LOG_DIRECTORY}" + show_content_of_files_in_dir "${LOG_DIRECTORY}" +} + +analyze_gravity_list() { + echo_current_diagnostic "Gravity list" + local head_line + local tail_line + # Put the current Internal Field Separator into another variable so it can be restored later + OLD_IFS="$IFS" + # Get the lines that are in the file(s) and store them in an array for parsing later + IFS=$'\r\n' + local gravity_permissions=$(ls -ld "${PIHOLE_BLOCKLIST_FILE}") + log_write "${COL_LIGHT_GREEN}${gravity_permissions}${COL_NC}" + local gravity_head=() + gravity_head=( $(head -n 4 ${PIHOLE_BLOCKLIST_FILE}) ) + log_write " ${COL_CYAN}-----head of $(basename ${PIHOLE_BLOCKLIST_FILE})------${COL_NC}" + for head_line in "${gravity_head[@]}"; do + log_write " ${head_line}" + done + log_write "" + local gravity_tail=() + gravity_tail=( $(tail -n 4 ${PIHOLE_BLOCKLIST_FILE}) ) + log_write " ${COL_CYAN}-----tail of $(basename ${PIHOLE_BLOCKLIST_FILE})------${COL_NC}" + for tail_line in "${gravity_tail[@]}"; do + log_write " ${tail_line}" + done + # Set the IFS back to what it was + IFS="$OLD_IFS" +} + +analyze_pihole_log() { + echo_current_diagnostic "Pi-hole log" + local head_line + # Put the current Internal Field Separator into another variable so it can be restored later + OLD_IFS="$IFS" + # Get the lines that are in the file(s) and store them in an array for parsing later + IFS=$'\r\n' + local pihole_log_permissions=$(ls -ld "${PIHOLE_LOG}") + log_write "${COL_LIGHT_GREEN}${pihole_log_permissions}${COL_NC}" + local pihole_log_head=() + pihole_log_head=( $(head -n 20 ${PIHOLE_LOG}) ) + log_write " ${COL_CYAN}-----head of $(basename ${PIHOLE_LOG})------${COL_NC}" + local error_to_check_for + local line_to_obfuscate + local obfuscated_line + for head_line in "${pihole_log_head[@]}"; do + # A common error in the pihole.log is when there is a non-hosts formatted file + # that the DNS server is attempting to read. Since it's not formatted + # correctly, there will be an entry for "bad address at line n" + # So we can check for that here and highlight it in red so the user can see it easily + error_to_check_for=$(echo ${head_line} | grep 'bad address at') + # Some users may not want to have the domains they visit sent to us + # To that end, we check for lines in the log that would contain a domain name + line_to_obfuscate=$(echo ${head_line} | grep ': query\|: forwarded\|: reply') + # If the variable contains a value, it found an error in the log + if [[ -n ${error_to_check_for} ]]; then + # So we can print it in red to make it visible to the user + log_write " ${CROSS} ${COL_LIGHT_RED}${head_line}${COL_NC} (${FAQ_BAD_ADDRESS})" + else + # If the variable does not a value (the current default behavior), so do not obfuscate anything + if [[ -z ${OBFUSCATE} ]]; then + log_write " ${head_line}" + # Othwerise, a flag was passed to this command to obfuscate domains in the log + else + # So first check if there are domains in the log that should be obfuscated + if [[ -n ${line_to_obfuscate} ]]; then + # If there are, we need to use awk to replace only the domain name (the 6th field in the log) + # so we substitue the domain for the placeholder value + obfuscated_line=$(echo ${line_to_obfuscate} | awk -v placeholder="${OBFUSCATED_PLACEHOLDER}" '{sub($6,placeholder); print $0}') + log_write " ${obfuscated_line}" + else + log_write " ${head_line}" + fi + fi + fi + done + log_write "" + # Set the IFS back to what it was + IFS="$OLD_IFS" +} + +tricorder_use_nc_or_ssl() { + # Users can submit their debug logs using nc (unencrypted) or openssl (enrypted) if available + # Check for openssl first since encryption is a good thing + if command -v openssl &> /dev/null; then + # If the command exists, + log_write " * Using ${COL_LIGHT_GREEN}openssl${COL_NC} for transmission." + # encrypt and transmit the log and store the token returned in a variable + tricorder_token=$(< ${PIHOLE_DEBUG_LOG_SANITIZED} openssl s_client -quiet -connect tricorder.pi-hole.net:${TRICORDER_SSL_PORT_NUMBER} 2> /dev/null) + # Otherwise, + else + # use net cat + log_write "${INFO} Using ${COL_YELLOW}netcat${COL_NC} for transmission." + # Save the token returned by our server in a variable + tricorder_token=$(< ${PIHOLE_DEBUG_LOG_SANITIZED} nc tricorder.pi-hole.net ${TRICORDER_NC_PORT_NUMBER}) + fi +} + + +upload_to_tricorder() { + local username="pihole" + # Set the permissions and owner + chmod 644 ${PIHOLE_DEBUG_LOG} + chown "$USER":"${username}" ${PIHOLE_DEBUG_LOG} + + # Let the user know debugging is complete with something strikingly visual + log_write "" + log_write "${COL_LIGHT_PURPLE}********************************************${COL_NC}" + log_write "${COL_LIGHT_PURPLE}********************************************${COL_NC}" + log_write "${TICK} ${COL_LIGHT_GREEN}** FINISHED DEBUGGING! **${COL_NC}\n" + + # Provide information on what they should do with their token + log_write " * The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only." + log_write " * For more information, see: ${TRICORDER_CONTEST}" + log_write " * If available, we'll use openssl to upload the log, otherwise it will fall back to netcat." + # If pihole -d is running automatically (usually throught the dashboard) if [[ "${AUTOMATED}" ]]; then - echo "::: Debug script running in automated mode, uploading log to tricorder..." - tricorder=$(cat /var/log/pihole_debug.log | nc tricorder.pi-hole.net 9999) + # let the user know + log_write "${INFO} Debug script running in automated mode" + # and then decide again which tool to use to submit it + tricorder_use_nc_or_ssl + # If we're not running in automated mode, else - read -r -p "::: Would you like to upload the log? [y/N] " response + echo "" + # give the user a choice of uploading it or not + # Users can review the log file locally (or the output of the script since they are the same) and try to self-diagnose their problem + read -r -p "[?] Would you like to upload the log? [y/N] " response case ${response} in - [yY][eE][sS]|[yY]) - tricorder=$(cat /var/log/pihole_debug.log | nc tricorder.pi-hole.net 9999) - ;; - *) - echo "::: Log will NOT be uploaded to tricorder." - ;; + # If they say yes, run our function for uploading the log + [yY][eE][sS]|[yY]) tricorder_use_nc_or_ssl;; + # If they choose no, just exit out of the script + *) log_write " * Log will ${COL_LIGHT_GREEN}NOT${COL_NC} be uploaded to tricorder.";exit; esac fi - # Check if tricorder.pi-hole.net is reachable and provide token. - if [ -n "${tricorder}" ]; then - echo "::: ---=== Your debug token is : ${tricorder} Please make a note of it. ===---" - echo "::: Contact the Pi-hole team with your token for assistance." - echo "::: Thank you." - else - echo "::: There was an error uploading your debug log." - echo "::: Please try again or contact the Pi-hole team for assistance." + # Check if tricorder.pi-hole.net is reachable and provide token + # along with some additional useful information + if [[ -n "${tricorder_token}" ]]; then + # Again, try to make this visually striking so the user realizes they need to do something with this information + # Namely, provide the Pi-hole devs with the token + log_write "" + log_write "${COL_LIGHT_PURPLE}***********************************${COL_NC}" + log_write "${COL_LIGHT_PURPLE}***********************************${COL_NC}" + log_write "${TICK} Your debug token is: ${COL_LIGHT_GREEN}${tricorder_token}${COL_NC}" + log_write "${COL_LIGHT_PURPLE}***********************************${COL_NC}" + log_write "${COL_LIGHT_PURPLE}***********************************${COL_NC}" + log_write "" + log_write " * Provide the token above to the Pi-hole team for assistance at" + log_write " * ${FORUMS_URL}" + log_write " * Your log will self-destruct on our server after ${COL_LIGHT_RED}48 hours${COL_NC}." + # If no token was generated + else + # Show an error and some help instructions + log_write "${CROSS} ${COL_LIGHT_RED}There was an error uploading your debug log.${COL_NC}" + log_write " * Please try again or contact the Pi-hole team for assistance." fi - echo "::: A local copy of the Debug log can be found at : /var/log/pihole_debug.log" + # Finally, show where the log file is no matter the outcome of the function so users can look at it + log_write " * A local copy of the debug log can be found at: ${COL_CYAN}${PIHOLE_DEBUG_LOG_SANITIZED}${COL_NC}\n" } -### END FUNCTIONS ### -# Create temporary file for log -TEMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX) -# Open handle 3 for templog -exec 3>"$TEMPLOG" -# Delete templog, but allow for addressing via file handle. -rm "$TEMPLOG" - -# Create temporary file for logdump using file handle 4 -DUMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX) -exec 4>"$DUMPLOG" -rm "$DUMPLOG" - -# Gather version of required packages / repositories -version_check || echo "REQUIRED FILES MISSING" -# Check for newer setupVars storage file -source_file "/etc/pihole/setupVars.conf" -# Gather information about the running distribution -distro_check || echo "Distro Check soft fail" -# Gather processor type -processor_check || echo "Processor Check soft fail" - -ip_check 6 ${IPV6_ADDRESS} -ip_check 4 ${IPV4_ADDRESS} - -daemon_check lighttpd http -daemon_check dnsmasq domain -daemon_check pihole-FTL 4711 -checkProcesses - -# Check local/IP/Google for IPv4 Resolution -testResolver 4 "${IPV4_ADDRESS%/*}" -# If IPv6 enabled, check resolution -if [[ "${IPV6_ADDRESS}" ]]; then - testResolver 6 "${IPV6_ADDRESS%/*}" -fi -# Poll dnsmasq Pi-hole specific queries -testChaos - -debugLighttpd - -files_check "${DNSMASQFILE}" -dir_check "${DNSMASQCONFDIR}" -files_check "${WHITELISTFILE}" -files_check "${BLACKLISTFILE}" -files_check "${ADLISTFILE}" - - -header_write "Analyzing gravity.list" - - gravity_length=$(grep -c ^ "${GRAVITYFILE}") \ - && log_write "${GRAVITYFILE} is ${gravity_length} lines long." \ - || log_echo "Warning: No gravity.list file found!" - -header_write "Analyzing pihole.log" - - pihole_length=$(grep -c ^ "${PIHOLELOG}") \ - && log_write "${PIHOLELOG} is ${pihole_length} lines long." \ - || log_echo "Warning: No pihole.log file found!" - - pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \ - && log_write "${PIHOLELOG} is ${pihole_size}." \ - || log_echo "Warning: No pihole.log file found!" - -header_write "Analyzing pihole-FTL.log" - - FTL_length=$(grep -c ^ "${FTLLOG}") \ - && log_write "${FTLLOG} is ${FTL_length} lines long." \ - || log_echo "Warning: No pihole-FTL.log file found!" - - FTL_size=$(du -h "${FTLLOG}" | awk '{ print $1 }') \ - && log_write "${FTLLOG} is ${FTL_size}." \ - || log_echo "Warning: No pihole-FTL.log file found!" - -tail -n50 "${FTLLOG}" >&3 - -trap finalWork EXIT - -### Method calls for additional logging ### -dumpPiHoleLog +# Run through all the functions we made +make_temporary_log +initiate_debug +# setupVars.conf needs to be sourced before the networking so the values are +# available to the other functions +source_setup_variables +check_component_versions +check_critical_program_versions +diagnose_operating_system +processor_check +check_networking +check_name_resolution +process_status +parse_setup_vars +check_x_headers +analyze_gravity_list +show_content_of_pihole_files +analyze_pihole_log +copy_to_debug_log +upload_to_tricorder diff --git a/advanced/Scripts/update.sh b/advanced/Scripts/update.sh index 779f6c8d..71b7cecd 100755 --- a/advanced/Scripts/update.sh +++ b/advanced/Scripts/update.sh @@ -10,10 +10,7 @@ # This file is copyright under the latest version of the EUPL. # Please see LICENSE file for your rights under this license. - - # Variables - readonly ADMIN_INTERFACE_GIT_URL="https://github.com/pi-hole/AdminLTE.git" readonly ADMIN_INTERFACE_DIR="/var/www/html/admin" readonly PI_HOLE_GIT_URL="https://github.com/pi-hole/pi-hole.git" @@ -43,7 +40,7 @@ GitCheckUpdateAvail() { # @ alone is a shortcut for HEAD. Older versions of git # need @{0} - LOCAL="$("git rev-parse @{0}")" + LOCAL="$(git rev-parse "@{0}")" # The suffix @{upstream} to a branchname # (short form @{u}) refers @@ -52,7 +49,7 @@ GitCheckUpdateAvail() { # (configured with branch..remote and # branch..merge). A missing branchname # defaults to the current one. - REMOTE="$("git rev-parse @{upstream}")" + REMOTE="$(git rev-parse "@{upstream}")" if [[ ${#LOCAL} == 0 ]]; then echo -e " ${COL_LIGHT_RED}Error: Local revision could not be obtained, ask Pi-hole support." @@ -82,7 +79,6 @@ GitCheckUpdateAvail() { } FTLcheckUpdate() { - local FTLversion FTLversion=$(/usr/bin/pihole-FTL tag) local FTLlatesttag @@ -131,7 +127,7 @@ main() { # re-install (i.e. update) FTL if ${FTL_update} && ! ${core_update}; then echo "" - echo -e " ${INFO} FTL out of date" + echo -e " ${INFO} FTL out of date" FTLdetect echo "" fi diff --git a/advanced/Scripts/webpage.sh b/advanced/Scripts/webpage.sh index a49d5283..42272122 100755 --- a/advanced/Scripts/webpage.sh +++ b/advanced/Scripts/webpage.sh @@ -31,7 +31,7 @@ Options: -k, kelvin Set Kelvin as preferred temperature unit -h, --help Show this help dialog -i, interface Specify dnsmasq's interface listening behavior - Add '-h' for more info on interface usage" + Add '-h' for more info on interface usage" exit 0 } @@ -89,6 +89,9 @@ SetWebPassword() { readonly PASSWORD="${args[2]}" readonly CONFIRM="${PASSWORD}" else + # Prevents a bug if the user presses Ctrl+C and it continues to hide the text typed. + # So we reset the terminal via stty if the user does press Ctrl+C + trap '{ echo -e "\nNo password will be set" ; stty sane ; exit 1; }' INT read -s -p "Enter New Password (Blank for no password): " PASSWORD echo "" @@ -218,18 +221,19 @@ Reboot() { } RestartDNS() { - local str="Restarting dnsmasq" - echo -ne " ${INFO} ${str}..." - if [[ -x "$(command -v systemctl)" ]]; then - systemctl restart dnsmasq + local str="Restarting DNS service" + [[ -t 1 ]] && echo -ne " ${INFO} ${str}" + if command -v systemctl &> /dev/null; then + output=$( { systemctl restart dnsmasq; } 2>&1 ) else - service dnsmasq restart + output=$( { service dnsmasq restart; } 2>&1 ) fi - - if [[ "$?" == 0 ]]; then - echo -e "${OVER} ${TICK} ${str}" + + if [[ -z "${output}" ]]; then + [[ -t 1 ]] && echo -e "${OVER} ${TICK} ${str}" else - echo -e "${OVER} ${CROSS} ${str}" + [[ ! -t 1 ]] && OVER="" + echo -e "${OVER} ${CROSS} ${output}" fi } @@ -402,7 +406,7 @@ SetHostRecord() { SetListeningMode() { source "${setupVars}" - + if [[ "$3" == "-h" ]] || [[ "$3" == "--help" ]]; then echo "Usage: pihole -a -i [interface] Example: 'pihole -a -i local' @@ -415,7 +419,7 @@ Interfaces: all Listen on all interfaces, permit all origins" exit 0 fi - + if [[ "${args[2]}" == "all" ]]; then echo -e " ${INFO} Listening on all interfaces, permiting all origins. Please use a firewall!" change_setting "DNSMASQ_LISTENING" "all" diff --git a/automated install/basic-install.sh b/automated install/basic-install.sh index 0878dc30..74e2a61d 100755 --- a/automated install/basic-install.sh +++ b/automated install/basic-install.sh @@ -12,28 +12,49 @@ # pi-hole.net/donate # -# Install with this command (from your Pi): +# Install with this command (from your Linux machine): # # curl -L install.pi-hole.net | bash + +# -e option instructs bash to immediately exit if any command [1] has a non-zero exit status +# We do not want users to end up with a partially working install, so we exit the script +# instead of continuing the installation with something broken set -e + ######## VARIABLES ######### +# For better maintainability, we store as much information that can change in variables +# This allows us to make a change in one place that can propogate to all instances of the variable +# These variables should all be GLOBAL variables, written in CAPS +# Local variables will be in lowercase and will exist only within functions +# It's still a work in progress, so you may see some variance in this guideline until it is complete + +# We write to a temporary file before moving the log to the pihole folder tmpLog=/tmp/pihole-install.log instalLogLoc=/etc/pihole/install.log +# This is an important file as it contains information specific to the machine it's being installed on setupVars=/etc/pihole/setupVars.conf +# Pi-hole uses lighttpd as a Web server, and this is the config file for it lighttpdConfig=/etc/lighttpd/lighttpd.conf +# This is a file used for the colorized output coltable=/opt/pihole/COL_TABLE +# We store several other folders and webInterfaceGitUrl="https://github.com/pi-hole/AdminLTE.git" webInterfaceDir="/var/www/html/admin" piholeGitUrl="https://github.com/pi-hole/pi-hole.git" PI_HOLE_LOCAL_REPO="/etc/.pihole" +# These are the names of piholes files, stored in an array PI_HOLE_FILES=(chronometer list piholeDebug piholeLogFlush setupLCD update version gravity uninstall webpage) +# This folder is where the Pi-hole scripts will be installed PI_HOLE_INSTALL_DIR="/opt/pihole" useUpdateVars=false +# Pi-hole needs an IP address; to begin, these variables are empty since we don't know what the IP is until +# this script can run IPV4_ADDRESS="" IPV6_ADDRESS="" +# By default, query logging is enabled and the dashboard is set to be installed QUERY_LOGGING=true INSTALL_WEB=true @@ -51,13 +72,19 @@ r=$(( r < 20 ? 20 : r )) c=$(( c < 70 ? 70 : c )) ######## Undocumented Flags. Shhh ######## +# These are undocumented flags; some of which we can use when repairing an installation +# The runUnattended flag is one example of this skipSpaceCheck=false reconfigure=false runUnattended=false +# If the color table file exists, if [[ -f ${coltable} ]]; then + # source it source ${coltable} +# Othwerise, else + # Set these values so the installer can still run in color COL_NC='\e[0m' # No Color COL_LIGHT_GREEN='\e[1;32m' COL_LIGHT_RED='\e[1;31m' @@ -68,7 +95,8 @@ else OVER="\r\033[K" fi - +# A simple function that just echoes out our logo in ASCII format +# This lets users know that it is a Pi-hole, LLC product show_ascii_berry() { echo -e " ${COL_LIGHT_GREEN}.;;,. @@ -97,41 +125,63 @@ show_ascii_berry() { # Compatibility distro_check() { +# If apt-get is installed, then we know it's part of the Debian family if command -v apt-get &> /dev/null; then - #Debian Family - ############################################# + # Set some global variables here + # We don't set them earlier since the family might be Red Hat, so these values would be different PKG_MANAGER="apt-get" + # A variable to store the command used to update the package cache UPDATE_PKG_CACHE="${PKG_MANAGER} update" + # An array for something... PKG_INSTALL=(${PKG_MANAGER} --yes --no-install-recommends install) # grep -c will return 1 retVal on 0 matches, block this throwing the set -e with an OR TRUE PKG_COUNT="${PKG_MANAGER} -s -o Debug::NoLocking=true upgrade | grep -c ^Inst || true" - # ######################################### - # fixes for dependency differences - # Debian 7 doesn't have iproute2 use iproute + # Some distros vary slightly so these fixes for dependencies may apply + # Debian 7 doesn't have iproute2 so if the dry run install is successful, if ${PKG_MANAGER} install --dry-run iproute2 > /dev/null 2>&1; then + # we can install it iproute_pkg="iproute2" + # Otherwise, else + # use iproute iproute_pkg="iproute" fi - # Prefer the php metapackage if it's there, fall back on the php5 packages + # We prefer the php metapackage if it's there if ${PKG_MANAGER} install --dry-run php > /dev/null 2>&1; then phpVer="php" + # If not, else + # fall back on the php5 packages phpVer="php5" fi - # ######################################### + + # Since our install script is so large, we need several other programs to successfuly get a machine provisioned + # These programs are stored in an array so they can be looped through later INSTALLER_DEPS=(apt-utils dialog debconf dhcpcd5 git ${iproute_pkg} whiptail) + # Pi-hole itself has several dependencies that also need to be installed PIHOLE_DEPS=(bc cron curl dnsmasq dnsutils iputils-ping lsof netcat sudo unzip wget) + # The Web dashboard has some that also need to be installed + # It's useful to separate the two since our repos are also setup as "Core" code and "Web" code PIHOLE_WEB_DEPS=(lighttpd ${phpVer}-common ${phpVer}-cgi) + # The Web server user, LIGHTTPD_USER="www-data" + # group, LIGHTTPD_GROUP="www-data" + # and config file LIGHTTPD_CFG="lighttpd.conf.debian" + # The DNS server user DNSMASQ_USER="dnsmasq" - test_dpkg_lock() { +# A function to check... +test_dpkg_lock() { + # An iterator used for counting loop iterations i=0 + # fuser is a program to show which processes use the named files, sockets, or filesystems + # So while the command is true while fuser /var/lib/dpkg/lock >/dev/null 2>&1 ; do + # Wait half a second sleep 0.5 + # and increase the iterator ((i=i+1)) done # Always return success, since we only return if there is no @@ -139,15 +189,16 @@ if command -v apt-get &> /dev/null; then return 0 } +# If apt-get is not found, check for rpm to see if it's a Red Hat family OS elif command -v rpm &> /dev/null; then - # Fedora Family + # Then check if dnf or yum is the package manager if command -v dnf &> /dev/null; then PKG_MANAGER="dnf" else PKG_MANAGER="yum" fi -# Fedora and family update cache on every PKG_INSTALL call, no need for a separate update. + # Fedora and family update cache on every PKG_INSTALL call, no need for a separate update. UPDATE_PKG_CACHE=":" PKG_INSTALL=(${PKG_MANAGER} install -y) PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l" @@ -162,113 +213,176 @@ elif command -v rpm &> /dev/null; then LIGHTTPD_CFG="lighttpd.conf.fedora" DNSMASQ_USER="nobody" +# If neither apt-get or rmp/dnf are not found else + # it's not an OS we can support, echo -e " ${CROSS} OS distribution not supported" + # so exit the installer exit fi } +# A function for checking if a folder is a git repository is_repo() { - # Use git to check if directory is currently under VCS, return the value 128 - # if directory is not a repo. Return 1 if directory does not exist. + # Use a named, local variable instead of the vague $1, which is the first arguement passed to this function + # These local variables should always be lowercase local directory="${1}" + # A local variable for the current directory local curdir + # A variable to store the return code local rc - + # Assign the current directory variable by using pwd curdir="${PWD}" + # If the first argument passed to this function is a directory, if [[ -d "${directory}" ]]; then - # git -C is not used here to support git versions older than 1.8.4 + # move into the directory cd "${directory}" + # Use git to check if the folder is a repo + # git -C is not used here to support git versions older than 1.8.4 git status --short &> /dev/null || rc=$? + # If the command was not successful, else - # non-zero return code if directory does not exist + # Set a non-zero return code if directory does not exist rc=1 fi + # Move back into the directory the user started in cd "${curdir}" + # Return the code; if one is not set, return 0 return "${rc:-0}" } +# A function to clone a repo make_repo() { + # Set named variables for better readability local directory="${1}" local remoteRepo="${2}" + # The message to display when this function is running str="Clone ${remoteRepo} into ${directory}" + # Display the message and use the color table to preface the message with an "info" indicator echo -ne " ${INFO} ${str}..." - # Clean out the directory if it exists for git to clone into + # If the directory exists, if [[ -d "${directory}" ]]; then + # delete everything in it so git can clone into it rm -rf "${directory}" fi + # Clone the repo and return the return code from this command git clone -q --depth 1 "${remoteRepo}" "${directory}" &> /dev/null || return $? + # Show a colored message showing it's status echo -e "${OVER} ${TICK} ${str}" + # Always return 0? Not sure this is correct return 0 } +# We need to make sure the repos are up-to-date so we can effectively install Clean out the directory if it exists for git to clone into update_repo() { + # Use named, local variables + # As you can see, these are the same variable names used in the last function, + # but since they are local, their scope does not go beyond this function + # This helps prevent the wrong value from being assigned if you were to set the variable as a GLOBAL one local directory="${1}" local curdir + # A variable to store the message we want to display; + # Again, it's useful to store these in variables in case we need to reuse or change the message; + # we only need to make one change here local str="Update repo in ${1}" + + # Make sure we know what directory we are in so we can move back into it curdir="${PWD}" + # Move into the directory that was passed as an argument cd "${directory}" &> /dev/null || return 1 - # Pull the latest commits + # Let the user know what's happening echo -ne " ${INFO} ${str}..." + # Stash any local commits as they conflict with our working code git stash --all --quiet &> /dev/null || true # Okay for stash failure - git clean --force -d || true # Okay for already clean directory + git clean --quiet --force -d || true # Okay for already clean directory + # Pull the latest commits git pull --quiet &> /dev/null || return $? + # Show a completion message echo -e "${OVER} ${TICK} ${str}" + # Move back into the oiginal directory cd "${curdir}" &> /dev/null || return 1 return 0 } +# A function that combines the functions previously made getGitFiles() { - # Setup git repos for directory and repository passed - # as arguments 1 and 2 + # Setup named variables for the git repos + # We need the directory local directory="${1}" + # as well as the repo URL local remoteRepo="${2}" + # A local varible containing the message to be displayed local str="Check for existing repository in ${1}" + # Show the message echo -ne " ${INFO} ${str}..." + # Check if the directory is a repository if is_repo "${directory}"; then + # Show that we're checking it echo -e "${OVER} ${TICK} ${str}" + # Update the repo, returning an error message on failure update_repo "${directory}" || { echo -e "\n ${COL_LIGHT_RED}Error: Could not update local repository. Contact support.${COL_NC}"; exit 1; } + # If it's not a .git repo, else + # Show an error echo -e "${OVER} ${CROSS} ${str}" + # Attempt to make the repository, showing an error on falure make_repo "${directory}" "${remoteRepo}" || { echo -e "\n ${COL_LIGHT_RED}Error: Could not update local repository. Contact support.${COL_NC}"; exit 1; } fi + # echo a blank line echo "" + # and return success? return 0 } +# Reset a repo to get rid of any local changed resetRepo() { + # Use named varibles for arguments local directory="${1}" - + # Move into the directory cd "${directory}" &> /dev/null || return 1 + # Store the message in a varible str="Resetting repository within ${1}..." + # Show the message echo -ne " ${INFO} ${str}" + # Use git to remove the local changes git reset --hard &> /dev/null || return $? + # And show the status echo -e "${OVER} ${TICK} ${str}" + # Returning success anyway? return 0 } +# We need to know the IPv4 information so we can effectively setup the DNS server +# Without this information, we won't know where to Pi-hole will be found find_IPv4_information() { + # Named, local variables local route - # Find IP used to route to outside world + # Find IP used to route to outside world by checking the the route to Google's public DNS server route=$(ip route get 8.8.8.8) + # Use awk to strip out just the interface device as it is used in future commands IPv4dev=$(awk '{for (i=1; i<=NF; i++) if ($i~/dev/) print $(i+1)}' <<< "${route}") + # Get just the IP address IPv4bare=$(awk '{print $7}' <<< "${route}") + # Append the CIDR notation to the IP address IPV4_ADDRESS=$(ip -o -f inet addr show | grep "${IPv4bare}" | awk '{print $4}' | awk 'END {print}') + # Get the default gateway (the way to reach the Internet) IPv4gw=$(awk '{print $3}' <<< "${route}") } +# Get available interfaces that are UP get_available_interfaces() { - # Get available UP interfaces. + # There may be more than one so it's all stored in a variable availableInterfaces=$(ip --oneline link show up | grep -v "lo" | awk '{print $2}' | cut -d':' -f1 | cut -d'@' -f1) } +# A function for displaying the dialogs the user sees when first running the installer welcomeDialogs() { - # Display the welcome dialog + # Display the welcome dialog using an approriately sized window via the calculation conducted earlier in the script whiptail --msgbox --backtitle "Welcome" --title "Pi-hole automated installer" "\n\nThis installer will transform your device into a network-wide ad blocker!" ${r} ${c} - # Support for a part-time dev + # Request that users donate if they enjoy the software since we all work on it in our free time whiptail --msgbox --backtitle "Plea" --title "Free and open source" "\n\nThe Pi-hole is free, but powered by your donations: http://pi-hole.net/donate" ${r} ${c} # Explain the need for a static address @@ -277,43 +391,54 @@ welcomeDialogs() { In the next section, you can choose to use your current network settings (DHCP) or to manually edit them." ${r} ${c} } +# We need to make sure there is enough space before installing, so there is a function to check this verifyFreeDiskSpace() { # 50MB is the minimum space needed (45MB install (includes web admin bootstrap/jquery libraries etc) + 5MB one day of logs.) # - Fourdee: Local ensures the variable is only created, and accessible within this function/void. Generally considered a "good" coding practice for non-global variables. local str="Disk space check" + # Reqired space in KB local required_free_kilobytes=51200 + # Calculate existing free space on this machine local existing_free_kilobytes=$(df -Pk | grep -m1 '\/$' | awk '{print $4}') - # - Unknown free disk space , not a integer + # If the existing space is not an integer, if ! [[ "${existing_free_kilobytes}" =~ ^([0-9])+$ ]]; then + # show an error that we can't determine the free space echo -e " ${CROSS} ${str} Unknown free disk space! We were unable to determine available free disk space on this system. You may override this check, however, it is not recommended The option '${COL_LIGHT_RED}--i_do_not_follow_recommendations${COL_NC}' can override this e.g: curl -L https://install.pi-hole.net | bash /dev/stdin ${COL_LIGHT_RED}