Commit graph

134 commits

Author SHA1 Message Date
Christian König
b30d729aa4 Simplify vw_adlist
Signed-off-by: Christian König <ckoenig@posteo.de>
2021-10-08 21:08:10 +02:00
Adam Warner
62f29ba3fa
Merge branch 'release/v5.4' into development 2021-09-11 21:35:11 +01:00
DL6ER
a1ee7d92a9
Add possibly missing capabilities
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-09-06 22:38:16 +02:00
Adam Warner
c88c943cda
Merge branch 'release/v5.4' into dev-release5.4
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2021-08-13 20:24:35 +01:00
DL6ER
43b88cd628
Merge pull request #4236 from MichaIng/patch-4
Update pihole-FTL.service
2021-08-10 16:09:41 +02:00
Adam Warner
913dcead7f
move chmod/chown of macvendor.db to pihole-FTL.service
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2021-08-05 19:02:36 +01:00
MichaIng
57c40cbd09 Update pihole-FTL.service
Make this script a bourne shell script, which requires the removal of only a single bashism, the "{n..m}" expansion. Furthermore, since POSIX echo has no reliable command line options, switch to printf when line breaks shall be omitted. On most distros/setups "sh" calls a much lighter bourne shell like dash, which inits and runs much faster than bash.

Remove unused PIDFILE variable, remove the single case of FTLUSER call and remove it as well. Using variables here might give the wrong impression that there is a change these can be varied. But both are hardcoded in many places throughout Pi-hole, so in this service script.

Consolidate and merge the commands to pre-create and set permissions for required files and directories. The /var/log/pihole directory is and was never used, the touch, chmod and chown call can be merged into one each to reduce overhead. Use "-f" option to to fail on missing database files instead of redirecting STDERR, which is otherwise helpful to debug other possible errors, like missing or corrupted commands, filesystem errors and such.

Do not use "which pihole-FTL" when setting capabilities when the hardcoded path /usr/bin/pihole-FTL is used for the actual daemon call. It makes sense to use the full path here, as the Pi-hole installer and updater installs it explicitly there, and so we prevent users from e.g. overriding it via /usr/local/bin/pihole-FTL too easily.

On pgrep and pkill calls, add the "-x" flag to assure that only "pihole-FTL" is matched and not "foo-pihole-FTL" or "pihole-FTL-bar".

Do not remove possible leftovers from previous pihole-FTL processes on start, but on stop instead. Since "start" includes a proceeding "stop" as well, on service start nothing changes, but on service stop, some resources are now freed.

Remove leading "$" from usage message. In bash this was omitted, as $'...' is a special syntax for escape sequence expansion, which is not applicable here. In dash it would be printed literally. To keep previous behaviour, it is hence removed.

Signed-off-by: MichaIng <micha@dietpi.com>
2021-07-24 14:55:54 +02:00
Faye Duxovni
aa5c15a728 give pihole its own logrotate state file
Signed-off-by: Faye Duxovni <duxovni@duxovni.org>

Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-07-09 20:00:10 -04:00
DL6ER
6b33b8b4c0
Store status of downloaded list (downloaded, using cache, some error, ...) and number of (in-)valid domains on this list in the gravity database. This updates the gravity databaes to version 14.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-28 08:08:49 +01:00
Michael Woolweaver
3a05ac27a2
Fix typo found by #3933
Signed-off-by: Michael Woolweaver <m.woolweaver@icloud.com>
2020-12-19 17:37:17 -06:00
DL6ER
12b9748c43
Fine-tune adlist:adte_modified trigger and don't default new date_updated column to now but use NULL instead
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-09-07 21:11:57 +02:00
DL6ER
4fd412d7c3
Add date_updated field in adlist table set when a list changes.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-09-07 20:59:23 +02:00
DL6ER
8f7f0881c6
Add /usr/sbin to cronjob PATH
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-07-21 21:21:24 +02:00
DL6ER
17aabf26f7
Also use pkill/pgrep in the FTL service script
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-07-21 21:21:17 +02:00
DL6ER
202ba055ea
Change UNIQUEness constraint from (domain) to (domain, type) in the domainlist table. This will allow duplicates which can be associated to different groups.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-06-02 10:49:35 +02:00
DL6ER
de8976da7a
Add CAP_SYS_NICE to FTL's capabilities so it can control its own niceness.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-05-28 21:18:31 +02:00
MichaIng
5c17e41cf1 Start pihole-FTL after network
"$network" on a systemd-driven OS leads to "After=network-online.target" in the generated systemd unit.
This target is no guarantee that all network interfaces have been fully configured, as it depends on the related network services types, but at least it reduces the risk that those have not fully finished their job when pihole-FTL starts. If this is the case, certain issues can occur:
- https://github.com/pi-hole/pi-hole/issues/2924
- https://discourse.pi-hole.net/t/have-to-pihole-restartdns-after-reboot/28772

Runtime files are now consistently created in "/run" instead of "/var/run". The second is a symlink to the first for backwards compatibility but on none-ancient distro versions one should use "/run", systemd even prints a warnings if service files use "/var/run". The service file used "/run" and "/var/run" both, in cases for the same files/directories before, which does not directly cause issues currently, due to the symlink, but is inconsistent at best.

Signed-off-by: MichaIng <micha@dietpi.com>
2020-05-16 13:38:18 +02:00
Dan Schaper
10addc5923
Merge pull request #3218 from JulienOrain/patch-1
fix typo
2020-05-14 09:48:59 -07:00
DL6ER
7cc35d3b04
Add update to gravity database version 12, renaming the Unassociated group to Default group.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-05-07 18:01:37 +02:00
Julien Orain
6d4844a0b3 docs(typo): fix typo
Signed-off-by: Julien Orain <julien.orain@gmail.com>
2020-03-18 21:41:54 +01:00
DL6ER
e528903488
Merge pull request #3107 from pi-hole/new/client_comments
Add timestamps and comment fields to clients table
2020-02-12 19:35:01 +01:00
DL6ER
92aa510bda
Add timestamps and comment fields to clients. This updates the gravity database to version 11.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-01-27 10:36:16 +00:00
DL6ER
6b04997fc3
DROP and reCREATE TRIGGERs during gravity swapping.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-01-27 10:12:05 +00:00
DL6ER
10c2dad48a Improve gravity performance (#3100)
* Gravity performance improvements.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not move downloaded lists into migration_backup directory.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not (strictly) sort domains. Random-leaf access is faster than always-last-leaf access (on average).

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Append instead of overwrite gravity_new collection list.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Rename table gravity_new to gravity_temp to clarify that this is only an intermediate table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Add timers for each of the calls to compute intense parts. They are to be removed before this finally hits the release/v5.0 branch.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Fix legacy list files import. It currently doesn't work when the gravity database has already been updated to using the single domainlist table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Simplify database_table_from_file(), remove all to this function for gravity lost downloads.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Update gravity.db.sql to version 10 to have newle created databases already reflect the most recent state.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Create second gravity database and swap them on success. This has a number of advantages such as instantaneous gravity updates (as seen from FTL) and always available gravity blocking. Furthermore, this saves disk space as the old database is removed on completion.

* Add timing output for the database swapping SQLite3 call.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Explicitly generate index as a separate process.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Remove time measurements.

Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-01-24 09:39:13 -08:00
Adam Warner
ec09b5843c
Merge branch 'development' into fix/do_not_force_local_resolver 2019-12-11 19:09:02 +00:00
DL6ER
620e1e9c73
Do not force nameserver 127.0.0.1 through resolvconf in pihole-FTL.service
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-09 12:23:42 +00:00
DL6ER
8a119d72e2
Ensure database permissions are set up correctly by the service script.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-09 12:17:55 +00:00
chrunchyjesus
476975540a
make some shebangs comply to posix standard 2019-11-05 22:33:00 +01:00
DL6ER
3cb4f6d9d4
We cannot create vw_gravity before having created vw_whitelist as the former depends onthe later. This commit changes the order in which the tables are created.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-09-26 13:50:54 +02:00
DL6ER
acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-07 10:33:08 +02:00
DL6ER
0405aaa3da
Review comments and fixing stickler complaints.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-06 09:32:41 +02:00
DL6ER
2fb4256f84
Rename table to "auditlist"
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-05 16:28:36 +02:00
DL6ER
1dbe6c83c3
Add database upgrading mechanism for adding the audit table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-05 13:54:18 +02:00
DL6ER
23b3a9a650
Add DISTINCT to the view\'s SELECT queries to avoid domain duplication in case they are enabled in multiple groups at the same time.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-01 09:44:20 +02:00
DL6ER
f91421418a
Rename adlists -> adlist table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 23:21:10 +02:00
DL6ER
3a14e8b013
Check whitelist_by_group.group_id instead of group.enabled against NULL.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 23:20:13 +02:00
DL6ER
ad97e95f2f
Add required name column in group table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 23:06:15 +02:00
DL6ER
5b01facd73
Fix logix on the views: AND ( OR )
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 20:03:22 +02:00
DL6ER
6fe637b9ee
Rename groups -> group
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 19:43:18 +02:00
DL6ER
94d83dbb2d
Unify the two group tables into a single "groups" table. This allows managing adlists and domains together.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 19:33:02 +02:00
DL6ER
87c115fc86
Change implementation to use linking tables (domain_groups, adlists_groups). This allows adding domains to multiple lists conveniently.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 19:04:38 +02:00
DL6ER
0774c4e5ca
vw_gravity should actually filter using vw_whitelist.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-16 18:52:23 +02:00
DL6ER
01850c2128
Automatically recreate standard group after deletion to avoid violating foreign key constraints even if foreign key enforcing is disabled (SQLite's default behavior)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-16 17:15:36 +02:00
DL6ER
cbc210b014
Add adlist_groups and mark group_id columns as NOT NULL.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-16 17:05:33 +02:00
DL6ER
c53ee4202b
Add filtering by domain groups to gravity database.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-16 16:50:51 +02:00
DL6ER
ae98fde321
Try to obtain PID from PIDFILE. If this fails (file does not exist or is empty), fall back to using pidof + awk
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-31 09:18:12 +02:00
DL6ER
f6213d4f4d
Use last PID in case pidof returns multiple PIDs for pihole-FTL
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-30 15:26:27 +02:00
DL6ER
d46ff1d2d7
Remove UNIQUE on gravity table as the only column is already PRIMARY KEY. Primary keys must contain UNIQUE values, and cannot contain NULL values
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 22:08:30 +02:00
DL6ER
2f698904e4
Use PRIMARY KEY instead of NOT NULL also in gravity and info tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 18:03:35 +02:00
DL6ER
106f079afc
Improve indentation (single tab) in SQLite template
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:57:23 +02:00
DL6ER
6c1e3a17f7
Remove NOT NULL constraint on list ids as this is implied by PRIMARY KEY (even though most, if not all, examples on the web still use both attributes at the same time)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:54:29 +02:00
DL6ER
5e26e8245b
Order results of queries to list views by id key
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:51:10 +02:00
DL6ER
983117d788
Add id (primary key, autoincrement) field to lists tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:47:15 +02:00
DL6ER
d5781fb110
Fix extra space in front on END statement in CREATE TRIGGER commands
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 16:38:41 +02:00
DL6ER
487718512d
Remove DISTINCT attribute from views. They are not needed as the corresponding data is obtained from fields with UNIQUE attribute so duplications are already prevented at domain insertion. The elimination of the duplication check on imort through the views slightly reduces the load during import (I observed about 8 percent time saved on my NanoPi NEO).
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 16:37:23 +02:00
DL6ER
2180531a84
Add trigger for automatic updates of the date_modified column and improve formating in the gravity.db template
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-29 19:43:35 +02:00
DL6ER
a932209143
Set default for date_modified to be the current time
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-29 17:21:59 +02:00
DL6ER
126f7101ff
Add (optional) date_modified field and use vw_whitelist view in vw_blacklist
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-28 14:44:03 +02:00
DL6ER
bd0215524b
Add vw_adlists view which returns only enabled adlists
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:36:00 +02:00
DL6ER
5bb8163674
Change type of whitelist, blacklist, regex, and adlists time column to INTEGER and insert a DEFAULT clause that returns the current Unix time as such an integer
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:26:15 +02:00
DL6ER
61cfd2f9f9
Add adlists table
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:48:06 +02:00
DL6ER
0e14d11a20
Merge branch 'development' into new/gravitydb 2019-02-22 23:06:17 +01:00
DL6ER
5ca1bfc11c
Delete /dev/shm/FTL-* files on (re)start of pihole-FTL
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-08 15:00:35 +01:00
DL6ER
644ec36e64
Review comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-06 18:57:48 +01:00
DL6ER
9984647ebb
After renaming column in regex, it also needs to be renamed in vw_regex
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:41:11 +01:00
DL6ER
c3c60e10f1
Rename regex field from domain to filter because this described better what it is
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:07:12 +01:00
DL6ER
8a2363621d
Rename fields from DateAdded to dateadded
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:06:46 +01:00
DL6ER
dbbf21071b
Invert logic to use ENABLED instead of DISABLED. We make sure to set "enabled = 1 WHERE enabled IS NULL" in gravity_store_in_database()
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:05:11 +01:00
DL6ER
93f1859bab
Store regex in gravity.db as well
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 15:13:18 +01:00
DL6ER
dcf0a605cf
Use views for all tables and set disabled column to false for those tables that support it
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 14:14:14 +01:00
DL6ER
710036adae
Add DISABLED field to white- and blacklist defaulting to false. The two views respect the DISABLED fields for both lists.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 13:34:08 +01:00
DL6ER
09c4c88a6d
Create and fill gravity.db during pihole -g
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 13:04:31 +01:00
DL6ER
d0125d5aad
Suppress warning if Pi-hole is not used as DHCP server (/etc/pihole/dhcp.leases is not present in this case)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-11-19 17:38:19 +01:00
jeremysherriff
d3ac8bb6f8 Silence updateGravity job unless errors occur
No need for append/logrotate as we are only interested in the latest output
Signed-off-by: jeremysherriff <jeremysherriff@gmail.com>
2018-10-19 19:10:13 +13:00
DL6ER
b043000044
Improve template by removing it. PHP cannot digest comments starting in "#" as of v7.0.0
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-09-28 12:08:39 +02:00
DL6ER
c338eab3c6
According to the PHP documentation, PHP 7.0.0+ does not longer recognize hash marks (#) as comments. As FTL has been trained to understand PHP-style (comment lines start with ";") as well, we switch to using them.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-09-28 11:11:30 +02:00
DL6ER
b9d47d0dd2
Merge pull request #2378 from pi-hole/fix/pihole-FTL-no-caps
Start pihole-FTL as root if capabilities are not supported by the system
2018-08-21 14:11:24 +02:00
DL6ER
ec2426b24d
Add clarifying comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-19 19:55:47 +02:00
DL6ER
5cd3b11391
Start pihole-FTL as root if capabilities are not supported by the system
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-19 14:32:19 +02:00
DL6ER
d30b565d98
Touch dhcp.leases to ensure it exists. There are some systems out there where the installer didn't finish on the first run and some users don't have this file being created. Although /etc/pihole should be owned by pihole:pihole, pihole-FTL sometimes fails to open this file and - if this is the case and DHCP is enabled - refuses to start altogether.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-19 13:53:34 +02:00
Dan Schaper
af2ec61965
Convert [] to ()
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-07-29 08:17:57 -07:00
Dan Schaper
94b7c8eb59
Added header for file. Not using standard Pi-hole copyright header.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-07-28 19:58:23 -07:00
Dan Schaper
f2c675cff6
Formatted pihole-FTL.conf for readability.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-07-28 16:16:16 -07:00
Dan Schaper
b79ff05e29
Move template files to own subdir. Include pihole-FTL.conf in templates.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-07-27 15:21:25 -07:00