Pi-hole/pi-hole
Pi-hole/pi-hole
1
0
Fork 0
mirror of https://github.com/pi-hole/pi-hole.git synced 2024-11-27 08:33:18 +00:00
Code Issues Projects Releases Packages Wiki Activity
6470 commits 8 branches 109 tags 27 MiB
Commit graph

573 commits

Author SHA1 Message Date
DL6ER
19d3489bcb
gravity_DownloadBlocklistFromUrl needs ${domain} but it was declared local before and was not passed as argument, making gravity currently depend on undefined behavior. It seems to be working well in the vast majority of cases, however, it seems we have at least one report where it is not working. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-11-22 20:56:23 +01:00
Christian König
de6e61705e
Remove local.list and openVPN traces 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-11-06 21:40:32 +01:00
Christian König
fd8fdd3513
Use suffixed temp file 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-10-23 21:36:18 +02:00
DL6ER
7523c49f62
Swapping the databases must be the last step before the cleanup. Otherwise, FTL may be reloading from an only partially completed database causing spurious errors and/or warnings. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-10-22 08:14:11 +02:00
RD WebDesign
27522fbc18
Add a final message to gravity 
The terminal version doesn't really need a final message, but this will be
read by the web interface to show a success message

Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2023-10-16 17:19:44 -03:00
Adam Warner
aba41b45b0
Some verbiage change to outputs 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-10-04 16:20:38 +01:00
DL6ER
f19a56cf87
Merge branch 'development-v6' into new/antigravity 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-09-11 11:43:56 +02:00
Christian König
cdc1c1ace1
Remove user agent when downloading adlists 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-09-03 23:40:58 +02:00
Christian König
eca84e0986
Remove user agent when downloading adlists 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-08-27 14:59:47 +02:00
DL6ER
2a03671fb9
Reinstall gravity cleanup on error 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-07-18 08:13:36 +01:00
DL6ER
35512c4dc9
Fix adlist.list migration step failing during tests 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-07-18 08:13:36 +01:00
DL6ER
00cbb8bc8a
Add antigravity support to gravity 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-07-18 08:13:36 +01:00
Adam Warner
a3bb3872bf
adlist table now contains 11 columns, not 10. 
Co-authored-by: Christian König <ckoenig@posteo.de>
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-06-18 21:10:12 +01:00
Christian König
7351a4d3b1
Merge branch 'development' into development-v6 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-06-01 15:45:05 +02:00
DL6ER
6c302c9bc8
Adlist properties need to be stored in the new ("temp") database rather than the old. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-05-30 17:01:58 +02:00
Adam Warner
ea748822ef
REVISIT: I don't _think_ this line is nessacery. Why should gravity restart FTL if it is offline? It might be offline on purpose. 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-05-27 11:53:00 +01:00
Adam Warner
a91eb48d48
Remove some code from gravity that gets values from setupVars.conf / pihole-FTL.conf - use getFTLConfigValue from utils.sh instead 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-05-27 11:51:16 +01:00
Adam Warner
cd17040f95
setupVars.conf not existing is no longer a critical error 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-05-27 11:51:16 +01:00
Adam Warner
dbc6b814b2
Should not need to reloadDNS manually any more as FTL will detect that it needs to reload by itself 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-05-27 11:51:16 +01:00
DL6ER
3695610300
Allow running pihole -g without root (run as user pihole) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2023-05-27 11:51:16 +01:00
DL6ER
73733308ba
Use parseList function offered by pihole-FTL 2023-05-15 19:25:56 +02:00
Christian König
eaded9fdb1
Remove forgotten variables 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-05-11 22:16:21 +02:00
Christian König
a3e610dbf2
Don't use '--suffix' in mktemp 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-05-10 06:52:51 +02:00
yubiuser
cd638b102f
Filter ABP CSS selectors (#5247) 2023-05-04 12:01:20 +02:00
Christian König
dd3a7a4edb
Only delete lines containing separator when preceded by a letter to reduce false positiv (deleting valid comments) 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-08 14:59:26 +02:00
Christian König
d10d59303e
There is no ! but an $ rule 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 21:44:49 +02:00
Christian König
aaf828117d
Remove unecessary $cmd_ext 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 12:36:50 +02:00
Christian König
9c4e74ffa7
Remove special handling of pgl.yoyo.org 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 12:23:11 +02:00
Christian König
2a0f720153
Don't delete lines containing / as they should count as invalid 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 10:25:25 +02:00
Christian König
cc17fe18a9
Remove lines with ABP extended CSS selector 
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 10:14:59 +02:00
Christian König
61ff5b2c76
Unifiy sed commands 
Co-authored-by: RD WebDesign <github@rdwebdesign.com.br>
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-04-07 09:44:31 +02:00
RD WebDesign
66bfa606a7
Using a better text for the comment 
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2023-03-29 14:17:41 -03:00
RD WebDesign
32fb2e69ff
Spliting the regex into TLD_pattern and subdomain_pattern 
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2023-03-29 00:05:01 -03:00
RD WebDesign
c71460e4b6
Allow TLD blocking using ABP style 
This validates patterns without dots (only for abp style), allowing TLDs to be blocked

Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2023-03-28 17:21:50 -03:00
Adam Warner
0656ceb149
Speed things up a bit with some humble greps. Consolodate regexes and remove the need for so many mapfile/arrays 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>

Use temp files for parsing and remove when done.

Always rm the non-domains temp file.

exit 1 if gravity database creation fails.

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: DL6ER <dl6er@dl6er.de>
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
 2023-03-25 11:19:23 -07:00
kot0dama
76b7453f90
Add configurable GRAVITY_TMPDIR variable into setupVars 
Signed-off-by: Loïc Gomez <89980752+kot0dama@users.noreply.github.com>
 2023-03-22 19:40:24 +09:00
Christian König
71e262c37f
Revert "Allow final dot (root zone)" 
This reverts commit 0b5da9f0da.

Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-03-14 19:50:53 +01:00
Christian König
0b5da9f0da
Allow final dot (root zone) 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-03-04 10:36:07 +01:00
Christian König
16385af3ef
Use dedicated pattern variable to make RegEx reusable 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-26 21:16:45 +01:00
Christian König
6cb0be82ca
Add flag abp_domains into info table to signal if abp domains have been found 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-26 10:34:17 +01:00
Christian König
821c7dc190
Add info when list cotains ABP style domains 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-24 22:18:41 +01:00
Christian König
eedd93d782
Remove temporary files 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-24 21:55:22 +01:00
Adam Warner
1f3f849106
Remove lines starting with ! or [ to account for ABP style comments and header 
Also splits the piped "one-liner" in ParseFileIntoDomains into individually commented commands (makes for easier reading and debugging)

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-02-22 21:21:38 +01:00
Christian König
ca74152d1d
Allow adding ABP style blocklists 
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-22 21:21:13 +01:00
Christian König
a4bdf2454b
Don't use intermediate strings to filter false positives in gravity 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2023-02-16 10:39:21 +01:00
Adam Warner
9939cf1d77
Rename some of the variables to hopefully make the process a little clearer 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-01-21 23:57:49 +00:00
Adam Warner
79f4a7cef0
Maintain a list of valid hostnames (taken from the top of StevenBlack's host file), and discount them when calculating the number of "invalid" domains in a given list 
Soften the output message when reporting on unsuable domains

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2023-01-20 18:52:14 +00:00
Christian König
14ab586603
Start counting with 1 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-12-11 15:12:28 +01:00
Christian König
372070ab39
Improve detection of incorrecr lines during gravity 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-12-11 15:12:28 +01:00
Christian König
4004a93d1a
Don't double check for lines containing no domains in gravity 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-12-11 15:12:28 +01:00
RD WebDesign
090727b1d3
Remove unnecessary return 
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2022-10-17 15:24:49 -03:00
RD WebDesign
8b98fd06ed
Remove unused old code 
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
 2022-10-16 15:54:24 -03:00
Christian König
230d6a435f
Semi hardcode the connection_timeout value 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-08-31 21:41:57 +02:00
Christian König
53c0982c8b
Set connection timeout for curl 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-08-31 14:43:20 +02:00
Adam Warner
64dc002c3f
Same per @rdwebdesign suggestion. 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2022-08-01 20:38:31 +01:00
Adam Warner
6c7a28a7b5
Dont use bash keywords/programs as variable names 2022-08-01 18:38:15 +01:00
Christian König
b486786041
Add database healthy checks to debug script 
Fix health check output comparison in gravity script as well
Add note about waiting time

Signed-off-by: Christian König <ckoenig@posteo.de>
 2022-07-10 23:32:47 +02:00
Adam Warner
0be987239e
Adds spellcheck action to the smoke-tests (also moves editorconfig-checker to smoketests away from it's own action) 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2022-07-02 14:04:08 +01:00
Peter Russell
052a1f294e
Update gravity.sh 
Co-authored-by: yubiuser <ckoenig@posteo.de>
 2022-04-20 10:44:05 +02:00
Peter Russell
325f20d5c2
Update gravity.sh 
Co-authored-by: yubiuser <ckoenig@posteo.de>
 2022-04-20 10:43:56 +02:00
Peter Russell
1c28da7bbd
Update gravity.sh 
Co-authored-by: yubiuser <ckoenig@posteo.de>
 2022-04-20 10:43:39 +02:00
jpgpi250
aba5e884eb Update gravity.sh 
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
 2022-04-20 10:19:44 +02:00
DL6ER
8cbffa179d
Replace remaining sqlite3 calls by calls to our embedded pihole-FTL sqlite3 engine and remove sqlite3 as dependency in the installer. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-30 11:18:17 +01:00
DL6ER
534f9a63bf
Copy database tables earlier into the new gravity database to avoid foreign key contraint violations when adding gravity entries refering to an empty adlist table 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-30 10:36:20 +01:00
DL6ER
f0f5cc52d9
Use internal SQLite3 engine in more places in gravity.sh 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-29 22:39:45 +01:00
DL6ER
ce86157067
Fix gravity in case there are no adlists at all or all are disabled (#4535) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-08 14:15:26 +01:00
DL6ER
3d3bb45a46
Merge pull request #4288 from pi-hole/new/gravity_repair 
Implement fully-automated gravity database recovery method
 2021-12-22 21:08:01 +01:00
DL6ER
d2a98ae954
Document -r recover force case 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-22 19:53:52 +01:00
DL6ER
2e1ce7fc87
Apply suggestions from code review 
Co-authored-by: yubiuser <ckoenig@posteo.de>
 2021-12-22 19:52:08 +01:00
DL6ER
54c58327f1
Merge pull request #4450 from pi-hole/unblock_NODATA 
Unblock adlist domain during gravity run in NODATA mode
 2021-12-21 22:08:14 +01:00
yubiuser
db5e94b14a
use +short and omit obsolet awk 
Co-authored-by: DL6ER <DL6ER@users.noreply.github.com>
 2021-12-21 22:01:34 +01:00
DL6ER
7167e6d5e4
Apply suggestions from code review 
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-21 16:20:02 +01:00
DL6ER
b06efb6ab7
Declare variables local 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 14:00:46 +01:00
DL6ER
ab4bce4787
Allow users to force recovery even when checks are okay using "pihole -g -r recover force" 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
DL6ER
469c179b32
Return early from recovery routine when integrity checks didn't show any database errors. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
DL6ER
190ab79606
Implement fully-automated gravity database recovery method. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
Dan Schaper
533a77d6d5
Add database function failure guards. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 11:36:55 -08:00
Dan Schaper
76ae75689c
Check for DNS before run. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 11:09:11 -08:00
Dan Schaper
a780fc59e2
Set DBFile permissions on creation. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 10:56:42 -08:00
Christian König
e243c562c2
Unblock adlist domain during gravity run in NODATA mode 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-03 09:17:19 +01:00
Christian König
647ba6ec9d
Rename variables to improve comprehensibility 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-02 23:13:01 +01:00
Christian König
bfda52ed79
Fix number of invalid domains 
Co-authored-by: abesnier <besnier_antoine@yahoo.fr>
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-01 11:07:17 +01:00
jbzdarkid
bc8150adfa Clean up bash script formatting 
Done with the help of beautysh (a python-based bash formatter)

Signed-off-by: jbzdarkid <jbzdarkid@gmail.com>
 2021-11-25 14:12:09 -08:00
yubiuser
38bb4a4908
Remove unused wildcard_regex_converter.sh (#4369) 
* Remove unused wildcard_regex_converter.sh

Signed-off-by: Christian König <ckoenig@posteo.de>

* Remove regexconverter

Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-10-05 13:09:16 +01:00
jpgpi250
3439045228 fix Pi-hole v5.4 update - syntax error #4322 
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
 2021-09-12 18:24:15 +02:00
Adam Warner
4ac4bc5c18
Merge branch 'development' into fix/gravity_no_locallist 2021-07-04 00:48:09 +01:00
jpgpi250
a2d5b8050a Update gravity.sh 
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
 2021-06-14 20:27:10 +02:00
DL6ER
f90dffbf1a
Do not put hostname and pi.hole into local.list - this will be handled by FTL after FTL#1111 has been merged 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-04-16 11:41:26 +02:00
DL6ER
f536718aaa
Use possibly custom gravity.db location in the copying SQL script 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-18 16:46:59 +01:00
DL6ER
d25240fe9f
pihole -g should respose GRAVITYDB in pihole-FTL.conf 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-18 09:00:27 +01:00
DL6ER
126b9ae381
Add missing three new columns to pihole -g -r 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-02 20:08:09 +01:00
DL6ER
57fe3b6621
Merge pull request #3951 from pi-hole/new/gravity_adlist_infos 
Store gravity details in gravity.db adlist table
 2021-02-28 17:01:19 +01:00
Adam Warner
9ec012289f
Merge pull request #4017 from mwoolweaver/patch-2 
fix suspected typo in comment
 2021-01-26 08:48:05 +00:00
Michael Woolweaver
c42b97ddb2
fix typo in comment 
Signed-off-by: Michael Woolweaver <m.woolweaver@icloud.com>
 2021-01-26 01:04:37 -06:00
freddii
7c0c30fb0b fixed typos 2021-01-19 19:33:38 +01:00
DL6ER
1791fe22f6
Merge pull request #3912 from mdujava/basic_auth 
Fix validation of adlist url
 2020-12-31 12:19:32 +01:00
DL6ER
a2625df5e2
Remove "The list is new" comment because it is superfluous 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-30 11:27:34 +01:00
DL6ER
0944807491
Actually store correct number of domains for the individual lists (and not the sum of the so far collected number of domains) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 20:36:49 +01:00
DL6ER
a216848c1d
Only update time of last list change when we see a list for the first time or when it really changed content 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 20:28:09 +01:00
DL6ER
8c56f54a1e
Compare checksum of downloaded list against older checksums to see if the list content changed since the last download 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 09:54:25 +01:00