Commit graph

493 commits

Author SHA1 Message Date
Christian König
9c4e74ffa7
Remove special handling of pgl.yoyo.org
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 12:23:11 +02:00
RD WebDesign
66bfa606a7
Using a better text for the comment
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-29 14:17:41 -03:00
RD WebDesign
32fb2e69ff
Spliting the regex into TLD_pattern and subdomain_pattern
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-29 00:05:01 -03:00
RD WebDesign
c71460e4b6
Allow TLD blocking using ABP style
This validates patterns without dots (only for abp style), allowing TLDs to be blocked

Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-28 17:21:50 -03:00
Adam Warner
0656ceb149
Speed things up a bit with some humble greps. Consolodate regexes and remove the need for so many mapfile/arrays
Signed-off-by: Adam Warner <me@adamwarner.co.uk>

Use temp files for parsing and remove when done.

Always rm the non-domains temp file.

exit 1 if gravity database creation fails.

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: DL6ER <dl6er@dl6er.de>
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
2023-03-25 11:19:23 -07:00
kot0dama
76b7453f90
Add configurable GRAVITY_TMPDIR variable into setupVars
Signed-off-by: Loïc Gomez <89980752+kot0dama@users.noreply.github.com>
2023-03-22 19:40:24 +09:00
Christian König
71e262c37f
Revert "Allow final dot (root zone)"
This reverts commit 0b5da9f0da.

Signed-off-by: Christian König <ckoenig@posteo.de>
2023-03-14 19:50:53 +01:00
Christian König
0b5da9f0da
Allow final dot (root zone)
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-03-04 10:36:07 +01:00
Christian König
16385af3ef
Use dedicated pattern variable to make RegEx reusable
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-26 21:16:45 +01:00
Christian König
6cb0be82ca
Add flag abp_domains into info table to signal if abp domains have been found
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-26 10:34:17 +01:00
Christian König
821c7dc190
Add info when list cotains ABP style domains
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-24 22:18:41 +01:00
Christian König
eedd93d782
Remove temporary files
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-24 21:55:22 +01:00
Adam Warner
1f3f849106
Remove lines starting with ! or [ to account for ABP style comments and header
Also splits the piped "one-liner" in ParseFileIntoDomains into individually commented commands (makes for easier reading and debugging)

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-02-22 21:21:38 +01:00
Christian König
ca74152d1d
Allow adding ABP style blocklists
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-22 21:21:13 +01:00
Christian König
a4bdf2454b
Don't use intermediate strings to filter false positives in gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-16 10:39:21 +01:00
Adam Warner
9939cf1d77
Rename some of the variables to hopefully make the process a little clearer
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-01-21 23:57:49 +00:00
Adam Warner
79f4a7cef0
Maintain a list of valid hostnames (taken from the top of StevenBlack's host file), and discount them when calculating the number of "invalid" domains in a given list
Soften the output message when reporting on unsuable domains

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-01-20 18:52:14 +00:00
Christian König
14ab586603
Start counting with 1
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
Christian König
372070ab39
Improve detection of incorrecr lines during gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
Christian König
4004a93d1a
Don't double check for lines containing no domains in gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
RD WebDesign
090727b1d3
Remove unnecessary return
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2022-10-17 15:24:49 -03:00
RD WebDesign
8b98fd06ed
Remove unused old code
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2022-10-16 15:54:24 -03:00
Christian König
230d6a435f
Semi hardcode the connection_timeout value
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-08-31 21:41:57 +02:00
Christian König
53c0982c8b
Set connection timeout for curl
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-08-31 14:43:20 +02:00
Adam Warner
64dc002c3f
Same per @rdwebdesign suggestion.
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2022-08-01 20:38:31 +01:00
Adam Warner
6c7a28a7b5
Dont use bash keywords/programs as variable names 2022-08-01 18:38:15 +01:00
Christian König
b486786041
Add database healthy checks to debug script
Fix health check output comparison in gravity script as well
Add note about waiting time

Signed-off-by: Christian König <ckoenig@posteo.de>
2022-07-10 23:32:47 +02:00
Adam Warner
0be987239e
Adds spellcheck action to the smoke-tests (also moves editorconfig-checker to smoketests away from it's own action)
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2022-07-02 14:04:08 +01:00
Peter Russell
052a1f294e
Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:44:05 +02:00
Peter Russell
325f20d5c2
Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:43:56 +02:00
Peter Russell
1c28da7bbd
Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:43:39 +02:00
jpgpi250
aba5e884eb Update gravity.sh
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
2022-04-20 10:19:44 +02:00
DL6ER
8cbffa179d
Replace remaining sqlite3 calls by calls to our embedded pihole-FTL sqlite3 engine and remove sqlite3 as dependency in the installer.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-01-30 11:18:17 +01:00
DL6ER
534f9a63bf
Copy database tables earlier into the new gravity database to avoid foreign key contraint violations when adding gravity entries refering to an empty adlist table
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-01-30 10:36:20 +01:00
DL6ER
f0f5cc52d9
Use internal SQLite3 engine in more places in gravity.sh
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-01-29 22:39:45 +01:00
DL6ER
ce86157067
Fix gravity in case there are no adlists at all or all are disabled (#4535)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-01-08 14:15:26 +01:00
DL6ER
3d3bb45a46
Merge pull request #4288 from pi-hole/new/gravity_repair
Implement fully-automated gravity database recovery method
2021-12-22 21:08:01 +01:00
DL6ER
d2a98ae954
Document -r recover force case
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-12-22 19:53:52 +01:00
DL6ER
2e1ce7fc87
Apply suggestions from code review
Co-authored-by: yubiuser <ckoenig@posteo.de>
2021-12-22 19:52:08 +01:00
DL6ER
54c58327f1
Merge pull request #4450 from pi-hole/unblock_NODATA
Unblock adlist domain during gravity run in NODATA mode
2021-12-21 22:08:14 +01:00
yubiuser
db5e94b14a
use +short and omit obsolet awk
Co-authored-by: DL6ER <DL6ER@users.noreply.github.com>
2021-12-21 22:01:34 +01:00
DL6ER
7167e6d5e4
Apply suggestions from code review
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-12-21 16:20:02 +01:00
DL6ER
b06efb6ab7
Declare variables local
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-12-21 14:00:46 +01:00
DL6ER
ab4bce4787
Allow users to force recovery even when checks are okay using "pihole -g -r recover force"
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-12-21 13:57:03 +01:00
DL6ER
469c179b32
Return early from recovery routine when integrity checks didn't show any database errors.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-12-21 13:57:03 +01:00
DL6ER
190ab79606
Implement fully-automated gravity database recovery method.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-12-21 13:57:03 +01:00
Dan Schaper
533a77d6d5
Add database function failure guards.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-12-20 11:36:55 -08:00
Dan Schaper
76ae75689c
Check for DNS before run.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-12-20 11:09:11 -08:00
Dan Schaper
a780fc59e2
Set DBFile permissions on creation.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-12-20 10:56:42 -08:00
Christian König
e243c562c2
Unblock adlist domain during gravity run in NODATA mode
Signed-off-by: Christian König <ckoenig@posteo.de>
2021-12-03 09:17:19 +01:00