Commit graph

3807 commits

Author SHA1 Message Date
DL6ER
3aa838bbe4
Implement black- and whitelist searching with SQL statements. We use the ESCAPE clause in the LIKE query as the underscore "_" wildcard matches any single character but we want to suppress this behavior (underscores can be legitimate part of domains)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-04 13:02:14 +02:00
DL6ER
0bc112ce52
Query only those columns we actually use when showing domains on the CLI. Show date of last modification instead of addition date for domains.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-02 17:11:39 +02:00
DL6ER
d46ff1d2d7
Remove UNIQUE on gravity table as the only column is already PRIMARY KEY. Primary keys must contain UNIQUE values, and cannot contain NULL values
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 22:08:30 +02:00
DL6ER
0cfecd31d3
Invert logic to reduce nesting
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 22:07:14 +02:00
DL6ER
008e88b84b
Only migrate legacy list files when we create gravity.db in the same gravity run
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 18:04:31 +02:00
DL6ER
2f698904e4
Use PRIMARY KEY instead of NOT NULL also in gravity and info tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 18:03:35 +02:00
DL6ER
106f079afc
Improve indentation (single tab) in SQLite template
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:57:23 +02:00
DL6ER
6c1e3a17f7
Remove NOT NULL constraint on list ids as this is implied by PRIMARY KEY (even though most, if not all, examples on the web still use both attributes at the same time)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:54:29 +02:00
DL6ER
540c57f755
Add comment for automigrated list entries
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:51:25 +02:00
DL6ER
5e26e8245b
Order results of queries to list views by id key
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:51:10 +02:00
DL6ER
983117d788
Add id (primary key, autoincrement) field to lists tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:47:15 +02:00
DL6ER
b372f808dd
Fix lint errors
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:06:14 +02:00
DL6ER
bc9b623638
Modify query.sh to use the gravity database
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 16:56:16 +02:00
DL6ER
d5781fb110
Fix extra space in front on END statement in CREATE TRIGGER commands
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 16:38:41 +02:00
DL6ER
487718512d
Remove DISTINCT attribute from views. They are not needed as the corresponding data is obtained from fields with UNIQUE attribute so duplications are already prevented at domain insertion. The elimination of the duplication check on imort through the views slightly reduces the load during import (I observed about 8 percent time saved on my NanoPi NEO).
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 16:37:23 +02:00
DL6ER
2180531a84
Add trigger for automatic updates of the date_modified column and improve formating in the gravity.db template
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-29 19:43:35 +02:00
DL6ER
a932209143
Set default for date_modified to be the current time
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-29 17:21:59 +02:00
DL6ER
4946b5907b
Move migrated files to backup directory instead of deleting them. This has the benefit that gravity.db can be recreated at any time by: 1. deleting gravity.db, 2. copying all files from the migration_backup directory back to /etc/pihole, 3. running pihole -g again.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-28 22:15:47 +02:00
DL6ER
efeba594ae
Add support for manipulating adlists from the CLI
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-28 21:39:06 +02:00
DL6ER
5e1241acd2
Merge branch 'new/gravitydb' of github.com:pi-hole/pi-hole into new/gravitydb 2019-04-28 14:44:34 +02:00
DL6ER
126f7101ff
Add (optional) date_modified field and use vw_whitelist view in vw_blacklist
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-28 14:44:03 +02:00
Mark Drobnak
4596d9a577
Merge branch 'development' into new/gravitydb 2019-04-26 23:40:28 -04:00
DL6ER
b4ae142149
Remove printf escaping (we will realize it differently) and ensure we're using single quotes for strings (although double quotes are possible, too)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 20:54:01 +02:00
DL6ER
3fe43ce1d9
Rely on default value (database provided) for date_added when adding a new domain
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 18:16:04 +02:00
DL6ER
8524aecfed
Don't set gravity database permissions
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:56:11 +02:00
DL6ER
e04dc9d2bb
Remove sqlitekey variable as the first column of the regex table is now called domain as well
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:44:42 +02:00
DL6ER
3dcb9722d3
Remove global haveSourceUrls variable and just skip downloading and processing adlists if there aren't any to be downloaded
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:39:35 +02:00
DL6ER
bd0215524b
Add vw_adlists view which returns only enabled adlists
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:36:00 +02:00
DL6ER
53e3ff2b24
Only add non-empty lines when moving file content into database tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:32:05 +02:00
DL6ER
5bb8163674
Change type of whitelist, blacklist, regex, and adlists time column to INTEGER and insert a DEFAULT clause that returns the current Unix time as such an integer
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:26:15 +02:00
DL6ER
a891d64395
We should always exit after displaying a list, even if it is empty
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:13:36 +02:00
DL6ER
6b1d2523e8
Also print datetime a domain was added when listing the domains on the CLI
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:07:16 +02:00
DL6ER
08ee9526ad
Review comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:06:01 +02:00
DL6ER
0efc46260f
Use both old and new strings for detecting whether this is a Pi-hole dnsmasq config file
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 16:16:30 +02:00
DL6ER
6a01a1257f
Merge pull request #2597 from xCh12i5/RegEx-Fix
Solves RegEx issue.
2019-04-25 15:27:35 +02:00
DL6ER
a2a7e80007
Add --web for sending a confirmation string when executed from the web interface. This is necessary as the "Reloading DNS service" message does not always propagate through to the web interface when reloading pihole-FTL takes a while.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 15:01:13 +02:00
DL6ER
cce66a13b6
Fix output in list.sh when adding a domain that is already present in the target list
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 14:54:41 +02:00
DL6ER
43bced7997
Add more comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 12:45:08 +02:00
DL6ER
788cd78321
Reduce code duplication and add special handling of | character as it might appear in regex filter string
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 12:30:38 +02:00
DL6ER
e5d1cb5a2e
Rewrite list.sh to operate on database-based lists
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 12:10:42 +02:00
DL6ER
d94bbfca7f
Finish migrating gravity.db to using our new gravity database. This removes the --skip-download, --blacklist-only, --whitelist-only, and --wildcard-only options as it is not necessary to run gravity when modifying them from now on (gravity really only builds the gravity database and the local.list file).
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 11:23:06 +02:00
DL6ER
465a39427b
Source adlists from gravity database
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:46:18 +02:00
DL6ER
2664ac0efc
Improve error reporting
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:18:37 +02:00
DL6ER
c1277705d8
Copy existing whitelist.txt, blacklist.txt, regex.list, and adlists.list to the database. We remove the files afterwards as the content lives in the database now
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:14:45 +02:00
DL6ER
61cfd2f9f9
Add adlists table
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:48:06 +02:00
DL6ER
ee4dabeca6
Merge branch 'development' into new/gravitydb
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:12:19 +02:00
DL6ER
e57d0fb93c
Remove the empty templates gravity.list, white.list, black.list. The old source files whitelist.txt, blacklist.txt, and regex.list still exist and need to be removed in a follow-up commit
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:05:08 +02:00
DL6ER
ee20164207
Remove detect_pihole_blocking_status() as we can modify the database at any time
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:01:31 +02:00
DL6ER
d7ee44960e
Remove moving around of the gravity and blacklist files as we do not use them any longer
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 18:57:38 +02:00
DL6ER
c13ebd3a9d
Replace Pi-hole ID string with another line we intend to keep
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 18:55:25 +02:00