Commit graph

22 commits

Author SHA1 Message Date
redmatrix
650f882265 generalise the site black|white allow lists, also add a channel black|white list for future use 2015-11-29 20:26:00 -08:00
redmatrix
302c5afdee don't try to pull public content from dead sites 2015-11-29 16:28:35 -08:00
redmatrix
dbeae16dc1 add site_type to queries, first few - several more to go. 2015-09-28 04:21:29 -07:00
friendica
29436081a8 slow progress removing bitfields on item table 2015-01-21 16:06:25 -08:00
friendica
1c249a5b06 some poller optimisations and a fix for undefined (empty) dbtype which shows up as a mysql error that 'rand' isn't found. 2014-11-16 15:51:42 -08:00
Habeas Codice
1a5a5c7edb PostgreSQL support initial commit
There were 11 main types of changes:
- UPDATE's and DELETE's sometimes had LIMIT 1 at the end of them. This is not only non-compliant but
it would certainly not do what whoever wrote it thought it would. It is likely this mistake was just
copied from Friendica. All of these instances, the LIMIT 1 was simply removed.
- Bitwise operations (and even some non-zero int checks) erroneously rely on MySQL implicit
integer-boolean conversion in the WHERE clauses. This is non-compliant (and bad programming practice
to boot). Proper explicit boolean conversions were added. New queries should use proper conventions.
- MySQL has a different operator for bitwise XOR than postgres. Rather than add yet another dba_
func, I converted them to "& ~" ("AND NOT") when turning off, and "|" ("OR") when turning on. There
were no true toggles (XOR). New queries should refrain from using XOR when not necessary.
- There are several fields which the schema has marked as NOT NULL, but the inserts don't specify
them. The reason this works is because mysql totally ignores the constraint and adds an empty text
default automatically. Again, non-compliant, obviously. In these cases a default of empty text was
added.
- Several statements rely on a non-standard MySQL feature
(http://dev.mysql.com/doc/refman/5.5/en/group-by-handling.html). These queries can all be rewritten
to be standards compliant. Interestingly enough, the newly rewritten standards compliant queries run
a zillion times faster, even on MySQL.
- A couple of function/operator name translations were needed (RAND/RANDOM, GROUP_CONCAT/STRING_AGG,
UTC_NOW, REGEXP/~, ^/#) -- assist functions added in the dba_
- INTERVALs: postgres requires quotes around the value, mysql requires that there are not quotes
around the value -- assist functions added in the dba_
- NULL_DATE's -- Postgres does not allow the invalid date '0000-00-00 00:00:00' (there is no such
thing as year 0 or month 0 or day 0). We use '0001-01-01 00:00:00' for postgres. Conversions are
handled in Zot/item packets automagically by quoting all dates with dbescdate().
- char(##) specifications in the schema creates fields with blank spaces that aren't trimmed in the
code. MySQL apparently treats char(##) as varchar(##), again, non-compliant. Since postgres works
better with text fields anyway, this ball of bugs was simply side-stepped by using 'text' datatype
for all text fields in the postgres schema. varchar was used in a couple of places where it actually
seemed appropriate (size constraint), but without rigorously vetting that all of the PHP code
actually validates data, new bugs might come out from under the rug.
- postgres doesn't store nul bytes and a few other non-printables in text fields, even when quoted.
bytea fields were used when storing binary data (photo.data, attach.data). A new dbescbin() function
was added to handle this transparently.
- postgres does not support LIMIT #,# syntax. All databases support LIMIT # OFFSET # syntax.
Statements were updated to be standard.

These changes require corresponding changes in the coding standards. Please review those before
adding any code going forward.

Still on my TODO list:
- remove quotes from non-reserved identifiers and make reserved identifiers use dba func for quoting
- Rewrite search queries for better results (both MySQL and Postgres)
2014-11-13 12:21:58 -08:00
friendica
3d0d36b2bc fix diaspora reshare tags 2014-10-10 19:14:37 -07:00
friendica
c6d07feff5 This is long overdue - use a symblic constant NULL_DATE instead of the easily mis-typed sequence '0000-00-00 00:00:00' 2014-09-08 20:35:15 -07:00
friendica
148f4e8795 issue #578 - this won't fix old items but should work going forward. The extra time slop shouldn't be needed and will in fact result in possibly undesired redundancy. 2014-09-01 19:55:43 -07:00
friendica
69d3e5468d provide the site blacklist everywhere it is needed 2014-08-21 21:07:32 -07:00
friendica
1476812e90 add another important note 2014-07-22 17:21:13 -07:00
friendica
b29f087cdc provide ability to blacklist sites from "Discover". 2014-07-22 17:17:33 -07:00
friendica
e10509e2bd soft revert 9bb7811 until we can look at it more closely. 2014-04-11 14:01:33 -07:00
friendica
9bb7811d74 Discover page: when importing public items, alter the item so that it has the characteristics of one that was sourced by the sys channel instead of just being stored under the sys channel uid. This should allow comments and likes to flow upstream if permissions allow and may fix issue #398. Permission may still be denied by the original poster, but without this the comment/like is treated as a forgery and is blocked from transmission. 2014-04-10 18:06:09 -07:00
Thomas Willingham
a2b949f339 Typo in externals SQL. 2014-04-05 22:55:38 +01:00
friendica
27fec9cb64 more efficient public feed fetching 2014-04-02 15:13:50 -07:00
friendica
fc5f3f4800 externals/discover: try up to 3 times to find a site that actually has content. Then give up until the next queue run. 2014-03-28 15:58:58 -07:00
friendica
52c456d254 don't include standalone test sites in external host list 2014-03-27 20:57:45 -07:00
friendica
8c3efd20f4 this should work a bit better 2014-03-27 18:53:14 -07:00
Thomas Willingham
6394f06635 typo 2014-03-28 00:36:38 +00:00
friendica
1f931c0e3c firehose: provide system prefs to allow folks to fine tune to their needs 2014-03-27 16:40:10 -07:00
friendica
a0674af416 firehose fixes and optimisations. In particular get rid of the unresponsive script warning when trying to load updates (sine they aren't our posts, we can't check for unseen, hence we can't really load updates). Also make the url selection pluggable. 2014-03-27 15:35:29 -07:00