Commit graph

40 commits

Author SHA1 Message Date
friendica
0134a41015 more work on queue optimisations 2015-03-09 19:57:35 -07:00
friendica
64e7c8b20f notes on what's needed to fix the queueing algorithms. 2015-03-04 01:26:02 -08:00
Habeas Codice
1a5a5c7edb PostgreSQL support initial commit
There were 11 main types of changes:
- UPDATE's and DELETE's sometimes had LIMIT 1 at the end of them. This is not only non-compliant but
it would certainly not do what whoever wrote it thought it would. It is likely this mistake was just
copied from Friendica. All of these instances, the LIMIT 1 was simply removed.
- Bitwise operations (and even some non-zero int checks) erroneously rely on MySQL implicit
integer-boolean conversion in the WHERE clauses. This is non-compliant (and bad programming practice
to boot). Proper explicit boolean conversions were added. New queries should use proper conventions.
- MySQL has a different operator for bitwise XOR than postgres. Rather than add yet another dba_
func, I converted them to "& ~" ("AND NOT") when turning off, and "|" ("OR") when turning on. There
were no true toggles (XOR). New queries should refrain from using XOR when not necessary.
- There are several fields which the schema has marked as NOT NULL, but the inserts don't specify
them. The reason this works is because mysql totally ignores the constraint and adds an empty text
default automatically. Again, non-compliant, obviously. In these cases a default of empty text was
added.
- Several statements rely on a non-standard MySQL feature
(http://dev.mysql.com/doc/refman/5.5/en/group-by-handling.html). These queries can all be rewritten
to be standards compliant. Interestingly enough, the newly rewritten standards compliant queries run
a zillion times faster, even on MySQL.
- A couple of function/operator name translations were needed (RAND/RANDOM, GROUP_CONCAT/STRING_AGG,
UTC_NOW, REGEXP/~, ^/#) -- assist functions added in the dba_
- INTERVALs: postgres requires quotes around the value, mysql requires that there are not quotes
around the value -- assist functions added in the dba_
- NULL_DATE's -- Postgres does not allow the invalid date '0000-00-00 00:00:00' (there is no such
thing as year 0 or month 0 or day 0). We use '0001-01-01 00:00:00' for postgres. Conversions are
handled in Zot/item packets automagically by quoting all dates with dbescdate().
- char(##) specifications in the schema creates fields with blank spaces that aren't trimmed in the
code. MySQL apparently treats char(##) as varchar(##), again, non-compliant. Since postgres works
better with text fields anyway, this ball of bugs was simply side-stepped by using 'text' datatype
for all text fields in the postgres schema. varchar was used in a couple of places where it actually
seemed appropriate (size constraint), but without rigorously vetting that all of the PHP code
actually validates data, new bugs might come out from under the rug.
- postgres doesn't store nul bytes and a few other non-printables in text fields, even when quoted.
bytea fields were used when storing binary data (photo.data, attach.data). A new dbescbin() function
was added to handle this transparently.
- postgres does not support LIMIT #,# syntax. All databases support LIMIT # OFFSET # syntax.
Statements were updated to be standard.

These changes require corresponding changes in the coding standards. Please review those before
adding any code going forward.

Still on my TODO list:
- remove quotes from non-reserved identifiers and make reserved identifiers use dba func for quoting
- Rewrite search queries for better results (both MySQL and Postgres)
2014-11-13 12:21:58 -08:00
friendica
88f8900ac5 various diaspora issues 2014-08-29 17:31:40 -07:00
friendica
c6b67518d6 more queue tweaks 2014-08-28 22:55:40 -07:00
friendica
00004e1a12 the queue driver is universal now - we don't need to filter it. 2014-08-28 22:42:19 -07:00
friendica
8a62b824f9 lots of little Diaspora issues 2014-08-28 21:49:13 -07:00
friendica
50a7e9ce8f this should improve queue performance dramatically. 2014-03-24 19:40:36 -07:00
friendica
f09f2998dc unknown column 2013-03-11 17:42:12 -07:00
friendica
ea3940c4b0 start formatting for Doxygen 2013-02-25 17:09:40 -08:00
friendica
7cd014e93c make preview work fix some other nits 2013-01-06 04:07:10 -08:00
friendica
791302f317 more delivery diagnostics 2012-12-31 14:45:26 -08:00
friendica
4e23e700f4 requeue failed deliveries 2012-12-30 21:44:59 -08:00
friendica
e9c87a69ce cleanup more obsolete stuff 2012-12-18 16:29:57 -08:00
Mike Macgirvin
7acba41dc6 more cleanup 2012-10-23 21:26:12 +11:00
friendica
d683df17bd Removing connectors we won't be needing - this is probably going to break some shit. 2012-07-18 04:49:15 -07:00
friendica
94fabe3a29 upstream fixes and a lot of taxonomy stuff 2012-07-10 06:28:02 -07:00
Alexander Kampmann
355c42cb30 Merge branch 'master' of https://github.com/friendica/friendica
Conflicts:
	include/config.php
	update.php
2012-04-05 13:39:15 +02:00
friendica
94fac6d767 queue optimisation - back off delivery attempts to once per hour after the first 12 hours. 2012-03-17 03:43:02 -07:00
friendica
8aa2552372 add remove_user hook (it looks like dreamhost changed all my file permissions, this will make a nasty commit) 2012-01-18 16:21:30 -08:00
Friendika
d6b446d63f starting the big delivery shakeup 2011-09-22 04:11:39 -07:00
Friendika
177e0e15d7 allow zero delivery interval on dedicated servers 2011-08-29 00:59:44 -07:00
Friendika
b61d535f81 remove item from queue 2011-08-28 22:59:49 -07:00
Friendika
846c4cea7c implement delivery queue in case notifier gets killed 2011-08-28 21:41:42 -07:00
Friendika
1861dc1fae don't allow fullscreen for youtube iframe - this makes it hard to visit the network page when they throw an error 2011-08-26 01:32:22 -07:00
Friendika
8f6ae2b660 missing salmon key? report it. 2011-08-24 20:40:08 -07:00
Friendika
2aa23843e4 D* queue 2011-08-24 01:21:24 -07:00
Friendika
994011ddb6 load db configs (config,system) for all "executables" 2011-06-30 01:15:18 -07:00
Friendika
7b02e178a4 more work on facebook re-queue 2011-06-05 02:18:18 -07:00
Friendika
182582eed8 load hooks inside queue 2011-06-05 02:09:31 -07:00
Friendika
a27391c33a facebook queueing on failure, sync update.php with boot.php update version to avoid race condition 2011-06-03 01:16:17 -07:00
Friendika
1eb177235b add queue_deliver plugin hook 2011-06-02 18:26:06 -07:00
Fabio Comuni
ef33ca6750 modded scripts to be runned in cli so they can be included and executed in main program 2011-01-28 14:04:18 +01:00
fabrixxm
2020ea23b1 Mods to script runned via CLI to permits to run also included 2011-01-24 22:01:56 +01:00
Friendika
e15b1a8755 don't attempt to deliver queued items to sites which have already timed out 2010-11-25 23:42:03 -08:00
Friendika
60ef474d46 log queue startup 2010-11-25 18:31:33 -08:00
Friendika
847c264787 wrong function name 2010-11-25 17:27:56 -08:00
Friendika
81e1e4245b premature removal of queue item 2010-11-23 17:08:35 -08:00
Friendika
fb58801aa4 schedule the queue re-delivery process 2010-11-22 15:30:52 -08:00
Friendika
0bcb68200e queue/redeliver module 2010-11-22 14:53:25 -08:00