Commit graph

75 commits

Author SHA1 Message Date
redmatrix
b33a9a71f6 more work on import & sync of private mail and conversations 2015-10-14 17:40:50 -07:00
redmatrix
e45958b967 add plugin hooks to channel sync/import/export 2015-09-23 18:27:04 -07:00
redmatrix
4abfd4053a sync likes 2015-09-10 19:18:12 -07:00
redmatrix
75d521d42a menu import and sync 2015-09-08 17:52:39 -07:00
redmatrix
650e98b2a9 work on event sync 2015-09-07 21:01:49 -07:00
redmatrix
6d1b64065a consolidate import_items/sync_items 2015-09-07 18:14:30 -07:00
redmatrix
d79e81a069 import and sync chatrooms 2015-09-03 18:44:40 -07:00
redmatrix
c83da012ff profile import error #34 2015-09-03 16:19:59 -07:00
redmatrix
c22da3da1c start to modularise the clone import and sync functions 2015-09-02 20:12:40 -07:00
redmatrix
d105f324af import apps 2015-09-01 22:27:09 -07:00
redmatrix
6cc60ba477 initial work on obj import, rename import_profile_photo to import_xchan_photo to more accurately state its purpose. 2015-09-01 22:03:10 -07:00
redmatrix
417da13353 this is a bit harder than anticipated. 2015-08-31 21:12:37 -07:00
redmatrix
039f56f390 mostly doc tweaks, some import/export work 2015-08-31 20:58:05 -07:00
redmatrix
d2fec56456 Merge https://github.com/redmatrix/redmatrix into pending_merge
Conflicts:
	mod/import.php
2015-08-26 16:00:13 -07:00
redmatrix
bd2d99aced import fixes 2015-08-26 15:56:32 -07:00
redmatrix
71308fe8c1 syntax error in last checkin 2015-08-26 13:50:51 -07:00
redmatrix
da649e397e import fixes 2015-08-26 13:49:07 -07:00
redmatrix
b634c08d0e resolve conflict 2015-08-11 22:41:27 -07:00
redmatrix
2d9d4dfeb3 Merge https://github.com/redmatrix/redmatrix into pending_merge
Conflicts:
	doc/history.md
	mod/import.php
2015-08-11 22:35:36 -07:00
redmatrix
81c740178d convert abook_flags from hubzilla during cloning 2015-08-10 16:16:26 -07:00
redmatrix
0d1a541e88 Merge branch 'master' of https://github.com/redmatrix/redmatrix
Conflicts:
	mod/import.php
2015-07-14 22:40:22 -07:00
redmatrix
9b15dbde1c group fixes to import 2015-07-14 22:38:04 -07:00
redmatrix
5487b22dd7 fix import of group members 2015-07-14 22:19:21 -07:00
redmatrix
89fbfd2bf8 more stuff to resume failed imports 2015-07-14 22:04:07 -07:00
redmatrix
a87e318403 log the import progress 2015-07-14 21:30:13 -07:00
redmatrix
caaf7192ff provide resumable imports if things go wrong 2015-07-14 21:02:54 -07:00
redmatrix
e8f19c1e15 Merge branch 'master' of https://github.com/redmatrix/redmatrix
Conflicts:
	mod/connedit.php
2015-07-14 17:10:13 -07:00
redmatrix
d5c085a3c6 cleanup of import to allow cross-product import going one way. Note - items/content are not yet importable 2015-06-30 20:21:09 -07:00
redmatrix
1e3dffc29d Merge branch 'master' of https://github.com/redmatrix/redmatrix 2015-06-30 20:19:50 -07:00
redmatrix
86b12ea750 lots of work on clone/import from redmatrix to hubzilla (won't go back the other way) 2015-06-30 20:18:48 -07:00
redmatrix
22cb09f537 warn if export data file is from an incompatible project or if the DB version differs significantly from the current DB. This is to avoid people cloning a hubzilla channel back to red and potentially stuffing up the works. Hopefully we can migrate upwards someday. Downwards is always going to be an issue. 2015-06-29 22:37:18 -07:00
redmatrix
ae6612025d Merge branch 'master' of https://github.com/redmatrix/redmatrix
Conflicts:
	include/items.php
	util/messages.po
2015-06-21 20:51:07 -07:00
redmatrix
0ad71b3efa import: Improve the descriptive text for this feature 2015-06-21 19:11:07 -07:00
redmatrix
68534fd5dc some diaspora import issues uncovered during dry-run tests 2015-06-21 17:30:23 -07:00
redmatrix
a6b9d1595e automatically gunzip Diaspora export files if we're on a *nix system 2015-06-19 17:18:01 -07:00
redmatrix
6bd90816dd very early attempt diaspora import from diaspora export file. Currently the json export has to be gunzipped prior to uploading and we're net yet adding connections (which should come soon) and we can't import items until they add guids to the export file - so consider it a work in progress. This first attempt should create a channel and get your profile details setup providing there isn't a nickname conflict on the server. That will also be handled later. 2015-06-19 16:39:03 -07:00
redmatrix
be0459a98b convert the abook fields 2015-06-14 21:08:00 -07:00
friendica
6679734135 Merge branch 'master' into tres
Conflicts:
	include/Contact.php
	include/ItemObject.php
	include/api.php
	include/attach.php
	include/diaspora.php
	include/dir_fns.php
	include/enotify.php
	include/event.php
	include/expire.php
	include/items.php
	include/notifier.php
	include/notify.php
	include/photos.php
	include/taxonomy.php
	include/text.php
	include/widgets.php
	include/zot.php
	mod/admin.php
	mod/channel.php
	mod/dirsearch.php
	mod/display.php
	mod/editwebpage.php
	mod/events.php
	mod/home.php
	mod/item.php
	mod/manage.php
	mod/mood.php
	mod/network.php
	mod/page.php
	mod/photos.php
	mod/ping.php
	mod/post.php
	mod/thing.php
	mod/viewsrc.php
	view/css/mod_events.css
2015-04-23 19:49:41 -07:00
friendica
0e74ebc6a9 title missing sending to diaspora (which doesn't support titles but let's not go there). 2015-02-09 15:12:15 -08:00
friendica
ee3eee425c that should take care of the bitfields in hubloc 2015-01-20 16:13:18 -08:00
friendica
ab2d11b37e mod/import: try ten times to create a unique webbie if the chosen one is in use. 2014-12-18 19:48:23 -08:00
Habeas Codice
1a5a5c7edb PostgreSQL support initial commit
There were 11 main types of changes:
- UPDATE's and DELETE's sometimes had LIMIT 1 at the end of them. This is not only non-compliant but
it would certainly not do what whoever wrote it thought it would. It is likely this mistake was just
copied from Friendica. All of these instances, the LIMIT 1 was simply removed.
- Bitwise operations (and even some non-zero int checks) erroneously rely on MySQL implicit
integer-boolean conversion in the WHERE clauses. This is non-compliant (and bad programming practice
to boot). Proper explicit boolean conversions were added. New queries should use proper conventions.
- MySQL has a different operator for bitwise XOR than postgres. Rather than add yet another dba_
func, I converted them to "& ~" ("AND NOT") when turning off, and "|" ("OR") when turning on. There
were no true toggles (XOR). New queries should refrain from using XOR when not necessary.
- There are several fields which the schema has marked as NOT NULL, but the inserts don't specify
them. The reason this works is because mysql totally ignores the constraint and adds an empty text
default automatically. Again, non-compliant, obviously. In these cases a default of empty text was
added.
- Several statements rely on a non-standard MySQL feature
(http://dev.mysql.com/doc/refman/5.5/en/group-by-handling.html). These queries can all be rewritten
to be standards compliant. Interestingly enough, the newly rewritten standards compliant queries run
a zillion times faster, even on MySQL.
- A couple of function/operator name translations were needed (RAND/RANDOM, GROUP_CONCAT/STRING_AGG,
UTC_NOW, REGEXP/~, ^/#) -- assist functions added in the dba_
- INTERVALs: postgres requires quotes around the value, mysql requires that there are not quotes
around the value -- assist functions added in the dba_
- NULL_DATE's -- Postgres does not allow the invalid date '0000-00-00 00:00:00' (there is no such
thing as year 0 or month 0 or day 0). We use '0001-01-01 00:00:00' for postgres. Conversions are
handled in Zot/item packets automagically by quoting all dates with dbescdate().
- char(##) specifications in the schema creates fields with blank spaces that aren't trimmed in the
code. MySQL apparently treats char(##) as varchar(##), again, non-compliant. Since postgres works
better with text fields anyway, this ball of bugs was simply side-stepped by using 'text' datatype
for all text fields in the postgres schema. varchar was used in a couple of places where it actually
seemed appropriate (size constraint), but without rigorously vetting that all of the PHP code
actually validates data, new bugs might come out from under the rug.
- postgres doesn't store nul bytes and a few other non-printables in text fields, even when quoted.
bytea fields were used when storing binary data (photo.data, attach.data). A new dbescbin() function
was added to handle this transparently.
- postgres does not support LIMIT #,# syntax. All databases support LIMIT # OFFSET # syntax.
Statements were updated to be standard.

These changes require corresponding changes in the coding standards. Please review those before
adding any code going forward.

Still on my TODO list:
- remove quotes from non-reserved identifiers and make reserved identifiers use dba func for quoting
- Rewrite search queries for better results (both MySQL and Postgres)
2014-11-13 12:21:58 -08:00
friendica
79684e05cc fix import sql error 2014-10-13 21:24:06 -07:00
friendica
bca4d540e0 disable email notifications when bulk importing content. 2014-10-08 15:15:20 -07:00
friendica
d8c03526ed optionally include wall items (posts) in import/export 2014-09-25 20:21:23 -07:00
friendica
6a82ccecd0 when importing channels - use the new location notification message to tell your original site where it sits in terms of primary. 2014-09-16 05:14:06 -07:00
friendica
5f9c326ad7 channel export with items 2014-09-15 21:31:32 -07:00
friendica
bbc9e4427e honour service class restrictions for total_identities, total_channels ("friends") and total_feeds both when importing channels and subsequently when syncing clones. Limits are based on the local system - additional entries are silently dropped. 2014-09-15 17:17:00 -07:00
friendica
c6d07feff5 This is long overdue - use a symblic constant NULL_DATE instead of the easily mis-typed sequence '0000-00-00 00:00:00' 2014-09-08 20:35:15 -07:00
friendica
9196c9eef0 We really can't do this without a hubloc. I was hoping we could, but notifier is setup to take hublocs, not xchans. 2014-08-22 21:37:08 -07:00