Commit graph

95 commits

Author SHA1 Message Date
Matthew Exon
4ff8c9e3a1 Document new redirect_url return value 2016-01-25 11:38:36 +01:00
Matthew Exon
930da0aa44 Return redirect_url as part of the return array from z_fetch_url instead of via global state 2016-01-25 11:05:17 +01:00
Matthew Exon
3ada0093f9 Note the final URL used to retrieve content after all redirects 2016-01-25 07:41:56 +01:00
Matthew Exon
677903b316 Report errors during cURL fetch 2016-01-25 02:56:47 +01:00
Michael Vogel
5ec2c7e965 When a server isn't reachable there were timeout problems with fetching image data 2016-01-18 15:38:38 +01:00
rabuzarus
3df5c17124 reformat todo according to doxygen style 2015-12-25 23:17:34 +01:00
Michael Vogel
6a15dded0e Bugfix: The timeout value of z_fetch_url couldn't be set manually 2015-11-03 00:48:49 +01:00
Michael Vogel
2840bf2293 New option to permit crawlers 2015-10-03 23:16:40 +02:00
Michael Vogel
2038e65a57 Bugfix: The function "z_fetch_url" created warnings under certain circumstances. 2015-05-16 22:38:41 +02:00
fabrixxm
6258a21434 map bbcode tag from Red 2015-05-15 20:41:10 +02:00
Michael Vogel
dbe8275ae5 Added some more timestamp measuring 2015-03-07 23:14:26 +01:00
Michael Vogel
7c4a1a059d New fields for the term table, improved query for the tag search. Changed the cache handling for rendered bbcode. 2015-03-07 21:24:39 +01:00
Michael Vogel
9e55dbf1db There is already a setting for that ... 2015-03-02 00:32:46 +01:00
Michael Vogel
5cb6d6e537 We have to disable the SSL checks, otherwise the communication will fail for servers with self-signed certificates 2015-03-01 10:53:49 +01:00
Fabrixxm
caa21e6021 Merge branch 'master' into develop 2015-02-24 09:09:02 +01:00
Johannes Schwab
1cd1f7587c fix validate_email 2015-02-23 12:20:05 +01:00
Matthew Exon
2bb8b77123 Close cURL handle before recursing, so that cookie file is flushed 2015-02-11 03:42:07 +00:00
Matthew Exon
d12b929c96 Accept lowercase location header 2015-02-10 03:14:08 +00:00
Michael Vogel
aa6275356a The function to fetch the original url (without shortener) failed if a linked file was too large. 2015-01-04 15:31:47 +01:00
Michael Vogel
0c60eb18f7 Some more typical tracking parameter is removed 2014-12-01 22:52:51 +01:00
fabrixxm
6db73c0b6d fix update routine to support update from 3.2
disable PDO support
2014-09-07 17:28:38 +02:00
Michael Vogel
7af5c3c560 The useragent string doesn't contains of the words "Mozilla" anymore - but the name "Friendica" and the base url 2014-09-07 10:47:37 +02:00
Michael Vogel
07bc8ca166 fetch_url: Logging the data only when at the "data" loglevel 2014-08-07 08:22:32 +02:00
Michael Vogel
e974e586cc Unified user agent string when fetching data. 2014-08-07 07:58:20 +02:00
Michael Vogel
6f5bc8d9a7 Setting the user agent to "Mozilla/5.0 (compatible; Friendica)" 2014-07-10 23:13:07 +02:00
Michael Vogel
6a28ae8b52 Just some better handling of tracker variables (url encoding) 2014-05-30 01:02:32 +02:00
Michael Vogel
135a403cec Added some tracking variables from facebook in the function to clean up URLs. 2014-05-29 11:43:37 +02:00
Michael Vogel
5ae9c5e538 Added some logging 2014-05-24 11:01:33 +02:00
Michael Vogel
6b461a74cb Cleaning up tracking parameters in urls. 2014-05-18 20:32:06 +02:00
Michael Vogel
426e0045ee The function "short_url" is used in several addons, so it deserves a central place. 2014-05-03 12:07:34 +02:00
Michael Vogel
a64948b7f7 New function "original_url" (moved from an addon) that follows all redirects to the original url. 2014-04-04 10:55:30 +02:00
Michael Vogel
8c1d31decc Showing "view full size" is now configurable 2014-02-11 23:44:45 +01:00
fabrixxm
ad5a47aecf add fetaure switch for acl-automention 2014-01-03 14:40:43 +01:00
Mats Sjöberg
19de78e511 Fixed subscription problem with new GNU Social. 2013-11-21 19:41:06 +01:00
Michael Vogel
759015e5fd Removed many "limit 1" at update and delete queries since they were superflous und generated massiv logfile entries 2013-11-02 16:13:11 +01:00
Matthew Exon
11163c6fb3 Allow calls to fetch_url to specify a cookiejar file, for sites that refuse to deliver content without setting a cookie 2013-08-05 10:59:22 +08:00
Matthew Exon
6a38705567 Replicate redirect URL building logic for post requests 2013-07-29 22:34:09 +08:00
Matthew Exon
eb9bd03ef8 If the header returns an absolute path as a redirect, don't simply append the path to the old URL, remove the old absolute path first.
For example, original URL:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html

Redirect from header:

/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Incorrect result:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Correct result after this patch:

http://www.theregister.co.uk/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/
2013-07-29 10:15:39 +08:00
friendica
886c3d1472 bad checkin 2013-05-02 22:19:16 -07:00
Matthew Exon
5b158dbcfb Make the content type available after calling fetch_url 2013-04-01 12:36:43 +08:00
tobiasd
e7c705c9a1 Merge pull request #654 from mexon/filter_var
Use filter_var instead of parse_url to validate URLs, since the latter r...
2013-03-26 11:31:11 -07:00
Matthew Exon
c7cfc6b170 Use filter_var instead of parse_url to validate URLs, since the latter returns true even for invalid URLs, in particular the empty string 2013-03-21 16:27:12 +08:00
Matthew Exon
c53bdfb163 Restore redirect functionality removed by change 452ff19d 2013-03-14 14:09:42 +08:00
friendica
280b72b9e6 Merge https://github.com/friendica/friendica into pull 2013-02-21 16:43:01 -08:00
friendica
615b060352 rev update 2013-02-21 16:42:34 -08:00
Michael Vogel
fdee002f73 Several improvements for performance measuring 2013-01-26 18:35:39 +01:00
Michael - piratica.eu
2c93067682 Performance: Adding several variables for performance measuring. 2013-01-26 16:46:43 +01:00
Michael - piratica.eu
7ab335c9d6 Diaspora: Auto-adding users as friend who already sharing with us 2013-01-20 14:08:28 +01:00
Michael Vogel
452ff19d73 Redirects sometimes only seems to send the hostname and no path. "fetch_url" respects this. 2013-01-13 18:51:48 +01:00
Michael Vogel
5501dd0759 Searching for tags has been enabled in all places. The tag detection has been improved for imports with missing line feeds. 2013-01-13 17:13:01 +01:00