Full-Text RSS 3.9.7

Update – 6 March 2021: PHP 8 bug fix: Warnings produced processing some sites (fixed in ContentExtractor.php). Please download version 3.9.11 from our customer portal.

Update – 2 March 2021: Improved JSON+LD extraction and fixed warnings generated with PHP 8 on some sites. Please download version 3.9.10 from our customer portal.

Update – 26 February 2021: Improved JSON-LD handling and fixed a bug with extracted JSON-LD elements not getting cleared between item fetches on a feed. Please download version 3.9.9 from our customer portal.

Update – 13 February 2021: Some users experienced problems with version 3.9.7 fetching content when running it on servers with a slightly older version of OpenSSL. If you had trouble, please download version 3.9.8 from our customer portal to fix the issue.

Full-Text RSS version 3.9.7 is now available. Full-Text RSS is used by software developers and news enthusiasts to extract article content from news sites and blogs, and to convert RSS feeds that contain only extracts of stories into full-text feeds.

Existing customers can download the latest version through our customer portal.

What’s changed in 3.9.7?

You’ll find a full changelog at the end, but here are the main changes.

PHP 8 compatible

We tested this release with PHP 8 and removed code that was deprecated.

JSON-LD aware

JSON-LD, or JavaScript Object Notation for Linked Data, is used by publishers to embed machine-readable data about articles on their websites. It’s become common practice in recent years.

We’ve noticed cases where the data that Full-Text RSS previously looked to extract from meta tags or HTML elements inside the body are, on some sites, only found inside JSON-LD objects. As such, we’ve added code to our content extractor in this version to look inside these objects for the article title, author and date.

Automatic site config updates fixed

Automatic site config updates through the Full-Text RSS admin pages stopped working recently. We rely on Github for these and the HTTP headers we looked for had changed, so we’ve now updated our code to handle it.

SubToMe.com offers a convenient way to subscribe to a feed in your favourite feed reader. In previous versions we used it in the feed preview and in a <atom:link rel="related"...> field in the feed output. Unfortunately it hasn’t been updated since 2015, so some of its subscribe endpoints (e.g. Feedly’s) are no longer valid.

We now point users to our own subscribe page. Here’s an example: https://subscribe.fivefilters.org/?name=FiveFilters.org&url=https%3A%2F%2Fblog.fivefilters.org%2Ffeed.xml

Note: this is currently hardcoded to use subscribe.fivefilters.org, we plan to offer this as part of the Full-Text RSS package in the future.

Installing on a VPS

If you’d like to try the new version out on a VPS, we can recommend Hetzner Cloud (free €20 credit with link if you’re new to Hetzner). Please see our installation instructions.

Feedback

Please let us know if you have any trouble using this new version.

Changelog

Full-Text RSS 3.9.10 (2021-03-02)

  • Improve JSON+LD extraction and fix warnings generated with PHP 8 on some sites
  • Minor improvements and API parameter description updates

Full-Text RSS 3.9.9 (2021-02-26)

  • Better handling of JSON+LD elements
  • Bug fix: JSON+LD extracted elements not cleared between item fetches on a feed

Full-Text RSS 3.9.8 (2021-02-13)

  • Bug fix: Full-Text RSS failed to fetch content on some servers with older versions of OpenSSL.

Full-Text RSS 3.9.7 (2021-02-11)

  • Bug fix: Item titles (and feed title and description) no longer double-encoded when they contain characters that need encoding
  • Bug fix: Automatic site config updates (if configured via admin page) stopped working due to Github changes
  • Bug fix: Query string param ‘&images=0’ to remove images from output should now work correctly
  • Bug fix: Proxy servers (with auth) and PECL HTTP extension should now work correctly if configured in config file
  • Look inside JSON+LD elements to extract title, author, date (use ‘skip_json_ld: yes’ in site config file to disable)
  • Compatible with PHP 8
  • HTML5-PHP library updated to version 2.7.4
  • SimplePie library updated to version 1.5.6
  • Change subtome.com URLs in the <atom:link rel="related" href="[url]"> attribute to subscribe.fivefilters.org (uses more recent feed reader subscribe endpoints)
  • Minor fixes