Introducing Sfursat: a WordPress-powered social web publishing platform

tl;dr: Sfursat is being developed as a flexible platform based on WordPress as backend CMS, with several options for the frontend, ranging from simply WordPress itself (therefore effectively Sfursat is WordPress in this scenario), to static site generators fetching data from WordPress-as-an-API, to Single Page Apps, to packaged Firefox OS apps.

Sfursat Begins: Static Site Generators

I have been an enthusiastic user of static site generators for more than a decade - well before they became trendy: and, as a developer, what’s not to love? - you get all the convenience of simple text-based formats for source content, ability to easily version content and templates, predictable and testable builds of output pages and deploys… and speed, speed, speed and scalability, of course.

Quite predictably, i even wrote my own SSG back then: it was originally called TextFlow, was written in Perl and it used Template::Toolkit for templates and YAML for (meta)data. It took me half a day to write, and it served me well for my own document workflows - and it still does.

In 2009 i started working on a nightmarish legacy site (it had been ‘managed’ with FrontPage and then Dreamweaver for almost five years), and while preparing a proper migration to a true multi-user CMS i added a new workflow type to my TextFlow site generator: using XML for data, XSLT for templates, and a crazy but lovely Git-backed storage layer.

Sfursat Reloaded: git-based distributed awesome

The nice thing about this storage layer, which made it different from most SSGs, is that you could plug in any number of Git repositories at arbitrary ‘mount points’ of a master document space, apply XSLT transformations managed in multiple Git repositories, and (guess what) retrieve configurations from multiple Git repositories. Oh, and it could deploy via make and Git to any number of destinations. And it was fast.

This second version of TextFlow did its job perfectly until the migration of legacy content to WordPress was done. However, the whole TextFlow-based project had a bus factor of 1, and all the content updates were done by me even though whoever was writing content would have been very much able to publish it directly, if using a web-based CMS.

Fast forward to 2014, and i’m mainly developing for WordPress these days, keeping things sane thanks to the awesome Pods framework, while still using SSGs (mainly Nanoc) for some sites which are seldom updated.

Plain static output: overcoming WordPress

Still, for most client projects at Xelera we don’t really need WordPress to deliver pages and assets on the frontend: in virtually all the projects we have been developing lately, and most of the sites we target with our vertical product (more on this in a later post), once each item has been published (be it a page or a Pods item), any pages based on the published content won’t change until one of the items on which they are based is changed again. No comments, no pingbacks, no member content, no restricted content…

In fact, each page is effectively cached through W3 Total Cache, while a further scalable caching layer in front of the whole WordPress is managed through Varnish, with media items served through a CDN.

So, it would make sense to just really serve static assets off a scalable cloud filesystem rather than somehow “staticifying” WordPress as we do.

Fully semantic content management: overcoming Static Site Generators

On the other hand, even for the simplest microsites we almost always need to manage fully semantic content: we define several content types besides pages, posts and categories, which basically means relying on Pods both to enter/update content and to process during each request all the content needed to generate frontend pages.

As most pages rely on a multiplicity of content objects managed through Pods, and as each content object is typically used on several pages (for example, a person’s data is used on the person’s individual profile page, on a team page, on the pages of each publication authored by the person, and so on), this is the only way to make sure that editors need to update each bit of content only once and this is then kept nicely in sync across all the relevant pages.

Even for personal projects (where using a text editor and Git from the command line is not an issue), the convenience of Pods (or an equivalent system) to manage content objects and relationship between them is hard to beat. It is surely possible to break content up in semantic pieces even when using flat files (for example by using YAML, JSON or XML to mark up content, data and metadata), but even in single-user scenarios, without a solid SQL (or even NoSQL) backend things can get out of control fairly quickly.

Sfursat: Semantic backend, any frontend

Sfursat seamlessly blends the convenience of a robust backend with the power and speed of a static site generator for the frontend.

In recent projects, we have increasingly been developing WordPress theme code with a strict separation of data processing and templating: basically our OOP code builds PHP data structures that are then fed to a Twig templating layer (using HAML syntax through our twig-haml-templating library), which generates the HTML pages. All WordPress and Pods API calls are relegated to the OOP layer so that the Twig templating layer becomes just one of the possible views of the generated data structures.

It is then easy to imagine a further step: thanks to the new WP_API and the Pods plugin for this, the PHP data structures used by Twig can be encoded as JSON, and eventually fed to… pretty much anything. Recently we have been using JSON data as the Model for AngularJS and Ember apps embedded in WordPress pages, as underlying data structure for map-based visualizations with Odyssey, and we’re working on a Firefox OS app generated from the JSON dump of a subset of WordPress/Pods data already used as to generate an event microsite.

What we are now developing with Sfursat is a set of content pipelines that allow to directly integrate WordPress, used mainly as an API engine, with static site generators such as Metalsmith, Broccoli Taco, Nanoc - whatever. The integration is initially going to be one-way (WordPress to static site generators) but bidirectional integration is on the roadmap, allowing for example to treat WordPress as not much more than a clone of a Git repo, able to synchronize its contents from any other clone in the same fashion one would keep several clones of the Git repository used to generate a website through a static site generator.

To be clear, this is nothing new in terms of design (see for example the nice discussion at Improve WordPress with NodeJS, via this rather old StackOverflow thread), but with Sfursat we are creating reusable libraries to allow anyone to easily assemble any of our pipelines for their own WordPress operations.

Sfursat: a Social web platform

And here’s where the social side of Sfursat lies: once your WordPress site effectively becomes “just” an API, you can assemble your frontend(s) from ready-made components developed by others, and - if applicable - make your website’s data available to third parties to use within other web frontends.

Content producers and editors can still enjoy the familiar and easy-to-use WordPress+Pods dashboard, while enjoying maximum flexibility in terms of how content ends up being displayed, distributed and used, as well as making it possible to develop any frontend code with very efficient dev tooling (see Broccoli Taco for an example of this) which are just not “the WordPress way”.