Arto Bendiken on Drupal

The Universal Timeline Aggregator

Arto — Fri, 29 Dec 2006 09:37:02 -0800

For those who haven’t yet come across it somewhere on the web, may I recommend checking out the Timeline widget developed by David F. Huynh of the MIT Simile project. It’s a snazzy DHTML/JavaScript tool for visualizing chronological events on a scrollable, graphical timeline — sort of a Google Maps for temporal information.

I’ve been working quite a bit with the Simile widget lately, co-developing (with David Donohue) a module that integrates the widget into Drupal, allowing Drupal sites to display any CCK / Views content as graphical timelines.

Today, inspired by Alexandre Passant’s RSS2Timeline implementation, I sat down to code up a generic web service that can take any Atom or RSS feed and convert it into a JSON-based event source for the Timeline widget. My goal was to make it absolutely trivial to embed live Atom/RSS timelines into blogs and whatnot, so that anyone with basic HTML skills could use timelines without having to go through the relatively complex technical setup the widget requires.

I hereby present the Universal Timeline Aggregator , available at http://timeline.to/ (I’ve been snapping up Tonga’s dot-to domains since getting the ultimate vanity domain, ar.to, as a Christmas present; the “to” preposition works rather nicely for the present purpose, too.)

Here’s a screen capture (and live example, if you click on it) of the sort of timeline display you can create in a minute or two using the timeline.to service:

Timeline view of “recent Ruby on Rails development”:http://dev.rubyonrails.org/timeline.

Embedding a live, interactive Atom or RSS timeline into any site is now as easy as copying and pasting the following HTML snippet, with the appropriate modifications:

<iframe src="http://timeline.to/http://www.mysite.com/rss.xml"
  width="500" height="400"
  scrolling="no" frameborder="1"
  marginwidth="0" marginheight="0"></iframe>

Just replace http://www.mysite.com/rss.xml with a real URL address to an Atom or RSS feed, and modify the width and height as you like. Autodiscovery of feeds is supported to a reasonable extent, so in most cases you won’t even need the exact URL to the feed; the website’s URL address itself will do.

Here’s another example as a screen capture, this time showing the popup box that opens when a timeline event is clicked:

Timeline view of Planet Scheme, showing a preview of a blog entry.

Starting out on the timeline.to implementation today, I had to actually pause a moment to contemplate which technology to use: Python, Ruby or PHP — it wasn’t quite as clear-cut a decision as usual.

Case in point, while I haven’t done much Python coding recently (since defecting to the Ruby camp), the language does have some excellent libraries and frameworks going for it — including arguably the best Atom/RSS parser in existence, the Universal Feed Parser written by Mark Pilgrim. Considering the staggering number of malformed and invalid feeds out there, a good parser is essential (I won’t get further, right this moment, into the delicious irony of having to parse XML formats using regular expressions).

On the other hand, Ruby also has (at least) two relatively comprehensive and decent libraries for feed parsing, FeedTools and Syndication. Unfortunately, in my experience neither library is quite up there with the Universal Feed Parser yet, and neither seems particularly active recently.

In the end, underdog PHP won out on this project on purely practical points: since my Timeline module for Drupal is written in PHP, it makes sense to try and reuse code both ways between the timeline.to service and Drupal.

Investigating the current best way to parse both Atom and RSS feeds with PHP, I learned of a new feed parser library for PHP called SimplePie, which has been gaining a lot of momentum lately (indeed, it seems to be on track for eventually surpassing MagpieRSS as the de-facto RSS parser for PHP). The SimplePie developers are apparently in the process of porting the 3000+ unit tests from Pilgrim’s parser, which certainly seems a promising prospect for creating a truly robust parser.

SimplePie is also bundled with the Feedparser Drupal module, so again, it all just makes sense. The library turned out to be quite painless to work with, and has, so far, been able to parse all the feeds I’ve thrown at it. (I did have to disable SimplePie’s ad-removal feature, as that was eating up the entry descriptions on some Atom feeds.)

Feedback on the timeline.to service is welcome. If there’s sufficient interest, I will consider adding further functionality such as iCalendar support, and perhaps mashup features allowing multiple feeds and data sources to be combined into a single timeline display.

Easier Drupal Debugging with Trace

Arto — Sun, 03 Dec 2006 14:28:33 -0800

Trace is a project that started, like these things tend to, as an innocent, experimental Drupal module I hacked together to implement all of the core hooks exported by Drupal.

I was at the time working on Boost and wanted to gain a deeper understanding of the conditions and exact sequence in which the Drupal module hooks were invoked, so I took the API listing and wrote out the skeleton for each hook, sprinkling logging statements to capture and output the arguments passed to the hooks by Drupal.

The initial output looked like the following; the comma-separated list after each hook name is the list of modules which implement that particular hook in my development installation:

[HOOK] hook_init: boost
[HOOK] hook_menu: filter, node, system, user, watchdog, boost, ...
[HOOK] hook_node_grants: 
[HOOK] hook_db_rewrite_sql: node
[HOOK] hook_node_info: page, story
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy
[HOOK] hook_db_rewrite_sql: node
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy
[HOOK] hook_link: node, comment, taxonomy
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy
[HOOK] hook_db_rewrite_sql: node
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy 
[HOOK] hook_link: node, comment, taxonomy  
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy
[HOOK] hook_db_rewrite_sql: node
[HOOK] hook_nodeapi: boost, comment, menu, path, taxonomy
[HOOK] hook_link: node, comment, taxonomy
[HOOK] hook_footer: 
[HOOK] hook_help: filter, node, system, user, watchdog, boost, ...
[HOOK] hook_exit: boost

Tracing Drupal Hook Invocations

In the course of things it dawned on me that there was, of course, a much better way to do all this — namely by dynamically defining the hook functions on the fly, and injecting into each a call to debug_backtrace() to obtain not only the arguments passed to the function (which could be gotten with func_get_args(), after all) but also the full stack trace (call history) that led to the hook invocation.

Saddened as I was to obliterate the couple of hundred lines of code I had at that point, I took the plunge and was left with three dozen lines of code that did the same thing, just better — the joys of exploratory programming in a (somewhat) dynamic language.

As a secondary effect, this also gave me a facile way to implement a kill switch that rather thoroughly disabled the module when necessary:

The Trace master kill switch.

Stack Traces for Drupal Hooks

Next, I wrote helper functions to output the stack traces in a sensible, programmer-friendly format (in fact, using PHP syntax), so that the output looked like this:

[HOOK] hook_menu: filter, node, system, user, watchdog, boost, ...
       trace_menu(FALSE)
       call_user_func_array('trace_menu', array(FALSE)) @ `includes/module.inc':203
       module_invoke_all('menu', FALSE) @ `includes/menu.inc':1196
       _menu_append_contextual_items() @ `includes/menu.inc':220
       menu_get_menu() @ `includes/menu.inc':391
       menu_execute_active_handler() @ `index.php':15

Since the list of hooks to tap into had become metadata, it was natural enough to externalize all that into an INI file, while at the same time adding support for non-core hooks. This called for a user interface for controlling which hooks to activate and trace:

Trace settings for hook invocations.

Error Logging with Stack Traces

When coding the stack tracing, another light bulb went on about how this could be valuable for debugging PHP errors — not just in a development environment, where other debugging solutions might be available as well, but even on production servers. If one could get useful stack traces from a production box, that would aid tremendously in tracking down elusive PHP errors from environments which could be running rather disparate PHP/Apache/MySQL stacks when compared to the development environment.

This turned out to be trivial to implement — simply a matter of overriding Drupal’s error handler to add stack frame collection and tracing, and chaining back to the original error handler afterwards. No harm done, except for more milliseconds lost on each error in the name of higher programmer productivity – good motivation to minimize the number of errors ;-)

Trace settings for PHP error tracing.

To reduce the time spent hunting for the causes of errors, I also added an option to output the actual source code line that triggered the error, just preceding the full stack trace up to that point. The following example illustrates one line in the Drupal 4.7 code base that looks like it could use a call to isset() to prevent triggering a number of spurious, if ultimately harmless, PHP notices:

[NOTICE] E_NOTICE: Undefined index:  sort @ `modules/comment.module':1677
         if ($_GET[$setting]) {
         _comment_get_display_setting('sort') @ `modules/comment.module':707
         comment_render((stdClass)array('nid' => '2', 'vid' => '2', 'type' => 'story', ...))
         node_show((stdClass)array('nid' => '2', 'vid' => '2', 'type' => 'story', ...))
         node_page()
         call_user_func_array('node_page', array()) @ `includes/menu.inc':418
         menu_execute_active_handler() @ `index.php':15

Extensible Trace Output Drivers

The module sort of snowballed from there. Upon consideration of how to best perform trace output logging in production environments, I came to the conclusion that it would be a good idea to add an abstraction layer between the actual tracing-related function calls and the subsequent physical output, which had until then been exclusively written to a dedicated log file.

This led to the concept of output targets or “drivers”, with File being the default target:

Trace output driver settings.

In addition to File, currently implemented is also a Syslog driver. The idea of syslog support is that you can have the Trace module activated (with minimal tracing options, e.g. only trace errors), on your production server and tie the output to your existing system log analysis and reporting facilities.

Two other drivers I’ve been considering implementing are a TCP socket driver and an e-mail backend. The former could, by its ephemeral nature, be useful in speeding up “deep tracing” (with the File backend, having all trace options enabled kills performance both due to the volume of data being written to disk as well as the constant fsync()‘ing), while the latter would, again, be useful for production environments: imagine your mission-critical Drupal application sending you a bug report e-mail when a PHP error occurs; said e-mail containing a wealth of information on the page request, server state, and a full stack trace of the code that caused the error.

The way that the output driver concept is currently implemented is quite flexible, by the way — the module actually defines a new hook, hook_trace(), which is invoked for each tracing event. (This hook itself can, obviously, not be traced.) Any modules that implement this hook — Devel would be a good future candidate — have access to all trace output as it happens; the existing Trace output drivers are concrete implementations of the hook that just happen to be provided by default.

Tracing Database Queries

I also tapped into Drupal’s undocumented SQL query tracing facilities (which ostensibly exist to support the Devel module) and added output for each query executed during the page request:

[QUERY] drupal_lookup_path: SELECT COUNT(pid) FROM url_alias
[QUERY] drupal_lookup_path: SELECT src FROM url_alias WHERE dst = ...
[QUERY] module_list: SELECT name, filename, throttle, bootstrap ...
[QUERY] cache_clear_all: DELETE FROM cache WHERE cid = 'variables'
[QUERY] cache_get: SELECT data, created, headers, expire FROM ...
[QUERY] node_access_view_all_nodes: SELECT COUNT(*) FROM ...
[QUERY] pager_query: SELECT COUNT(*) FROM node n WHERE n.promote  ...
[QUERY] pager_query: SELECT n.nid, n.sticky, n.created FROM node ...
[QUERY] drupal_lookup_path: SELECT dst FROM url_alias WHERE src ...
...

In the interests of brevity, I haven't included any timing information in any of the above examples of the trace output. In actuality, however, each trace log message includes a microsecond-precision time delta measured against the start time of the page request. (The actual accuracy, though, perhaps being somewhat more coarse-grained due to time lost to tracing.) The timing information for SQL queries was a bit of a pain to implement using the currently available query debugging facility in Drupal 4.7. As a result, the timestamp for each SQL query is, for the moment, nothing but a best-guess value. Not that this slight inaccuracy will matter for most uses, but once I've wrapped up version 1.0 of Trace I plan to submit a Drupal core patch that will not only make the tracing simpler (from the core point of view) but allow for significantly increased functionality, too.

Additional Debug Output

Just to supplement the information available in the trace output, an option was added for dumping the contents of the PHP superglobals at the start of the page request, and for including further debug information (such as the HTTP headers that were set during processing of the page request) at the of the response:

[REQUEST] 2006-12-03 22:46:12.306834 GET / HTTP/1.1
[DEBUG  ] $_ENV = array('PATH' => '/usr/bin:/bin', 'PWD' => '/', ...)
[DEBUG  ] $_SERVER = array('HTTP_ACCEPT' => '*/*', ...)
[DEBUG  ] $_COOKIE = array('DRUPAL_UID' => '1', 'PHPSESSID' => ...)
[DEBUG  ] $_REQUEST = array('DRUPAL_UID' => '1', 'PHPSESSID' => ...)
[DEBUG  ] $_GET = array('q' => 'node')
[DEBUG  ] $_SESSION = array('node_overview_filter' => array())
...
(other output omitted)
...
[DEBUG  ] drupal_set_message() = NULL
[DEBUG  ] drupal_set_header() = array('Content-Type: text/html; charset=utf-8')
[STATS  ] Total SQL queries: 59
[RESPNSE] 2006-12-03 22:46:12.453383 200 OK

Improving the Signal-to-Noise Ratio

Now that things were rolling, I was getting plenty of trace output – in fact, way too much. A single page request for my development site’s front page would easily generate 30 megabytes of output. Obviously, it was time to put some effort into cutting down on unnecessary output.

First, I added a setting for filtering which SQL queries would be traced and which summarily ignored:

Trace settings for database queries.

More importantly, however, it was getting to be a pain to browse the administrative interface with tracing enabled; to solve this, I implemented a filter to control trace activation based on page paths and wildcards, in a manner very similar to how block visibility is determined in Drupal:

Trace page filter settings.

The module’s fine-grained advanced settings are also instrumental in determining how much output will actually be generated:

Additional trace settings.

Current Status and Future Directions

At present, I have mostly finished the features that I want for a release one-oh of the module — with the notable lack of watchdog message tracing, which has not yet been implemented. The module should already be stable enough to serve as an everyday asset for intrepid Drupal developers, and any help in testing it would certainly be most appreciated.

Beyond 1.0, there are a number of directions the module could take. For one thing, I’m in the process of experimenting with some of PHP’s more advanced features which might provide rather powerful facilities that Trace could take advantage of. It is not inconceivable that something like a JavaScript-based, real-time PHP breakpointer or debugger could be built on top of these language constructs and the foundation already laid in Trace.

Before embarking on such flights of fancy, though, I’m interested in hearing feedback on how the module has been, or could be, useful for you. What’s the most valuable use case, in your opinion? Is it about troubleshooting production environments, empowering Drupal developers, or both? If you have any thoughts on the matter, please leave a comment or contact me directly.

Static Page Caching for Drupal 4.7

Arto — Sun, 28 May 2006 12:34:53 -0700

This weekend I tackled coding up a Drupal feature I’ve been sorely missing on many a past project: static page caching. My yet-to-be-named module is a replacement for Drupal 4.7’s built-in caching. Instead of storing the pre-generated cached pages in the database, the module stores them in a cache directory on the file system.

Big deal, right? Actually, as I’ll demonstrate below, it makes all the difference.

What Drupal’s built-in caching does is just cut down on the code that needs to run on each page request, while reducing the database access to a single query that retrieves the cached page to display. This in itself provides a marked speed-up, sure enough, but still necessitates invoking PHP on each page request, and what’s worse, opening a connection to the backend database. Those connections become a scarce resource once the going gets tough.

In contrast, my cache module exports Drupal pages into plain old static HTML files. When a page request can be satisfied from the static cache, PHP is bypassed in its entirety, and the web server serves the cached file straight from the disk at whatever ultimate top speed it is capable of (most HTTP servers these days, including behemoth Apache, are able to saturate a 100 Mbit/s pipe when serving static files from an adequate server box).

Here’s what this looks like in practice, from a quick benchmark on my PowerBook (1.67GHz PowerPC G4). The yellow bars represent page requests served from Drupal’s standard, database-backed cache storage, and the bluish bars shooting waaay to the right of them are how many page requests can be served when we throw off the yoke of PHP and shed our dependance on SQL:

As you can see, even in this trivial benchmark, the performance boost was more than an order of magnitude.

(This was a makeshift benchmark on an underpowered laptop. The benchmark consisted of timing a Drupal installation’s 16K front page using Siege. This was with Drupal 4.7.1, PHP 4.4.1, Apache 2.2.2 and MySQL 4.1, the latter three binaries as compiled and installed from DarwinPorts, all running pretty much with their stock settings. Additionally, I threw eAccelerator 0.9.5b2 into the mix as well, to give PHP performance a hand, since without it, Drupal’s database-backed caching had trouble completing the higher concurrency levels.)

Requests per second isn’t the whole story, either. The above chart doesn’t show the detrimental increase the standard caching experienced with regards to page response times (i.e. the time the visitor has to wait for a page to finish loading) as the load factor grew. In the chart below, shorter bars signify faster page loads:

The best thing about caching a Drupal site as static pages is that it’s dead simple, in every respect: it’s trivial to setup, easy to manage, and facile to scale. Throw in a good lightweight web server (Lighttpd comes to mind), and you’ll scale to the very limits of your hardware, with few arbitrary road blocks to hold you back. That’s not the case with Drupal’s database caching, which requires you to tune the MySQL configuration, and more often than not, mess around with operating system settings, such as file descriptor limits, to handle even moderate traffic.

As a testament to the above, it took me over an hour to configure the MySQL 4.1 daemon on my laptop in order to enable the database-backed cache to even pass the 100x concurrency level benchmark. Before I figured out how to adjust the measly limit of 256 file descriptors that Mac OS X gives processes by default, every other page request was failing with Drupal complaining of not being able to connect to the database. Unless you happen to have stocked away some sysadmin experience, a situation like this could have you scratching your head for a bit.

What’s worse, on a shared host mucking around with system-level stuff is often not even possible nor permitted. Many hosts also impose implicit, arbitary limits on the CPU time that you are allowed to consume on a daily basis, meaning that if you run a popular site with a dynamic CMS like Drupal, you may be facing a forced upgrade to a dedicated server.

Serving out static pages, on the other hand, is what web servers do best. You can run even a very popular site, as a static version, off a shared host without receiving that dreaded e-mail from the billing department.

Come to think of it, with this module, you might e-mail them for a discount since you wouldn’t even be using, or needing, your “fair share” of server resources. Better yet, depending on what kind of site you run, you might install Drupal on your own computer, downgrade to a cheaper hosting account that doesn’t provide PHP & MySQL support in the first place, and just upload the generated HTML files via FTP or rsync. The static page caching is multisite-compatible, meaning you could keep a single, private “master” copy of Drupal to manage and publish all your sites.

So, what’s the catch, you ask? Well, the usefulness of static page caching really depends directly on what kind of site you run; namely, how “dynamic” your site is. Obviously, if you make use of forms or features that submit information from the visitor back to Drupal (e.g. comments, the feedback module, etc.), your site can’t be exported into a 100% static format, since something still needs to handle receiving the form submissions (or to put it technically, POST requests are not cached).

A similar example would be if you have, for instance, a quotes block and consider it very important that it updates with a random quote on every page request, instead of every 5 minutes (say), as would be the case using static caching.

Other than the above considerations, the module’s first and foremost current limitation, shared with Drupal’s standard caching, is that only pages served to anonymous visitors are cached; requests from logged-in users are passed through to Drupal in a normal fashion. This means that if you run a large community where users need to register and login to participate, most of the user interaction will still need to be dynamic. (I have figured out a way to keep a user-specific cache, as well, but I’m not yet sure the implementation is worth the effort or complexity. If you think otherwise, please leave a comment to that effect, or drop me a private e-mail.)

What the static page cache is ideal for, then, are personal blogs, corporate sites, portals, directories & the like, where most of the site is targeted at anonymous users, and only the occasional feature (say, posting a comment or sending feedback) will need to bypass the cache and be handled by Drupal. For these kind of sites, you can probably benefit from static caching in over a good 95% of your site.

I have a long list of client sites running Drupal 4.6 that I’ll be upgrading to 4.7 as soon as possible in order to let them benefit from this module. But first, I will need to test the module on a couple different shared hosts, and of course, implement it on this site. I’ll polish the module up for general distribution soon after I complete the above. (For the time being, if you wish to give it a spin, please drop me a line.)

Till then, here are a couple of screenshots to tide everyone over:

The extended cache configuration in Drupal’s settings screen.

The administrative interface for the Ajaxy cache rebuild process.

Update (2006/06): some additional details are available from my post to the Drupal development mailing list.

Update (2006/10): there’s now a project site and issue tracker on drupal.org. Please note that the appropriate place to post support requests is in the issue tracker, not as comments to this page (support request comments will get summarily ignored).

Update (2007/07): Justin Miller has a great write-up about using Boost on Drupal 5.x, with lotsa technical details and a very cool logo to boot.

Update (2007/08): Boost is now available for Drupal 5.x, too. Many thanks to Alexander Grafov for the initial porting work.

Drupal vs Mambo

Arto — Wed, 08 Feb 2006 06:15:00 -0800

I originally wrote this article for the Xaneon Development site way back in August 2005. After Xaneon shut down and this article became unavailable, a number of people e-mailed me to ask for it, so I dug up my local copy and decided to publish it here. Note that since this was written, Mambo has become Joomla and both Drupal and Joomla have evolved and come out with new releases. Nevertheless, for what it’s worth, here follows the article in its original form.

Since our announcement regarding the impending demise of XE2, we’ve received a ton of e-mail from people asking us to elaborate on the reasons we decided to “defect” from Mambo to Drupal. Instead of answering this question over and over, we’ve decided to publish a brief (well, that was the intention, anyway…) overview of what has led us to, ultimately, abandon Mambo.

Table of Contents:

General Comparison (page 2)
Where Mambo Shines (page 2)
Technical & Architectural Differences (page 3)
Application Programming Interface (page 3)
Internationalization (page 4)
Search Engine Friendly URLs (page 5)
Multiple Sites with One Installation (page 6)
Fine-Grained Access Control (page 6)
Other Considerations (page 7)
In Conclusion (page 7)

Now, when making any comparison such as this, the first thing to realize is that it’s unlikely that one tool will fit every need and purpose. To an extent, comparing Mambo and Drupal is comparing apples to oranges. They are two very different systems, with widely differing goals and intended audiences. They just both happen to fit under the convenient heading of “content management systems”, meaning you can create decent websites with either one of them.

Also, the people on our team probably don’t represent the broad cross-section of Mambo users out there; our needs don’t necessarily fit their needs. We are. for the most part, people with strong technical backgrounds. Some of our reasons for abandoning Mambo may not resonate at all with the “average user”, if there is such a person. Unless you are a programmer, or have an equally deep technical understanding, take everything we say with a grain of salt.

Let’s start off with some background: we’ve used Mambo for a long time, and we’ve written a multitude of components, modules and mambots for various versions of Mambo, as well as contributed patches and improvements to existing Mambo add-ons. Most of our own projects were coded for internal purposes or limited distribution only, but some, like our popular XE2 component, which provided Search Engine Friendly (SEF) URLs for Mambo, were released to the Mambo community under open-source terms.

It was only recently, in the last few months, that we “became aware” of Drupal and overcame the initial threshold of trying it out. Funnily enough, this was mostly to steal some good ideas from their SEF implementation; instead, Drupal ended stealing us away from Mambo. So it goes sometimes. Read on for what caused us to abandon all the time and effort we had invested into the Mambo platform…

(...continued on next page)

General Comparison

Mambo has, for a long time, been ahead of competing CMS projects with regards to marketing. Mambo’s public image is pretty, appealing and very marketable to management. Mambo has no doubt benefited from the sponsoring company and trademark owner Miro’s advertising dollars in this regard.

Of course, this was all before the recent hostile project takeover by Miro; while there are people who will argue that any publicity is good publicity, we’ll just have to sit and see how this particular move will play out.

Drupal, in contrast, has been a community project from the start. There’s been no single strong corporate sponsor (though that’s changing with entities such as CivicSpace throwing their weight behind the project) to hold sway over the project, and the overall image has been somewhat less glitzy and more downplayed.

Where Mambo Shines

Mambo is certainly “easy on the eyes”: most people react very favorably to seeing the administration interface for the first time. Another aspect where Mambo is definitely ahead of the game is in installation friendliness, as well as add-on management (installation and uninstallation of components, modules, etc.)

In comparison, Drupal requires one to manually unzip add-ons on the server, possibly create the necessary SQL tables from supplied scripts, and there is no friendly installation “wizard” to guide you through first-time installation. (This is all evolving, though; it shouldn’t be many months before there is a comprehensive installation system available in Drupal.)

If the above points are very important considerations to you, as they certainly might be to less technically-savvy users, then you may not really benefit from this article. As stated, our team hails from a quite different user segment.

(...continued on next page)

Technical & Architectural Differences

Now we come to the real meat of the matter. During our sojourn into the dark art known as Mambo SEF, we’ve necessarily become quite familiar with the internal workings of Mambo. To be candidly honest, it’s not exactly impressive.

Mambo is a very limiting design. Pretty much the only “hook” into Mambo’s core, to allow any significant third-party extension to Mambo’s base functionality without modifying the core files, is the SEF support. (A cynical person might argue that even this “hook” exists only for the benefit of a certain commercial SEF extension developed by a former core developer.)

When we originally designed Xaneon Extensions for Mambo, the most interesting aspect for us was to provide a general “core extension” framework which would allow multiple components to utilize XE2 to tap into the core without disturbing the SEF support or interface. Essentially, a “multiplexing” of the one and only core hook, for the benefit of all. XE2’s own code is very extensible, allowing other developers or site administrators to override functionality as they need to. Hopefully someone else will take this idea, or even our code, and run with it.

Drupal’s architecture, in comparison to Mambo’s, is like a breath of fresh air. First off, reading through the core code was a pleasant experience: it’s not only cleanly designed, and with extensibility constantly in mind; it’s also well commented and documented. The design is incredibly flexible from the view point of an add-on developer coming from Mambo: there is no need for separate “components”, “modules” and “mambots”, as one Drupal module can perform the tasks of each, and much more besides. There is a general-purpose, all-pervasive “hook” system in place, allowing modules to override functionality in the entire lifecycle of content objects (known as “nodes” in Drupal), as well as perform actions at certain points in the handling of page requests.

Application Programming Interface

In Drupal there is actually an API for modules to use; this is a very important point. One reason for the inconsistent quality of Mambo components, and the proliferation of non-standard user interfaces, is no doubt that there are no standards, or at least no significant high-level programming interface that would facilitate the creation of third-party add-ons. Essentially, to create a Mambo add-on, one needs to start from scratch and reinvent the wheel every time. It gets frustrating after a while, even though the developer may learn to refactor and reuse some of his code.

(...continued on next page)

Nowhere are the above-mentioned design considerations more visible than in two add-on features we consider absolutely essential for both Mambo and Drupal: internationalization (i18n) and search engine friendly (SEF) URL addresses. Both require fairly low-level “hooks” into core functionality to function.

Internationalization

There is i18n support available in both systems, but in both Mambo and Drupal some patches are required to the core; this in our opinion is inexcusable, in both cases. Only unilingual users and websites would not need i18n, and those kind of websites are a luxury on the European side of the Big Pond, at least.

However, compare the patching required, and how the add-on is implemented in each case: that is, in Mambelfish versus Drupal’s i18n module. In such a comparison, Drupal’s clean design shines through very favorably: the i18n module is much less of a “dirty hack” than what’s required for Mambo, and builds upon the superb localization feature of Drupal. In fact, Drupal’s i18n patch is only about two dozen one-line code additions, and would be trivial to integrate into the Drupal core in a future release, should Drupal’s project leadership decide to do so.

More so, now having lots of practical experience building multilingual sites with both Mambo and Drupal, Drupal’s i18n works very well and as it is intended to, and feels fully integrated into the system, not “hackish” in any way.

This is not, however, intended to knock down the developer of Mambelfish; having reviewed the code and contributed patches to Mambelfish, we think he’s done pretty much the best job that’s possible in the constraints Mambo’s architecture places on him, and should rather be commended for his ingenuity in providing an adequate solution to a difficult problem.

(...continued on next page)

Search Engine Friendly URLs

This is a subject that has become intimately familiar to us. One of our earliest requirements for a CMS was adequate support for free-form, human-readable URLs. At the time we started to use Mambo, there really wasn’t any solution to this aside from some hacks shared in the Mambo forum, and a commercial SEF add-on for Mambo developed by one of the core developers.

To solve the problem, we wrote Xaneon Alias Manager 1.0, which allowed us to define friendly URLs manually for all of Mambo’s content pages. While it worked quite well, and hundreds of websites are still using XAM today, it was a labor-intensive solution for larger sites. Eventually, out of dissatisfaction with having to maintain the friendly URLs manually grew Xaneon Extensions 2.0 (XE2) for Mambo, which introduced automated friendly URLs for both content items and component-generated paths.

Unfortunately, while achieving a good measure of automation, the component’s complexity skyrocketed. Mambo itself is simply not designed for friendly URLs, and what’s worse, most of the third-party components for Mambo don’t take any measures to even try to support friendly URLs. As a result, writing a “perfect” SEF solution for Mambo is an incredibly labor-intensive task. We welcome anyone to try it as a Sunday-afternoon exercise…

Over the last year, many people have privately expressed their concerns to us about the fact that a Mambo core developer is charging for a SEF component that should, in their view, be integrated in Mambo’s core. From our own “inside perspective”, we can, on one hand, understand this developer charging for his component, since we know the work involved in developing a comprehensive Mambo SEF solution. (Come to think of it, charging for ours might have motivated us to actually finish it.) On the other hand, we must agree that Mambo should come with significantly better SEF support out-of-the-box, and that the current situation is curious, to say the least. We do believe the (former) core team to be in serious error on this matter.

Fortunately, the situation with Drupal is much better. SEF support, on a level comparable to our Alias Manager component, is fully integrated into the Drupal core, and there is an open-source add-on available which adds automated friendly URLs in a manner comparable to XE2. There are a few small convenience features we are considering implementing and contributing, but already Drupal’s SEF support is more integrated and usable than we could ever achieve in Mambo without patches to the core. It simply works as intended, and the code and solution is elegant.

(...continued on next page)

Multiple Sites with One Installation

Imagine if you could set up several sites with just one installation; all installed add-ons and themes would be instantly available for use in all these sites, and any security patches you need to apply (after all, all software has bugs, as we know all too well), need only be applied once.

And, to make it all even sweeter, imagine if you selectively share the configuration settings and databases tables between these sites so that, for instance, you could share user accounts between some sites, or maintain only one large content archive, selectively using and sharing articles between your sites. Wouldn’t this be great?

Guess what. Drupal does all of the above, and it works perfectly. It’s not too bad to set up, and once done, saves a bunch of time and effort.

We figured out a way to do most of the above in Mambo as well, since XE2 has some access to the relevant PHP variables before handing control over to Mambo’s core. The multi-site support in current beta versions of XE2 is buggy, but the concept should be sound. It would take a lot of work to bring it up to Drupal’s current level, though.

Fine-Grained Access Control

As far as access controls go, this is simply a no-contest. We all know how lacking Mambo’s access control scheme is; hopefully there will one day be an improvement to it.

Drupal allows us to define arbitary roles (for instance, “moderator”) for users, and assign permissions to these roles on a module-by-module basis (for example, one selection is “moderate comments”). It’s intuitive, very fine-grained and very flexible. There is even a module available that will automatically handle giving a user a new role after they’ve paid for a subscription with PayPal, as well as many other add-ons that allow true micromanagement of the website’s access control policies, should that be necessary.

(...continued on next page)

Other Considerations

Let’s just say the list of goodies Drupal delivers us is pretty long. We haven’t even discussed Drupal’s templating system (keywords: XHTML, CSS, total customizability, and optionally, Smarty) or the unlimited content hierarchies you can create with the built-in Drupal module called, appropriately enough, “taxonomy”.

We could go on for two dozen pages. It wouldn’t be difficult at all. But we don’t need to convince anyone, and our reasons aren’t necessarily reasons why someone else would switch. It would be silly to get religious about something like a content management system. As always, just pick the right tool for the job at hand. For us, that has for now in most cases proven to be Drupal.

In Conclusion

The bottom line is simply that Drupal allows us to be significantly more productive. In our experience, we are able to put together complex sites in a fraction of the time it would have taken us with Mambo. We can use Drupal modules such as Flexinode, which brings to mind some of the power of Lotus Notes, to quickly solve needs that would have cost us a lot of custom programming (or buying a custom component) in Mambo. We can take advantage of Drupal’s excellent templating to produce truly unique-looking sites with not much effort.

This site, xaneon.com, is at the moment still a Mambo installation, running our Xaneon Extensions for Mambo. It’s the last bastion of the old, standing against the coming wave of change. It’s unlikely to hold out much longer, though…

We don’t regret the time we spent with Mambo; we will always have a soft spot for it and appreciate the many friends and acquaintances we made along the way. We sympatize with the current plight of the “true” Mambo core development team and wish them the best of fortunes in their new endeavour. As we’ve stated previously, we might pop in at Open Source Matters from time to time to catch up on how the “rebel Mambo” project is doing. Goodbye, and thanks for all the fish!

References and discussion regarding this article: