Blosxoms, Bryars and Blikis
Recently we heard from Kake Pugh about the OpenGuides project, a wiki-based collaborative city guide system; previously, we heard from Brian Ingerson about his Kwiki wiki implementation. Guides, wikis, blogs … the new fashion in software engineering at the moment is the use of software to help organize, document, and facilitate collaboration – the social software movement is gaining momentum, and Perl is one of the best languages for it.
In this article we’ll look not just at some of the existing social software tools in Perl (focusing, naturally, on my own Bryar blog toolkit), but we’ll look at some ways to break, bend, and embed them in other tasks.
Basic Blogging with Blosxom
When I finally decided that the world would benefit from hearing my internal monologue, I naturally looked around for nice, simple blogging programs. Blogs are, essentially, simple things, and so I didn’t want all-singing, all-dancing web-based database-backed content management systems. I knew that if I were going to blog, it would have to be easy for me to do so, and there’s nothing easier than firing up a text editor in the next available terminal window and doodling my thoughts into it.
Thankfully, Rael Dornfest has the same sort of laziness as I, and created Blosxom, a very simple blogging tool, which simply reads a bunch of files from the file system, finds the most recent and relevant, packages them up together, and sends them at a browser. That was how I saw blogging.
Getting Blosxom up and running is, in keeping with the whole theme of Blosxom, quite simple. You need to download the blosxom zip file, unpack it, drop the blosxom.cgi file in your web server’s cgi-bin directory, and then edit the first few lines of it to tell it the name of your blog and where the entries will live, and you’re done.
Posting into the blog is just a matter of creating a file called something.txt in the blog directory. It doesn’t even matter what the something is. All that matters is the first line of the file, which becomes the title, and the rest, which becomes the story in raw HTML:
First post!
<p> Today I set up a blosxom blog. It was really easy! </p>
You can then, if you like, style your blog with custom header and footer files, CSS, custom HTML for each blog post, and so on.
Enter Bryar
Of course, back when I started using Blosxom, things were not quite so easy, and templating required hacking the Blosxom CGI file itself, and things just didn’t seem as neat as I wanted them to be. I liked Blosxom’s simplicity, but didn’t want to pay the price for it in terms of flexibility.
At the same time, I was learning the wonders of the Template Toolkit, and thought that a well-designed blog should merely collect Post objects and send them to a user-defined template to do what it wants with them.
So I started writing Bryar, which is designed to be a modular, extensible replacement for Blosxom. To make this happen, I needed to provide a default set of components that simply Does The Right Thing. Let’s take a look at that default set before we go any further.
When we install Bryar from CPAN, we’re encouraged to run a command to set up a new blog:
You probably want to run bryar-newblog in a likely home for
your blog once we've finished installing.
The bryar-newblog command simply sets up a decent default set of templates, creates a first blog post, and drops in a bryar.cgi driver, similar to blosxom. At this point, we’re ready to go.
Bryar’s default collection of modules consists of a Bryar::DataSource::FlatFile, which emulates Blosxom’s use of text files in the file system as a database of blog posts; there are Bryar::Frontend::CGI and Bryar::Frontend::mod_perl, which describe how URL parameters are retrieved from the server translated into a request for documents, Bryar::Document which represents each post, and Bryar::Collector which ties it all together. With the benefit of hindsight, that should have been called a Bryar::Controller, since the whole thing can be described as a Model-View-Controller application. But more on that later!
Since all of these things can be overridden, we need something to keep track of which classes we’re using, and the Bryar::Config class does that, reading an optional configuration file but otherwise providing mostly sensible defaults.
In the rest of this article we’re going to look at some interesting ways to override those classes, and build some more funky tools out of this social software framework.
Blogging the Past
Now it may surprise some people to find that I’m not really a programmer, although it may not come as a surprise to some of my colleagues.
I was trained in modern and classical Japanese language and literature, with a side order of linguistics. Programming was a hobby that turned out to make a bit more cash than conjugating 10th century verbs.
But as I began thinking a bit about blogging, I found an old Japanese diary of a lady-in-waiting to the Emperor, called the “Pillow Book”, by Sei Shonagon. It was something I’d read before and found quite interesting, but picking it up again in a blog context put it in a completely new light. It read just like a 10th century LiveJournal.
Here’s a sample entry:
Small children and babies should be fat. Provincial governors should be fat too, and other people who have done well for themselves. If they’re too thin, it makes you think they’re miserable.
As well as thoughts, observations, tales of life at the court, the book contains a large number of random lists, which really made me think of LiveJournal polls and surveys. Here’s another example:
Things I think are elegant
- A white coat over a violet waistcoat
- Duck’s eggs
- Sherbet ice, liana syrup, in a nice new silver bowl
- Rock crystal
- Wisteria
- Snow on plum blossoms
- A pretty little girl eating strawberries
I decided that it would be quite fun to translate the blog – uh, book – in a fairly modern, relaxed style, use fictitious but plausible dates for the entries, and run it as a real blog.
Naturally, I wanted to do this with Bryar, but there was going to be a slight problem; Bryar takes its entry dates from the Unix timestamp of the file, and try as I might, I could not persuade Unix to recognize epoch times a thousand years before the epoch. I had to find another way of getting the times.
But since Bryar is extensible, this is really easy – I just write a new data source class that gets the entry timestamp from somewhere else. I simply created a subclass of Bryar::DataSource::FlatFile called ...FlatFile::Dated, which takes its date from the first line, and title from the next line.
Telling the Bryar blog to use the new data source was simply a matter of writing a little config file called bryar.conf:
source: Bryar::DataSource::FlatFile::Dated
name: Notes on a Pillow - Sei Shonagon's blog
baseurl: http://blog.simon-cozens.org/shonagon/bryar.cgi
And the rest, of course, is just a simple matter of classical Japanese translation.
Blogging the House
But as well as a linguist, I’m also a geek, and I use Perl to control various things around the house – the house network, the division of the phone bill, and so on. This is all driven by Template Toolkit and Class::DBI.
It seemed useful, at the time, to add a blog into this great mass of centralized control, so we could pass phone messages around. Of course, in the end, it was easier to just put a big whiteboard in the lounge. XP’s “the simplest thing that could possibly work” wins again.
But it was fun, and it allowed me to use Bryar in a new way. This time, we’re using the original data source, collector, and other modules, but we’re plugging them into an existing templating system. We already have code to deal with the web server, handle the URL processing, and the output; we already have code to transform objects into HTML; all we need is to collect the recent documents and hand them on to the templates.
I did this by adding a Bryar object to our template variables, and calling the collect_current method on the Bryar collector, using Richard Clamp’s Template::Plugin::Class module. This gave a bunch of Bryar::Documents, which the template system could manipulate however it desired:
[% INCLUDE header %]
[% PROCESS blogmacros %]
<h2> Recent blog postings</h2>
[%
USE c = Class("Bryar::Collector");
FOR post = c.collect_current(blog);
display_post(post);
END;
%]
Everything’s a Wiki
So much for blogs, for the time being. The other movement in social software at the moment is Ward Cunningham’s wiki. A wiki has three important characteristics: a simple mark-up language, the ability for the viewer to modify the page, and the ability to create links from one wiki page to another.
Despite initial predictions that they’d be unusable due to widespread graffiti, vandalism, and other kinds of kicking down the sandcastle, wikis have turned out to work quite well in practice, with sites like Wikipedia collecting a great store of information from contributors both regular and occasional.
CPAN has a fair collection of wiki implementations, including Kwiki, CGI::Wiki, Apache::MiniWiki, and others. However, there may well be times when it makes sense to create a new implementation, particularly when the wiki is to be embedded in another application.
What does it mean to embed a wiki? An example will demonstrate it best. Imagine a web-mail system like IMP or SquirrelMail. Now we want to add the ability to write a short note or annotation on a given message, so that it can be referred back to later. If I get a mail from my friend Al arranging to meet up for a drink at the pub, I’d want to annotate the mail with Al’s mobile phone number, directions to the pub, and the fact that I need to take the book that I borrowed from him last month. Then, in an ideal world, the mail system would collect all the notes, work out the date and time of the meeting, and tell my calendaring software about it, but that’s another story for a different time.
The fact is that once we have these sticky notes for mail, we might want to use them to refer to other mail, group them together in fluid topics, and so on. For instance, if I mentioned in the annotation that this was an AlMeeting, I would expect that text to be magically marked up as a link to all the other AlMeeting mails. Here we’ve got the three elements of a wiki – annotation editing, a simple markup language, and the ability to link.
While the editing is a simple matter of linking your CGI parameters with your database, something that should be second nature to those who want to avoid writing code, the other two components take a little bit of work. Thankfully, chromatic has written a handy module, Text::WikiFormat.
With WikiFormat, you can pass a text string marked up in wiki format to Text::WikiFormat::format, and it will return you HTML:
print Text::WikiFormat::format(<<EOF);
= A Test Post =
I found this '''very''' interesting WikiLink recently.
EOF
__DATA__
<h1>A Test Post</h1>
<p> <br />
I found this <strong>very</strong> interesting
<a href="WikiLink">WikiLink</a> recently. <br />
</p>
This gives us the simple markup, but it doesn’t do much about the linkability; for an embedded wiki, we’re going to need finer-grained control over how the links are constructed. For instance, we might want to pass the link name to a particular URL with a particular CGI parameter. Thankfully, Text::WikiFormat supports two optional parameters in addition to the text to be formatted: the first is a hash reference overriding how the wiki text is marked up to HTML, and the second is a hash reference of options, such as the URL prefix for links.
So, if we have an application that displays a page given a parameter in the URL, we can just say:
Text::WikiFormat::format($text, {},
{ prefix => "http://www.myhost.int/wiki?page=" });
And there we have it, an instant wiki.
Of course, in the context of our webmail application, we may have to do something more convoluted. For instance, we might want a link to SimonCozens to resolve to all mail from that person, if it can be understood as a person who sends email, or to a similarly marked-up set of messages, like an AlMeeting, if it can’t. For this, we need to introduce a bit of logic and decide how to form the link.
As we mentioned above, Text::WikiFormat allows us to specify replacements for the way markup is translated to HTML. WikiFormat uses a hash-ref internally to turn wiki “tags” into HTML, and we can, in fact, put in a subroutine reference for the link tag and have that emit the appropriate HTML link.
sub markup {
my $text = shift;
Text::WikiFormat::format(shift, { link => \&my_link });
}
sub my_link {
my $linkname = shift;
my $name = $linkname;
($name=$linkname) =~ s/(?<=[a-z])(?=[A-Z])/ /g;
if (have_mail_from($name)) {
# Valid correspondent, link to page of their mail
my $id = correspondent_id($name);
return qq{<A HREF="/mail/search?correspondent=$id">}.$name.
"</A>";
}
# Not a correspondent, just a wiki page
return qq{<A HREF="/wiki?$linkname">}.$linkname.
"</A>";
}
What’s going on here? First, we receive the link name in StudlyCaps every time format sees a link. We then convert it to a real name, by inserting a space in the gaps between a lowercase character and an uppercase one. Now we have a name like “Simon Cozens” that we can search our mailstore for, and if we find someone, we can return the URL to a page about their mail. Otherwise, we return a wiki page. We’ve used our webmail annotations to refer to either real people or to other annotations, stealthily embedding a wiki in the application, with minimal effort.
Creating the Bliki
With that example of how to create a custom wiki from an existing application, let’s return to Bryar and our blogging software.
The concept of a bliki – a cross between a wiki and a blog – may seem like a bizarre one at first, but it’s such a natural synergy that it’s been independently invented by multiple people, including Martin Fowler and many others.
Depending on how you look at it, a bliki is either an ordinary wiki with a blog-style front-end showing the latest posts, or an ordinary blog with wiki-like markup and interconnections between the posts. The only difference between these views is how easy it is to edit the posts. We’ll stick with the “ordinary blog” viewpoint, since editing posts over the Web is a boring and solved problem, and one not really related to the technology we’re looking at today.
How are we going to extend Bryar to become a bliki? It’s obvious that Text::WikiFormat is going to be involved somewhere, and it would be helpful to use the Bryar::DataSource::FlatFile class too. First, we inherit from the FlatFile class, and override the make_document method, which is responsible for turning a filename into a Bryar::Document:
package Bryar::DataSource::FlatFile::Bliki;
use base 'Bryar::DataSource::FlatFile';
use strict;
use Text::WikiFormat;
sub make_document {
my ($self, $filename) = @_;
}
Here the format will be slightly different – the filename (such as “MyWikiPost.txt”) will be used to form both the title and the ID of the document, and the content of the document will be transformed via WikiFormatter wholesale.
sub make_document {
my ($self, $file) = @_;
return unless $file and open my $in, $file;
my $when = (stat $file)[9];
my $who = getpwuid((stat $file)[4]);
$file =~ s/\.txt$//;
my $dir = dirname($file);
$dir =~ s{^\./?}{};
my $category = $dir || "main";
local $/;
my $text = <$in>;
return Bryar::Document->new(
title => $file,
content => Text::WikiFormat::format($text, {}, {prefix => "id_"}),
epoch => $when,
author => $who,
id => $file,
category => $category,
);
}
We change the prefix for links to be a relative link of the form id_PostName, since this is how Bryar deals with references to other blog posts, by default. (Of course, we could change this by modifying the FrontEnd class, but this seems to work fine.)
It took 30 lines of code to convert Bryar into a simple bliki, many of which were stolen from the superclass. Now let’s look at something a bit different.
Everything’s a Blog
I mentioned when introducing Bryar that it was based on the Model-View-Controller (MVC) pattern, although I didn’t realize this while designing it.
The MVC pattern is described in this entry in Ward Cunningham’s own wiki, as being made up of three parts:
- A model, therefore is an object representing data or even activity, e.g. a database table or even some plant-floor production-machine process.
- A view is some form of visualization of the state of the model.
- A controller offers facilities to change the state of the model.
All kinds of things follow the MVC design. In Bryar, the model is the data source object; the view is the templating object; the controller is the Collector object. A web browser might be described as a controller that contains all its logic, a view which contains the user interface, and the pane of rendered HTML, and the model of the entire Worldwide Web.
But some web-based applications follow Bryar’s MVC approach particularly closely. Imagine, for instance, a company’s product catalog on the Web. The catalog data will be stored in some kind of database, and will be presented via a templating system, with a controller that will decide which products to show. Isn’t that exactly the same as reading some blog posts from a database (albeit potentially one that looks a lot like a filesystem), rendering them via templating system, with a controller deciding which posts to show?
Indeed, many of the views you will want of the catalog data are exactly what you’d want from a blog: display the most recent products, display all products, search by product name or description, and a detailed display of a single product.
We’ve seen it take 30 lines to turn a blog into a bliki. To turn a blog into a complete product catalog web site takes a subclass of the Bryar::DataSource::DBI class (to talk to the product database) and the Bryar::Document class (to take more details, a URL to a photo, and so on). The rest is a matter of templating, and that’s a designer’s job!
Social Software
I consider the emergence of interest in social software to be one of the most fascinating trends in software engineering this year. Two of the most powerful and popular aspects of this, wikis and blogs, are particularly well-suited for extension and embedding, and Perl is a particularly well-suited language for achieving this.
Although part of the point of this article was to demonstrate Bryar, there were several other important points. First, that there are plenty of Perl implementations of both wikis and blogs that you can choose from; second, that Perl makes it really easy to create your own blog or wiki and customize to your own purposes, including embedding them in an existing application.
But finally, the point was to encourage you to think about good design and the power of extensible applications; if you can create a tool that is both powerful and generalizable – just like Perl itself – it may end up doing wildly different things to what you initially intended!
Tags
Feedback
Something wrong with this article? Help us out by opening an issue or pull request on GitHub
