3 posts tagged “aggregation”

In the wake of Zeldman's post, two definitions:

• Shallow Aggregation: what I do. A single home page that pulls in concept from other sites, but where the links leave my site to go instead to the place the content is hosted.
• Deep Aggregation; what Tom Insam does. An entire site fed on and republishing content from other sites, in so far as possible. Allows a consistent look.

Questions.

• Are the joins better when they're more obvious (shallow) or better hidden and (arguably) less predictable?
• Is there a market for software for either sort of aggregation? (MT provide "Action Streams", which are shallow. There's Django code for deep aggregation.)
• Is there a future for either kind, given Friendfeed et al? 
• Given the questions above, is it even worth defining the above?
• Is aggregation the right word? Republishing? Personal mirroring?
• Why am I so keen on defining things as a way of understanding them?
• Does anyone else care?
  

I've been thinking a lot about aggregation (and lifestreaming) recently. Actually, that's not quite right; I've been thinking about it for a while, but the launch of friendfeed and another look at jaiku¹ brought it to the forefront again. (This post also overlaps somewhat with my Feeding the Daemons post over at my other blog, but I hope it's set out a bit more clearly.)

So, what do these services do? I'll try and make a couple of definitions that might be useful, even if only within the context of this post. Firstly, there's horizontal aggregation. This is at the level of a single service, but it spans users. Most of the entries in my Safari bookmark bar - pages I can get to with a single key combo - are like this: my friend's twitters, photos from my contacts, my del.icio.us network, and friends pages here and on LiveJournal. This means that I'm usually seeing the same sort of object (very short text, photos, links, and posts respectively). Lots of sites offer this (Tumblr's dashboard is another example).

Secondly, there's vertical aggregation. This spans services, but only takes in a single individual. My home page is an example of this, although it's not exhaustive. The best example I've seen is Les Orchard's accumulator, because it lets you control how much or how little you see, and it makes it very clear what's being collected. There are plenty of other examples; one of the first I saw was Jeremy Keith's stream.

Vertical aggregation is actually quite popular, and you can turn a bunch of services over to it; for a while that's what I used tumblr for. In the end, though, I decided I wanted a bit more control, and did it myself. More often, people don't have the infrastructure to collect everything together nicely, which is where services like LoudTwitter (which collects your tweets for the day to your blog) and Twitter Updater (which posts to Twitter announcing blog posts) come in. Unfortunately, they're a bit hit and miss; it's possible to configure circular posting, and I've seen people use tools to post their twitters to more than one of their blogs, which seems a bit much².

So, what's new with Friendfeed and Jaiku? Well, they're a combination of horizontal and vertical aggregators: they pull in updates from lots of people and lots of sites. The obvious problem is that, if you're dealing with purely horizontal or purely vertical aggregation, you'll never see the same item twice. If you start mixing them, though, duplication can become a real problem.

What's the solution? Well, I'm not sure, but it seems that filtering is going to be pretty important; not just the crude level of cutting out feeds, but individual items. In fact, full-blown email-type filtering ("if subject contains 'tweets for today' then hide") might well become important.

In fact, I wonder if the failure to tackle this is part of the problem with Facebook that the alpha geek crowd seems to have. They're the people who will import their web activity into the site, and so will their friends, meaning they're bombarded with not just the usual zombie/film/quiz noise, but also every twitter, photo and blog post. It's no surprise that many run from the information overload. Will real people suffer from this? Perhaps in time, but for the moment I suspect most Facebook users are getting about the right balance between no activity (so why use it?) and overload.

In conclusion, then. If you run a web site with a social component, please offer the tools to allow users see their friends activity easily on the site, and also to include their own data in their own aggregators. (To be honest, almost everyone does this already.) If you're running a service that allows users to pull in data and share it with friends, filtering is a must.

¹ Jaiku's odd, as it combines Twitter's status updates with aggregation, and that makes it much harder to explain to people. If it had concentrated on one or the other it might have found Twitter's market first.
² Movable Type 4's activity streams show that Six Apart know it's a problem that needs addressing. Oddly, Vox has enough information to build one, but it's not used.

Everything seems to be a follow-up these days.

It turns out that Pipes (now? always did and I never noticed?) exposes a y:published property, which looks suspiciously like the Perl DateTime object mentioned previously. Having seen that, I've now hammered three of the four feeds I want to aggregate into a single feed.

For the fourth source, Flickr, I need to decide if I'm doing a simple life stream (more on that concept at Jeremy Keith's journal), or something more complex where I can differentiate between a single photo with a narrative or days with many images, in the style of Phil Gyford's home page. Not that I suspect anybody cares much. Doesn't everyone just use RSS these days anyway?