more chaff

hybridd(8) - poetic nonsense for syslog

nobody@vox.com(Paul Mison) — Tue, 13 May 2008 08:43:49 +0100

Today's stupid software idea comes courtesy of the most recent episode of Battlestar Galactica, which featured (as some previous stories have) the Hybrid, the organic controller of a Cylon base star (aka the big pointy bad guy space ships). This week, though, saw probably more Hybrid than any other.

Hybrids, you see, continually babble, a stream of consciousness mixing what sounds like system diagnostics, physics and poetry. After the episode ended, I thought "wait, system diagnostics? Well, if I open up Console, I have those. What if there was an additional process - call it hybridd - which emitted poetry to go along with the more prosaic debugging and whatnot that my computer spits out?"

I have an idea how to do it, too. Algorithm::MarkovChain is a venerable Perl module that puts out almost, but not quite, meaningful sentences, based on an input corpus. Tie that in to the syslog function, a bit of Launch Services, and there you are. (I'm sure you could do a bogstandard Unix version too.)

A further step would be to replace the Console UI with one that boils down the actual computer stuff and tries to fit it in with the hybrid's poetry, but that idea's a lot harder to do well, I'm sure.

Anyway. hybridd - an idea whose time has come. And now, thanks to Tom Insam, here's a Perl version. Requires the aforementioned module, along with File::Tail and Unix::Syslog, both available at your nearest CPAN mirror.

Read and post comments | Send to a friend

Photoshop Express and Flickr

nobody@vox.com(Paul Mison) — Sun, 11 May 2008 09:31:46 +0100

When Adobe first launched Photoshop Express (from hereon in, PX) a month and a half or so ago, it featured integration with three online sites: Facebook, Photobucket, and Picasa. Unfortunately, I don't use any of those for photos. So when I saw that there was an update to add Flickr support, I dug out my old registration - the one that required me to claim I was living in the US, sigh - and had a very quick look.

Construction (Original)

Flickr appears under the "Other Sites" heading on the left nav - or is it a palette? - of the main window. Clicking the "Flickr" item asks you to authenticate, although somewhat oddly it uses desktop-style auth, so instead of using a nice redirect, PX instead uses a pop up window, which was naturally blocked. It also means you manually have to click about three more buttons than you would with web based auth. Perhaps this is explained by the amount of client-side code, but it still jarred for me. I expect users who don't have to wrangle API auth code would probably cope, though.

Once logged in, Px starts fetching images from Flickr. This is done pretty nicely- pulling each image is slow, so it will carry on if you're not doing anything else, and present the images it's already found for you. Images are sorted by date taken, which is odd if you're used to Flickr's photostream order, which is by date uploaded. (Dates are, naturally, in American format, which annoys me no end, but let's try and ignore that for now.) However, for me it stopped after just over 550 images, which is only about a tenth of the total. I'm not sure why, or how to get it to look for the rest.

Once the images are listed in the Flickr album, they're editable just like any other image available to PX. The tools aren't as sophisticated as those in the main desktop version of Photoshop, Photoshop Elements, or even iPhoto: there's no "levels" tool, and minimal highlight and shadow controls, for example. However, the white balance editor is pretty good, and there's a "Pop Color" effect for those images where you want a red London bus in a monochrome city. Beyond desktop apps, I'd also say that it compares fairly shabbily to Picnik, which is also web-based, but manages a much richer set of tools. Handily, Picnik's integrated into Flickr, making it even more likely to be used.

Construction (Edited)

After editing my image, I wondered where the "save" dialog was, and where I'd get to choose whether the original image was replaced, or who'd get permissions on the new image. It turns out that this is all done automatically. An edited image gets uploaded as a new photo, with your default permissions. The title and description are preserved, but tags and date metadata aren't. To me, this is a killer flaw. Firstly, I want the option to replace an existing image. Secondly, throwing away image metadata is something Photoshop hasn't done since about version 7; it's appalling that PX does this today. Thirdly, I want the option to set privacy levels.

Once again, Picnik's Flickr integration gets all of these things right - in fact, it even seems to have an option to bump images up your photostream with comments intact, which is a very clever trick indeed. In contrast, PX looks like it's hardly trying.

One place that Photoshop Express does try very hard - for publicity - is with images that are copied from its library to Flickr. You can explicitly copy an image into the internal library, create an "album" on Flickr (what's more usually called a set, there), and then copy it back to Flickr in that set. Doing this creates a description that lets everyone know you're using Photoshop Express, and, hey, would you like to use it too? I know everyone is after viral exposure these days, but please let me know you're doing it first and let me set something more sensible.

On the subject of albums, PX loads your Flickr sets as a list of albums, although for some reason this didn't happen the first time I tried it. They're listed in alphabetical order, which, like the "date taken" ordering, is a little odd - Flickr preserves set ordering, and it would be nice if PX would honour that, at least as an option. Opening an album, unfortunately, shows an empty screen, even if there are images in the set. I assume the photo download process is linear. Hopefully a later release will change this, and let the UI take priority, as well as adding caching - each time you open the web app, it has to fetch the list of photos and sets from Flickr afresh.

For all this criticism, I do recognise that Adobe's product is just a beta. On the other hand, given how slick Picnik is, and how nicely it's integrated, it's hard to see how Photoshop Express has much to offer Flickr users, other than a brand name.

Read and post comments | Send to a friend

Gecko, Firebug, and web development

nobody@vox.com(Paul Mison) — Wed, 07 May 2008 22:58:37 +0100

Opera just launched their alpha of Dragonfly, "the foundations of Opera's upcoming Developer Tools", which prompted Tom to note that

"Firebug seems to have defined the universe for this lot."

('This lot' includes Safari's Web Inspector, which is actually not that similar to Firebug - the JS debugger is a seperate app for a start - and IE8's developer tools, which I really should look at once I get a disposable Windows image.) It means that all four of the major browsers now ship with developer tools - an impressive change in the last year.

Tom's observation is mirrored by Michael Smith of the W3C: in his XTech presentation when he says that Firebug sets the standard by which all web development tools are to be judged.

The discussion reminded me of Steve Yegge's point in his long, but worthwhile, post about XEmacs in which he said

IDEs are draining users away, but it's not the classic fat-client IDEs that are ultimately going to kill Emacs. It's the browsers. They have all the power of a fat-client platform and all the flexibility of a dynamic system. I said earlier that Firefox wants to be Emacs. It should be obvious that Emacs also wants to be Firefox. Each has what the other lacks, and together they're pretty damn close to the ultimate software package

To be honest, I suspect what he really means here is Firefox + Firebug. At least, if he doesn't mean that, he should be. For me, doing serious web development now requires that combo, even though I dislike Firefox otherwise.

The really interesting point for me is that Firebug, unlike the three other browser development tools, is actually not under the Mozilla Foundation's control. Firefox ships with a DOM Inspector, but this is more of an internal developer tool. Firebug, a third party tool, builds on DOM Inspector's abilities, and it's built for Firefox because Mozilla have developed not just a browser, but a platform. Maybe when I criticised Gecko for their choice to build a platform as well as a browser I was missing something very important.

That extensibility means more than just Firebug, though. If there's a browser that you want to rewire from inside, using the same tools as you do to create web pages (more or less), it's the one from Mozilla. This is where Yegge is coming from, and I suspect that, hidden in a long post that's titled to attract only command-line editor users, it's a point that's likely to be missed.

Read and post comments | Send to a friend

Terminal's Big AppleScript Bug

nobody@vox.com(Paul Mison) — Tue, 29 Apr 2008 22:29:17 +0100

I've never been one of the people who's seen a need to use any terminal application in Mac OS X other than the one supplied by Apple in Utilities. It does the job, uses Monaco 9, doesn't anti-alias (or at least, can be made not to). It starts up in Mac-ish black on white, and generally Just Works.

10.5 saw a bit of an overhaul, with a much saner configuration interface (all in Preferences, rather than hidden in a rather baffling out-of-the-way inspector) and tabbed browsing. Now, I still don't use tabs on the Mac OS, personally; I like the established app/window split and don't see the need to bring a third level of indirection into play, especially when it doesn't even have consistent shortcuts. (Tabs on Windows? Now that's a different story.) In fact, for years I'd quite happily got by with a bunch of scripts in ~/Library/Scripts/Applications/Terminal/ that would neatly stack all the windows.

Sadly, the new version of Terminal also introduced an annoying AppleScript bug which renders these scripts less than useful. When positioning a window, the vertical positions aren't honoured correctly: instead, the window ends up 320 pixels up the screen from the desired location - OK if you want a window at the top, but certainly not if it's meant to be at the bottom right, which is my usual position. I mention this now because the bug in Terminal that broke my window arrangers will also affect a script to centre windows that TALlama (no really) posted in response to a lazytwitter invocation by John Siracusa. If you try to centre a Terminal window, it ends up jammed at the top of the screen, for no apparent reason.

Now, I'm not down with the cool kids who post radr:// URLs, so if anyone who's reading this is, it's really easy to replicate the error: get a Terminal window towards the bottom of the screen, run this script - which should do nothing, as it's merely putting the window back where it started - and watch your window shift around. Do that, report it, and hopefully eventually I'll be able to retire my "set voffset to 320 -- work around AppleScript bug" line.

tell application "Terminal"

set b to bounds of window 1

set bounds of window 1 to b

end tell

Anyway, thanks for listening, and here's hoping for a better Terminal AppleScript interface in 10.5.3.

Read and post comments | Send to a friend

Why Don't You?

nobody@vox.com(Paul Mison) — Mon, 28 Apr 2008 22:30:41 +0100

Chris Heathcote has a great post - everything ie anything - which looks at a couple of points in Clay Shirky's much-linked Web 2.0 conference keynote about television, web 2.0 content creation, and the "social surplus". Chris says

I find it a flimsy argument that grinding in World of Warcraft, watching Youtube videos, or I dunno, playing Sim City for 40 hours straight (Spore is going to kill me) is in any way better than watching TV, merely because it’s ‘doing something’.

Get a Life!: The Little Red Book of the White Dot

Last week, I noticed Iain Tait taking part in Mental Detox Week. Previously, this was Turn Off TV week, and years ago, when I read the White Dot book about television, and recognised a lot of what it was saying, and decided to do what it said. In the ten years since then, I've lived in a house with a TV for only three.

Did I ever miss TV? Not really. I had the radio for news, and much more besides - and the reports seemed much more raw, without the need for a camera crew to tag along (although I know usually the BBC was using the same people for both). Newspapers and magazines brought information to me. I still went to see films - in fact, one year early on I nearly saw a hundred films, many of them foreign or classic.

You'll notice something missing from this. Back then, home internet was still a luxury, with dial-up charges and per-minute call costs. It wasn't until 2000 that I got always-on internet. Over the years, I've slipped more and more into using the laptop for more and more of my leisure time. At first I was easily able to convince myself that this was different. I was creating and communicating. Look, here's all my friends on IRC. Here's the website I just built, and the photos I've taken.

Recently, though, I'm finding that belief harder to sustain. I can produce, but I also consume more. There's Flickr friends to surf, a dozen news sites to read. Along with IRC, there's Twitter- and I don't feel comfortable holding conversations there*. Instead of creating websites, I pour content into Vox and a blog - and there's less and less of a personal touch, more a sense of a piling up of stuff. I can spend an hour flicking though videos on YouTube, surfing through Orbital live performances, or I can spend an evening on iPlayer watching BBC Four documentaries.

Wait. I can spend an evening on iPlayer watching BBC Four documentaries. Isn't that what I was trying to get away from? Is it really that much better for me to be watching TV with the ability to be snarky about it via instant messaging? So it's tempting to switch the computer off, and do something less boring instead. No wonder Mental Detox Week includes computers as well as television now.

It's hard, though. It's harder than it was with TV. There, I was ignoring imaginary people. Now, I'd be ignoring friends, family, along with random - but real - strangers, whose work I've liked for one reason or another. I'm also finding it harder to socialise, now I've let ties wither down to merely digital connections. I have to use a computer at work, whereas avoiding TV there is trivial. Nonetheless, it's something I think would be worth some effort.

If I can't do any of that, though, I do need to make sure that I make my time at the laptop worth as much as possible. The Internet isn't inherently better than TV, but if I make the right decisions, I hope I can at least make sure it's not worse.

* I know plenty of people do. I don't.

Read and post comments | Send to a friend

More Aggregation Definitions

nobody@vox.com(Paul Mison) — Mon, 28 Apr 2008 12:00:21 +0100

In the wake of Zeldman's post, two definitions:

Shallow Aggregation: what I do. A single home page that pulls in concept from other sites, but where the links leave my site to go instead to the place the content is hosted.
Deep Aggregation; what Tom Insam does. An entire site fed on and republishing content from other sites, in so far as possible. Allows a consistent look.

Questions.

Are the joins better when they're more obvious (shallow) or better hidden and (arguably) less predictable?
Is there a market for software for either sort of aggregation? (MT provide "Action Streams", which are shallow. There's Django code for deep aggregation.)
Is there a future for either kind, given Friendfeed et al?
Given the questions above, is it even worth defining the above?
Is aggregation the right word? Republishing? Personal mirroring?
Why am I so keen on defining things as a way of understanding them?
Does anyone else care?

Read and post comments | Send to a friend

Further WebKit vs Gecko Thoughts

nobody@vox.com(Paul Mison) — Sat, 19 Apr 2008 23:50:03 +0100

Earlier this week, I published a post entitled Hackability: Gecko vs WebKit. Actually, it sort of snuck out in the middle of the night; as is my wont, I left it as a neighbourhood-only draft overnight, only to find that Simon had turned up and made some of the points I was going to go back and edit in as a comment. Since nobody ever reads comments on the internet, and also since Simon, Tom and myself had a bit of a discussion on the subject on IRC, I thought I'd follow up the post with some further thoughts.

Firstly, Simon made further mention of Gecko 2, the next-big-thing coming from Mozilla. It does sound like it's going to be a great deal of work, and that it's probably dumb for them to be gung-ho about fixing some of the Acid3 issues in the 1.9 branch when 2.0 is around the corner. On the other hand, the very fact a 2.0 is needed is a bit worrying, and I'd also argue that perhaps there should already be a code branch being worked on. In comparison, I strongly suspect that the WebKit team have been lucky enough to time any major reworkings of their codebase to be out of the public eye.

Secondly, Gecko and WebKit both turn out to be about the same age- nearly ten. Admittedly, KHTML was initially much more minimal¹. On the other hand, this turns out to be key to the point I really should have made in the first post.

One of WebKit's stated project goals is hackability. In contrast, Gecko's layout engine doesn't really have a project page; the closest thing there is to one admits that

much of the code for the cross-platform toolkit is mixed in with this code, it is described elsewhere

This is really, I suspect, where the two projects diverge. WebKit is designed as an embeddable renderer. It's pretty modular - one of the KHTML legacies is that the JavaScript implementation is separate, as you can see from the fact that Quartz Composer embeds JSCore but not WebKit. It does one thing, and does it pretty well. (The same is somewhat true of the most common browser using it, Safari.) Given this, it's no wonder that it's adding features and passing tests at a rate of knots.

In contrast, Gecko is part of Mozilla's platform. It's a fairly major part of XULRunner, a cross-platform application framework that hosts Firefox, Thunderbird and other apps. That makes it big and gnarly, and probably makes it harder to work with, but it also offers a great deal of flexibility. Firefox is so extensible because the UI is itself written in a markup language, and it can be modified and extended without learning (too much) new stuff. In contrast, there are no supported ways to extend Safari's interface².

So far, I think we all agreed. Different goals lead to different priorities, and as Simon put it:

Teams focus on different things. saying "Well it's Apple's policy not to do plugins" is like saying "It's Mozilla's policy not to spend resources on what could quite reasonably called a PR exercise" but, as we know from Perl, PR exercises and easy hackability keep a project alive.

Where we parted company was that Simon argued one of the keys to the success of Firefox was this extensibility. Since it comes at a cost, it would be good if one could argue that it had also had a benefit. Unfortunately, personally, I doubt that the success it's managed against IE (which, compared to the market share of its prehistoric ancestor, Netscape Nagivator, isn't actually that good - although numerical I'm sure Firefox has more installs than Navigator ever managed) has anything to do with extensibility. Do real people install extensions - even lauded ones like Firebug and AdBlock? Somehow, I doubt it.³

No, the success of Firefox is down to offering a familar enough UI (Opera fails here, however compelling the features are once you get through to them), free (again, until recently, an Opera weakness), and with better security than IE (perhaps less so now, but older versions of Microsoft's browser didn't get their reputation for nothing). That's not to say that the work on XUL isn't good - extensibility helps get geek mindshare, just like passing Acid tests - but again, I do wonder if it's costing more than it's worth. After all, if easy hackability keeps a project alive, what does a lack of the same do?

¹ A fair chunk of the discussion was about how KHTML and WebKit were related. I don't think it's particularly relevant here and nobody bothered with the relevant Slashdot forensics, so I'm going to skip it.
² Hopefully that wording is clear enough to be understood as meaning "things that aren't browser plugins". I believe that none of the other WebKit based browsers offer much of a UI customising API either.
³ I'm more willing to believe they install themes. Shudder.

Read and post comments | Send to a friend

More Transport Informatics

nobody@vox.com(Paul Mison) — Fri, 18 Apr 2008 11:30:23 +0100

Dan Hill posted an incredibly wide-ranging roundup of developments in transport informatics a couple of days ago on his site, and it's justifiably been widely linked to by the sort of people who care about such things. Of course, I'm one of them. I also tried to add a few things he missed in a rather dense, link-laden comment. Unfortunately, it was so link-laden that Typepad thinks it has to be comment spam, and it's now spent two nights in an approval queue, so I fear it's lost. Fortunately, I have my own blog, so here are some additional, somewhat London-centric notes.

Holisitic As Chris Heathcote noted when I mentioned the Guardian story on Helsinki's bus mapping, the statement that "every bus and tram is visible on a Google map" isn't true as of yet - the beta only covers a couple of trams and a few buses, if I'm remembering correctly. However, it's a promising start; London's iBus will generate similar raw data, and we'll see if the will's there to use it. Maybe someone should ask the mayoral candidates?

Here's a piece on the implications of Google Transit in the UK.

Rail The UK's National Rail maintains real-time arrivals and departures information, available on the web. It's probably unrealistic to map more than the trains going to a single terminus at the moment, because scraping doesn't scale, but that would be an interesting (hackday/barcamp?) project. A few Tube lines have similar information available, but that'll be harder to scrape into a single map.

Maritime There's a real-time high-resolution map of shipping in the San Francisco Bay available.

Walking Although not strictly informatics, on the subject of walking in London, there's a couple more initiatives; Legible London is a trial of maps for tube stations, bus maps and street pedestals, currently limited to the Bond Street area, but aiming at a unified signage standard for the entire city, and next week there's a Walk to Work day, encouraging people to walk, if not the entire distance, the first or last 30 minutes of their commute.

Edit: as often happens when I do something like this, it's now emerged. Thanks, Dan. I'll leave this up for posterity, though.

Read and post comments | Send to a friend

Hackability: Gecko vs WebKit

nobody@vox.com(Paul Mison) — Mon, 14 Apr 2008 23:59:31 +0100

This post has been brewing for a while, but I've been prompted to actually write it by seeing John Gruber's offhand remark on his most recent linked list entry, about CSS gradients in WebKit:

Just me, or is WebKit racing way ahead of Gecko in terms of support for cool new stuff?

No, it's not just him. WebKit, and Opera's layout engine Presto, raced towards Acid3 compliance in March, with both effectively reaching a photo finish on the 26th. Meanwhile, Microsoft hasn't even shipped a non-beta Acid2 passing browser¹; no surprise there. But where's Gecko, the Mozilla layout engine, the one that powers Firefox?

Well, to be blunt, it doesn't look as if they care much. We have one developer saying that Acid3 is basically worthless, and another (more diplomatically) stating that it's a missed opportunity and an exercise in making browsers jump hoops, rather than improve "real" functionality. As others (almost certainly more qualified than I am) have noted, this sounds a lot like the noises from Microsoft around the time of Acid2's release.

The thing is, I'm not here to kick Gecko, but to understand its problems, if it has them. Does the team's response to Acid3 mean it does? Possibly not on its own, but coupled with events like the move of Epiphany to WebKit², and the aforementioned speed of development on WebKit (and to a lesser extent Opera's Presto³) I have to wonder. Why is development there so slow?

One stated reason is that the Mozilla Foundation is on a rush to release Firefox 3, and it's certainly true that it is coming up for release. On the other hand, Apple certainly seem to be able to keep the open-source WebKit tree distinct from the version used in releases - Safari 3.1 shipped with an Acid3 score of 75 when nightlies were scoring 90-odd - so I should hope that's not the real reason. Maybe they're pulling people off the layout engine to work on the browser? That's not as stupid as it sounds for most apps, given the way the Firefox UI is set out using XUL, a markup language. Even so, it feels like a bad use of engineering. Maybe Gecko's reached that point where extending it's no fun. The language the team themselves uses, with talk of Gecko 2, makes me wonder if that's true.

I don't have answers, anyway, but I'd love to hear from people who do why Gecko is giving the appearance of stagnation, while WebKit seems full of life.

¹ Bafflingly, it seems that Microsoft develops not one, but four layout engines: Trident, for IE/Win, Tasman, originally for IE/Mac and now part of Office:Mac, and two unnamed engines, one in Word and Outlook 2007, and another in Expression Web Designer.
² Admittedly that's got contributing factors beyond merely Gecko; it sounds like the wrapper they were using to embed it (GTKMozEmbed) had some seriously nasty issues of its own.
³ I mentioned to Tom Insam that I was surprised I'd never heard of the name of this engine, but he sagely noted that, as it's not open source or embeddable, there's no reason I would have.

Read and post comments | Send to a friend

Rotating Video for Flickr - Solved

nobody@vox.com(Paul Mison) — Sun, 13 Apr 2008 19:47:11 +0100

As hoped, after my previous post detailing my failure to rotate a video for display on Flickr, I've had a useful hint which has since led to me successfully uploading a couple of videos.

Gareth pointed me at a page detailing how to use mencoder, part of mplayer, to rotate the video (along with a bunch of other steps at the end assuming you want your video to stay landscape in the end; I don't.) At first I had no luck getting it to work, but a download of the source and compile (which was error, if not warning, free on this Intel Mac) left me with a working command-line application. Starting with Scott Hansleman's invocation, I ended up using

./mencoder -vf rotate=1 -o output.mov -oac copy -fafmttag 1 \ -ovc lavc -lavcopts vcodec=mjpeg input.mov

to convert the file from the Ixus to one that Flickr would accept. The important bit is the "-vf" tag, which gives a list of video filters to apply; -oac and -ovc are for the output audio and video codes respectively, and I'm using mjpeg as it's the same as the input. (Using "copy" didn't work, probably because it was using the frames without rotating them).

Ideally there'd be a nice front end for this, but most of the mencoder GUI wrappers I've seen for Mac OS X were abandoned long ago. For now, using the command line is fine for me, and if you're trying to do the same, hopefully it'll work for you too.

Read and post comments | Send to a friend