5 posts tagged “programming”

When I posted about lib-flickr-minimal, I noted that the newly-launched flickr.places.placesForUser method made a more interesting demo of data you could fetch when authenticated than, say, showing a user's most recent private photos. Evidently the developers at Flickr agreed it was an interesting concept, because over the last couple of months that area of the API has been extended considerably, As a result, I've expanded the demo into an AppJet application of its own.

Where? What? When? is the result. It shows you, on a map, the locations with the most photos according to a given criterion: by default, that's a tag, but it can also show your photos, or those from your friends and family, or your contacts. You can then inspect a place and see the most recent relevant photos, or the most popular tags, for that location.

How did that evolve from the initial demo app? Instead of simply printing a table based on Flickr's response into the document, I directly plotted the results on the map. I added a small form to enable the choice of criteria, and when Flickr added the placesForTags method, I added that as a choice. Belatedly, I realised that would also work for users without authentication, so I removed the requirement to authenticate, and made tags the logged-out default. (The image above shows a slight change to the initial results: it's the same tag, London, but at the neighbourhood, not locality, level. All of the locations are within the greater city's area, which probably won't be a surprise, but that's not true for Paris. Evidently, what happens in Vegas doesn't always stay there.)

The design of the application isn't quite settled, but I knew I wanted to replace the standard Google Maps pushpins with partially-transparent circles. Initially, I went with red, but when I showed it to colleagues, they said it reminded them of maps of bomb blast radii, so I spent a while looking around for the right colour, before settling on a yellow. The circles themselves are scaled according to the natural log of the number of photos for that location; I played with square roots as well, but I feel that logarithms give the right sense of scale.

The last piece of work I did was adding tag display for locations, using the tagsForPlace method. These tags can be surfed: clicking on one will load a new search for the given tag. It's noticable that the first few tags for most places are almost always place names, while common tags seem to share a familiar pattern of scattered, similarly-sized circles across the US, Europe, south-east Asia and coastal Australia.

There's still a few things I could add; tag persistence in URLs (to make it easier to share pages), better loading indicators (especially initially), options on which photos are shown, and links to view the search on Flickr itself, for example. There's also a missing question: while the API methods support maximum and minimum times, I haven't yet added options to allow you to show When? However, for now I think I've done enough (and I'll note that the site has a link to view the source of the application, if you fancy hacking on it yourself.) Enjoy.

Previously I've done a couple of braindumps of blog posts I should have written. This time, they're site ideas, and I don't have the time to do them. Hell, I'm not even getting the half-finished features on snaptrip done; too busy putting in silly easter eggs. (At least people like the look of them.)

• twitter links to delicious
• especially for Siracusa and Gruber. Man, those two post so much.
  
• twitter link parser
  
• you know, if they had OAuth I'd be a lot happier about this one
• delicious network same-link collapsing
• when five people all link to the same thing, maybe it's really important
• so don't show it more than once
• delicious note detection
• show on a site the bits that people who leave notes on delicious choose to comment on
• xml-rpc to tumblr post bridge
• for flickr blogs, and the like
• auto-stream
• I'm sure I've mentioned this before
• use Google social graph APIs to fetch all the URLs for a user
• put out a stream like adactio's to show their activity
• but do it all magically!

Part of the point of jotting these all down is that execution > ideas, so if you want to take any of them and implement them, please do.
   

I've never been one of the people who's seen a need to use any terminal application in Mac OS X other than the one supplied by Apple in Utilities. It does the job, uses Monaco 9, doesn't anti-alias (or at least, can be made not to). It starts up in Mac-ish black on white, and generally Just Works.

10.5 saw a bit of an overhaul, with a much saner configuration interface (all in Preferences, rather than hidden in a rather baffling out-of-the-way inspector) and tabbed browsing. Now, I still don't use tabs on the Mac OS, personally; I like the established app/window split and don't see the need to bring a third level of indirection into play, especially when it doesn't even have consistent shortcuts. (Tabs on Windows? Now that's a different story.) In fact, for years I'd quite happily got by with a bunch of scripts in ~/Library/Scripts/Applications/Terminal/ that would neatly stack all the windows.

Sadly, the new version of Terminal also introduced an annoying AppleScript bug which renders these scripts less than useful. When positioning a window, the vertical positions aren't honoured correctly: instead, the window ends up 320 pixels up the screen from the desired location - OK if you want a window at the top, but certainly not if it's meant to be at the bottom right, which is my usual position. I mention this now because the bug in Terminal that broke my window arrangers will also affect a script to centre windows that TALlama (no really) posted in response to a lazytwitter invocation by John Siracusa. If you try to centre a Terminal window, it ends up jammed at the top of the screen, for no apparent reason.

Now, I'm not down with the cool kids who post radr:// URLs, so if anyone who's reading this is, it's really easy to replicate the error: get a Terminal window towards the bottom of the screen, run this script - which should do nothing, as it's merely putting the window back where it started - and watch your window shift around. Do that, report it, and hopefully eventually I'll be able to retire my "set voffset to 320 -- work around AppleScript bug" line.

tell application "Terminal"

    set b to bounds of window 1

    set bounds of window 1 to b

end tell

Anyway, thanks for listening, and here's hoping for a better Terminal AppleScript interface in 10.5.3.

A year or so ago, whilst writing groupr (RIP), I came up with what I thought was a useful name for something I found myself doing a lot: the API join. I'm fairly sure this is common to a lot of Web (2.0) APIs, but it's especially common with Flickr. For example, take groupr. First, it would do a call to get the groups you're a member of. For each of these, it then fetched the photos in the group. Obviously, this has a problem: as the number of groups you're a member of goes up, so does the number of calls to the API - and each call takes about a tenth of a second. The only way to mitigate this, and the solution groupr used, was to page the groups - and even that leaves you making as many calls as you have groups on the page.

The problem reared its head again when I was looking at doing a ffffound-inspired Flickr favourites app. I wanted to display the usual Flickr size, rather than square thumbnails (as Flickr's own favourites page does). Unfortunately, the standard call to get favourites didn't list the size of the photos, and I really didn't want to spend two seconds fetching them all. Other people have raised similar questions on the Flickr API group discussions; for example, here's one about getInfo and getExif, and here's another about getting photo sizes.

Imagine my surprise, then, when I looked at the documentation for flickr.photos.search and noticed a new argument to the "extras" parameter: o_dims. It turns out this returns the original height and width, and is also available in the favorites methods, so now it's possible to avoid doing those calls, and to embed derived height and width for web-scale images from a single call, even for the 36 or so images on Flickr's version of the page.

Of course, this is simply because the API has now moved the join deeper; instead of being at the API level it's being done inside Flickr (presumably at the database level). In fact, I suspect that last weekend's database downtime may not be unrelated (perhaps it was needed for the launch of Apple TV's Flickr slideshows?). It also doesn't help with the other methods, such as getExif (there's a reason I've moved some of my EXIF data to machine tags, which are fetchable with another extras parameter to many calls).

Facebook, interestingly, allows a SQL-like query language as part of their API access, but I wonder how they deal with queries that could bring the database to its knees. I do notice the line

In order to make your query indexable, the WHERE should contain an = or IN clause for one of the columns marked with a *.

Is that an enforced criteria, or is it merely a recommendation, and do they return long-running queries without results to keep up database performance? It's the sort of thing I'd love to see Flickr add to their API, but I can imagine the problems are far from trivial, and in the meantime, I'm very happy to see one API join bite the dust.

or, why I won't be migrating to Google Mashup Editor

Last winter I wrote a small web application called groupr which let you look at photos from your Flickr groups in one place. I did that partly because I thought it would be useful (it is, a bit) but also because I wanted to play with the platform the company I'd once worked for had released. That platform was called Zimki, and it was, in hindsight, a pretty ambitious thing for a small company to attempt.

Zimki is a hosted server-side Javascript application framework, complete with a data storage model and a templating engine (actually two). It makes it easy to quickly knock up a small web application, or at least it does once you wrap your head around it and get hold of tools to save you using the (frankly awful) web pages provided for editing your app. It's the closest I've seen to what Marc Andreessen has called level 3 platforms:

A Level 3 platform's apps run inside the platform itself -- the platform provides the "runtime environment" within which the app's code runs.

Indeed, this is exactly how Zimki works. Unfortunately, it's also closing by Christmas.

There's a big company out there who also have something that looks a bit like a level 3 platform. Google Mashup Editor also lets you run code in a hosted environment, build multi-page sites, and read and store data from the web. Unfortunately, there's no way I can port groupr to it.

Firstly, there's a very limited set of server-side computations allowed. If there's not a module or control for something, you can't do it. This is the first thing I ran across. groupr has a local config file with my Flickr API key and secret, and it uses MD5 to calculate the required parameter for the authentication step. It turns out that's impossible with GME.

Secondly, there seems to be no way to pass data in to the application other than by user interaction or feeds. Notably, you can't inspect HTTP query strings, the mainstay of web programming since 1994. Since Flickr's frob is returned in an HTTP parameter, this means it's impossible to ever use GME for a Flickr application that requires authentication. Well, actually, you could do everything on the client side, but then why use a hosted environment at all? I doubt I'd get enough use out of its templating language to justify the effort.

As with my complaints about Skitch, this is almost certainly a case of me wanting a product to be what it's not, but I can't help feeling that if Google aren't prepared to build a real Level 3 platform, nobody is. In the meantime, I suppose revert to writing my own apps with web frameworks that use SQL and which need work to scale, just like everyone else does.