Calling Yahoo REST web services with Ruby query parameter problems

I’m an idiot and req = Net::HTTP::Get.new(url.path+'?'+url.query) is my friend.

Just playing around with Ruby and the Yahoo Web Services. The example they give for calling a web service is this:

require 'net/http'

url = 'http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=madonna&results=1'
resp = Net::HTTP.get_response(URI.parse(url)) # get_response takes an URI object

data = resp.body

I am behind a proxy at work, so my code looks like this:

require 'net/http'

@proxy_addr = 'wwwcache.bath.ac.uk'
@proxy_port = 3128

url = URI.parse('http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=madonna&results=1')

req = Net::HTTP::Get.new(url.path)
res = Net::HTTP::Proxy(@proxy_addr, @proxy_port).start(url.host, url.port) {|http|
    http.request(req)
}

data = res.body

puts data

The response I get is:

<?xml version="1.0" encoding="UTF-8"?>
<Error xmlns="urn:yahoo:api">
  The following errors were detected:
  <Message>invalid value: appid (empty or missing)</Message>
  <Message>invalid value: query (empty or missing)</Message>
</Error>
<!-- ws01.search.re2.yahoo.com uncompressed Wed Nov 29 05:54:50 PST 2006 -->

So the query parameters are being stripped from the GET request. I have no idea why. Any ideas? I’ve not had this problem with making queries through our proxy, although a look with Wireshark could prove that they’re making it through.

Oops – I think I’m actually stripping the parameters myself there by creating a new Get using url.path but even changing to just do a puts http.request_get('WebSearchService/V1/webSearch?appid=YahooDemo&query=madonna&results=1').body doesn’t work, which I thought might have had a better chance.

Dumping BlogBridge

I was doing OK with BlogBridge, but in the last month it’s dropped all my data twice and then prevented me from re-importing all my feeds by disabling all the feed action buttons and menu items such as “add feed, import feed” etc.

Luckily it does an automatic export of your subscription list as OPML every time you make a change to the list of feeds that you read (in %profile%.bbfinalbackups), so importing it into another reader should be easy.

It’s also lucky that I don’t use any of the special features like “pinning”, or smartfeeds, because I’d be royally pissed off; as it is I’m just quite annoyed that I’m going to have to trawl, yet again, the painful mess of Windows desktop aggregators.

Twittering

I’ve finally signed up to twitter now I realise that it’s not SMS-only (because what fool pays for that?) and I can update via IM. It seems cute.

The IM bot didn’t recognise my Jabber address (or at least I didn’t receive anything at that address), although it did pick up my Gmail account. The bot always appears offline to me though, which means that it doesn’t show up in my contact list until I turn on “show offline contacts”, which is very annoying, and means I’ll probably forget all about it.

I’m also quite surprised that there aren’t already a host of plugins which pick up your IM status and post that to Twitter (or vice versa). A single point of entry for desktop status makes sense to me.

I see that there is a script which checks your iChat status and posts that off to Twitter every five minutes, but that’s not quite as integrated as I was hoping for.

Overall, I do like the notion of Twitter, at least in some sense, although, like Phil Gyford I was reasonably surprised that I’d have to set up yet another social network. I did some work on this last year, I’ll see if I can do anything more practical about it. There’s an obvious application for creating your FOAF file and using a service’s API for auto-creating a social network.

XBox 360 advertising splurge

I thought I was going mad this morning as I was reading the metro (a free daily newspaper in the UK) – there seemed to be ads for games *everywhere*. I realised a few pages in that they were all for the Xbox 360.

In fact, it was so outrageously obvious (two two-page spreads, including a Dead Or Alive calendar), that Kotaku also picked up on it and went as far as to photograph all the ads and put them online. To clarify, that’s about 42 ads in a 56-page newspaper.

Note to Microsoft: it is possible for you to spread your marketing you know – you don’t have to blow the budget all on one day!

UK Biometric Passports cracked

I hope that you’re not putting a lot of faith in the new biometric passports (my wife just got one) because any half-capable programmer with an RFID reader can obtain the details stored in them.

Within minutes of applying the three passports to the reader, the information from all of them has been copied and the holders’ images appear on the screen of Laurie’s laptop.

The information on the chip isn’t encrypted, but the conversation with it is, but the key is human-readable from the passport itself. Utter, utter madness.

Sony Ericsson W950i – better than the Nokia N91?

I like Symbian, and I can write code for it. This has mostly limited me to the Nokia brand of phones so far.

I am interested in getting a new phone, but am slightly out of touch on the latest devices.

Hence, my astonishment – the Sony Ericsson W950i runs Symbian.

Also, it appears to come bundled with mobile browser par excellence, Opera 8.

It also has a 4GB memory (can’t yet tell if this is hard drive based like the N91 or flash).

It is a very simple-looking handset, which definitely appeals to me after the years of the 6600 and now my 6630.

I guess the only bad sign to-date is this:

Available colours: Mystic purple

no camera. bugger.

Teaching website shambles

The Times Educational Supplement is the UK’s premier newspaper for those working in the world of primary, secondary and further education. They have a thriving online community, with an especially busy forum.

The TES used to be owned by News International, the Murdoch stable, but was sold at the end of last year and has since undergone “restructuring” (i.e. everyone who didn’t walk out was sacked) and so they’re pushed for cash.

They have just rebranded and redesigned and relaunched their website.

Dear God, what a mess. I really hope they didn’t pay very much for it, they need the money.

The new TES logo and some top-level navigation takes up the top 200 pixels (that’s 3 inches on my 19” screen at 1024×768). Incidentally, my wife, who is a teacher, has a laptop whose screen is 7 inches tall – that means almost 50% of any page she visits is instantly wasted.

It gets much, much worse and in fact it’s so painful that I’ll just skim the highlights and allow you to revel in the full horror yourselves

At 1024×768, none of the content of the forums is visible without scrolling because every single forum page has over a thousand pixels’ height worth of navigation (that includes the awful banner).

Despite the fact that there are actually over 50 forums, it’s impossible to tell, without scrolling down the page to tell which one you’re in at any given time (no clues in page title, headings, etc.).

At launch it didn’t work in Firefox or Opera (they use JavaScript for navigation between forums and used IE-specific code), and if you could get to a page, half of the CSS was not applied. Even people using the AOL browser (which is just embedded IE) were complaining of broken functionality.

In order to try and raise some cash, they have inserted larger ads at the right-hand side of every page. These are an un-resizable 300 pixels wide and only take up the first few hundred pixels of height for any page, meaning that if you’re in a forum thread, there’s a massive amount of white space going unused (not bad in itself, but given the wasted space elsewhere, a crying shame).

The app that powers it all is written in .Net and just browsing around I’ve already seen a couple of System.Object not found error messages (the default IIS ones in red text) and one OutOfMemoryException. That is so, so poor.

That just skims the UI of course, I’m actually too afraid to talk about the source code in case it comes to life and kills me in my sleep – something that over-complicated simply has to be almost sentient. It looks like it’s been produced via XSLT given some of the clues in the source, but surely someone who put that together wouldn’t then be capable of liberally littering the source with document.write and embedded CSS styles on element to which they’ve just assigned a class? Validating a forum page as HTML 4.01 Transitional gives you 190 errors. I can’t imagine daring to call myself a web developer and not running some basic validation on my output – I might even not make it perfectly valid, but over 100 isn’t even trying!

I would imagine that it didn’t help that they launched on a Friday, meaning a full weekend for the many, many users to post thousands of comments slating the new look, feel and functionality without any response from staff. This also meant that any problems were going to be experienced during the period of heaviest usage. Bad decision, TES-ers!

I really can’t imagine that anyone in charge of designing the new look and feel was a user of the old forum site, which, although it did have its own problems, were relatively minor – the site was straightforward and simple to use; it’s now just frustrating on a minute-by-minute basis which is more likely to drive people away than anything else. Just shrinking the heading and moving forum navigation to the mostly unused sidebar would be a massive gain to usability. Given that in the past few days Firefox functionality has been fixed, it’ll be interesting to see if anything else is done to improve the site.

Mobile Teletext

Barbie Horse Adventures sucks

I quite like teletext. The old, chunky teletext that is. It provides great, bite-sized chunks of information. To date the information hasn’t been republished on the internet – the positioning would be crazy; teletext can’t and shouldn’t compete with online resources.

However.

This week Teletext launched a mobile service (the website provides a number of mobile emulators so you can see . This makes perfect sense since the amount of text you can get on your mobile screen is quite similar to the amount displayed in the blocky old teletext (as you can see in the screenshot).

If you try and visit the mobile URL (such as this one), the server does some user-agent sniffing and redirects you to the HTML version. I tried a quick wget, specifically:

wget --user-agent='Nokia6630/1.0 (2.39.129) Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1' http://www.teletextmobile.co.uk/portal.aspx?pn=806&uid=1111702181083416953519691784019712536148&rnd=23112

and it does return a WML page, but it appears to be a portal page, and doesn’t maintain the page number I wanted to visit (806 in this case). When I visit the same page in my mobile browser it goes where expected.

I admit to being a wget noob, so where exactly am I going wrong? It’d be great to pull this content and push it into, say, a widget on my desktop, but not until I break the wget idiot-barrier.

OK, so installed the User Agent Switcher Firefox extension and configured it with the Nokia 6680 User-Agent (“Mozilla/4.0 (compatible; MSIE 5.0; Series80/2.0 Nokia9300/05.22 Profile/MIDP-2.0 Configuration/CLDC-1.1)”) and that worked a treat, giving me the content I wanted, but putting the same string into wget still gave me the Teletext Mobile splash page. Help welcome!