March 2009 – philwilson.org

Storing feedparser objects in couchdb

sudo apt-get install python-feedparser easy_install jsonpickle sudo apt-get install couchdb easy_install couchdb sudo couchdb

Open a new terminal

python import feedparser, jsonpickle from couchdb import Server s = Server('http://127.0.0.1:5984/') len(s) db = s.create('feeds') len(s) doc = feedparser.parse("http://feedparser.org/docs/examples/atom10.xml") doc['feed']['title'] len(doc.feed.links) pfeed = jsonpickle.encode(doc) db.create({'feed1' : pfeed})

outputs DOC_ID

cfeed = db['DOC_ID_HERE'] dfeed = jsonpickle.decode(cfeed['feed1']) dfeed['feed']['title'] len(dfeed.feed.links)

venus-ng

venus-ng is a fork of Venus which uses Newsgator to provide both the reading list and the feeds.

This means that venus-ng will, at a particular point in time, give you an accurate representation of your currently unread Newsgator feed entries. Here is the output from the newsgator.com web aggregator and venus-ng:

venus-ng does not mark feeds as read on the Newsgator server when in retrieves them, although that will likely get added when I have a test Newsgator account set up.

It is currently a fork because I’ve had to modify feedparser.py in a few ways which probably stop it working with other data sources:

I’ve changed the way it deals with passed-in urllib2 handlers
I’ve commented out the HTTP 401 response behaviour (since I’m passing it an HTTPBasicAuthHandler already)
It always passes through an additional X-NGAPIToken HTTP header containing a Newsgator API key

As far as I can tell, the handler refactoring should be fine, but the 401-handling and extra HTTP header seem like a deal-breakers.

I have no idea how to stop the 401 handler in _FeedURLHandler() conflicting with that in urllib2.HTTPBasicAuthHandler.

I suspect there is a good solution in subclassing urllib2.HTTPBasicAuthHandler to provide the additional Newsgator HTTP header but I’ve not worked out some of the details yet.

You can get the latest source via bzr get http://philwilson.org/code/venus-ng – there is a sample newsgator.ini file in the /examples directory, but it relies on you already having a Newsgator account and some feeds set up.

Once I’d traced through the Venus code to semi-understand it, this was quite straightforward to do (deal-breaking fork-causers aside) so were Google Reader to introduce an official API it would not take long to integrate.

Sending files to Ubuntu Intrepid over bluetooth

Out of the box you cannot send files over bluetooth to a computer running the latest stable version of Ubuntu, Intrepid Ibex, 8.10.

This is a bit of a problem for me since my main camera is my trusty Nokia N95. What you can do is send files to your phone, and browse its filesystem, just not receive. As you can imagine, there are some open bugs about this. The seemingly recommended solution of installing gnome-obex-server also installs apache2 and sendmail, which seems like a bit of overkill to me.

Enter Blueman. Just add these lines to your repositories config (you can follow these instructions)

deb http://ppa.launchpad.net/blueman/ppa/ubuntu intrepid main
deb-src http://ppa.launchpad.net/blueman/ppa/ubuntu intrepid main

and then sudo apt-get install blueman.

A quick click on System -> Preferences -> Bluetooth manager (it took a few minutes for the icon cache to update for me) and you have the new manager for your bluetooth devices. You don’t need to do anything special to be able to send files to Ubuntu now, just click “send” on your phone and you’ll be asked if you want to accept the file being sent. By default the files will be saved into your home directory. There’s more information on the site and in the forums. Have fun, kids!

Money is dull

I am in the process of buying my first house.

Unless you’ve already done it, you would not believe how tedious it is. You can read a lot about how stressful it is, but so far I’ve not found it very stressful, only irritating when I can’t get in touch with the bank. Other than that, it seems like an excuse for “financial institutions” (translate: black holes for your money) to send you loads of paperwork which you will have to read, check for naughty loopholes, sign, return and also send to someone else because why would any of the people you’re employing talk to one another?

Thankfully we’ve had the net to help us out. There are hundreds and probably thousands of sites which give advice on everything from mortgages, home inspections, how to negotiate price, how to use your estate agent (i.e. don’t, just use rightmove.co.uk – they actually have good information on pretty much everything), the difference between surveying and conveyance, moving and so on. It is all profoundly boring and yet you’re somehow expected to know it all for this one purchase, starting from a knowledge base of zero, which, in the scheme of things, despite total prices, isn’t really that expensive. sigh.

What it certainly is, is a cash cow. No wonder the banks et al. have been hoisted with the petard of their own greed, we’ve paid around £2,000 on our house move (you have to “buy” the right to get into hundreds of thousands of pounds of debt, and you have to “buy” the right to make sure the building is, y’know, actually *as described* and without fundamental problems) and the sellers could still pull out at any moment, if they wanted to.

Anyway, I cannot even imagine doing all of this without net connectivity. It would have taken *weeks* just to do what can now be done in hours. The preparation alone would have been a massive timesink, and I can totally see people getting even more robbed by banks by being sold their expensive add-on insurances which are mostly much more expensive than specialist sellers (which, of course, places like moneysupermarket will compare for you and moneysavingexpert will give advice on).

Hopefully you can gather from the tone of this that I think the whole charade is a big fat con, but more than that, just incredibly, incredibly boring.

Twitter on Ubuntu – use Gwibber

Gwibber is a native desktop Twitter client for Ubuntu. It will post updates to and receive updates from Twitter as well as a host of other services like Flickr, Facebook, Digg, can update your Pidgin status and pull in any other RSS or Atom feed you like.

It will sit in your notification area waiting until you get some messages which it will display as pop-ups in the corner of your screen, and won’t steal focus whilst it does so (take that, twitterfox!).

It is unobtrusive, attractive and easy to use. Theme templates are just HTML and CSS. I recommend giving it a go.

In Jaunty Jackalope it is in the Universe repositories (so sudo apt-get install gwibber will work) but for everyone else you can either follow the instructions on this page (which means you will get future updates) or go direct to the Gwibber PPA page, expand the section with your distro listed and grab the .deb file.

NewsGator + Venus?

I recently broke the graphics drivers on my Windows Vista installation, so re-partitioned and now run Ubuntu full-time at home.

On Windows I use FeedDemon as my full-time aggregator. It has a degree of speed and polish unmatched by any other web or desktop aggregator.

This means that all my feeds are automatically synced with newsgator.com – a web-based aggregator which is not fast and not particularly polished. Although it might be polished, I don’t know, it’s so slow that I tend to just give up (sync with Google Reader is coming).

FeedDemon has significantly raised the bar for any aggregator I use. Web-based tools no longer cut it, in particular when I have hundreds of feeds and, at times, thousands of unread items.

On Ubuntu the options for a native aggregator are Straw or Liferea. Both are currently undergoing rewrites. Liferea seems like the better option for me, and it has a plugin system which is appealing, but there’s no sync with any online tools.

NewsGator have an HTTP-basd API (PDF reference and sample code which requires a minor tweak to run) which is quite straightforward. It gives back data which can be consumed by the Universal Feed Parser. Venus uses the Universal Feed Parser in planet/spider.py after fetching data to create the cache which powers it.

This time last year I wrote a very very basic wxWidgets tool for browsing the Venus cache. A modification to planet/spider.py to use the NewsGator API would seem like an easy way forward, whilst gaining all the power of the Venus filters, plugins and existing XSLTs.

I might just have to try that.

Bazaar and Git

I’ve had cause to use both Bazaar and Git recently, which are very similar to one another and both of which I quite like.

Git seems slightly slicker, and gives me nicely colourised output, which I’m a sucker for. Then of course there’s GitHub, which is a very appealing way of easily publishing your code, forking and so on. All of these mean I’ll probably use it in the future.

The command set of Bazaar feels more natural to me, which I miss a lot in Git. Its parameters for seemingly simple tasks are comparatively obscure, and to this newbie, poorly explained by the command-line help, a consequence of which is that I have found the GitHub guides invaluable. Also, I always forget that in Ubuntu running sudo apt-get install git will not actually install git, but “GNU Interactive Tools, a file browser/viewer and process viewer/killer” (it’s sudo apt-get install git-core, for reference!).

There’s some interesting commentary on revision control systems as a whole, and the git command set in particular on Revision Control Systems suck and the series of posts he links to.