Downloading from the BBC streaming iPlayer is hard
December 17th, 2007 by 
Phil
(this was mostly written on Friday 14th December, although no-one else seems to have stepped up to the plate yet)
The new Flash-based BBC iPlayer streams its content over Adobe’s proprietary RTMP, making it very hard to download the stream and save it for later.
There are some native, commercial, applications to do this on Windows like Replay Media Catcher but as far as I can tell, nothing on Linux. There has been a first-pass MythTV integration but this seems to use an embedded browser window.
Red5 is a Java-based open-source Flash server which also comes bundled with some code for a sample client (download the tarball and then look in src/org/red5/samples/client). It might be possible to use this to download the stream; I haven’t yet tested. There also appear to be some code samples in Ruby and Python lying around, but these look very incomplete.
The URL for the stream is along the lines of this:
rtmp://217.243.192.52:1935/ondemand?_fcs_vhost=cp41752.edgefcs.net&auth=SECRET_KEY&aifp=v001&slist=STREAM_NAME
where both SECRET_KEY and STREAM_NAME come from a file of the name http://www.bbc.co.uk/mediaselector/3/stream/check/iplayer?pid=PID where PID is a BBC Programme ID such as b008h3zq as found on http://www.bbc.co.uk/programmes/ - the base URL for this address is found in the config.xml file linked to from every iPlayer page: http://www.bbc.co.uk/iplayer/emp/xml/config.xml
So this looks hackable, with the most promising chance probably being with Java at the moment.
OK I’m tempted to take the bait and have a look at doing this. I’d already got as far as working out the URL but was then kind of stumped for a way of downloading the rtmp stream (without using something like Orbit Downloader). Often you can just change the rtmp:// to http:// and the stream will work using the RTMPT protocol, but unfortunately this doesn’t seem to work.
Hopefully Red5 will provide the missing link, although from a quick glance at the source code I’m not entirely sure how to do it.
Java definitely seems the way to go though, as it would allow a signed applet to be added to the pages via greasemonkey which could provide a download button for the flv files.
Now I just have to work out how Red5 works!
Mark
Posted by Mark on December 18th, 2007 at 12:11 pmHey there. I’m somewhat affiliated with RTMPy and although you are correct in that RTMP implemenation is incomplete yet, there is active work being done on it. So far, we have alpha support for AMF through [PyAMF](http://pyamf.org/) which is being used as the basis for the RTMP implementation.
If you would be interesting in helping out with a client for downloading from BBC in the near future, that would be an intersting test for RTMP client functionality. Please stop by our mailing list if you feel so inclined :)
Posted by Arnar Birgisson on December 28th, 2007 at 3:10 pmIs there any kind of forum or wiki where people are working on this? I see MythTV working on this, and I was linked from a thread on the XBox Media Center forums: http://www.xboxmediacenter.com/forum/showthread.php?t=27063
I’ve gotten as far as decompiling the flash player to see what it’s doing and deciding that it’s going to take considerably longer than a dull Saturday afternoon to hack something together.
How are you getting the URLs? I’ve found the stream traffic was encrypted.
Posted by Iain on December 28th, 2007 at 10:57 pmIf I find the time, I’m going to try to do some work on an AppleTV player, so far it looks like these links may help.
http://osdir.com/ml/gnu.gnash.devel2/2006-11/msg00278.html
http://osflash.org/documentation/rtmp
But that still won’t cross the hurdle of actually getting the url in the first place.
Posted by Gareth on December 31st, 2007 at 7:28 pmCheck the gnash-dev mailing list archives for recent progress on RTMP support. Join the list to help contribute to making this work with the BBC web site and streams.
Posted by Gnash Developer on January 2nd, 2008 at 4:40 amThere’s another (free) app for capturing RTMP Flash streams called Orbit Downloader - http://www.orbitdownloader.com
Alas, it is also only available for Windows, but it does the job marvellously and you can’t fault the price.
Posted by Hubert on January 21st, 2008 at 2:17 pm