Downloading from the BBC streaming iPlayer is hard

(this was mostly written on Friday 14th December, although no-one else seems to have stepped up to the plate yet)

The new Flash-based BBC iPlayer streams its content over Adobe’s proprietary RTMP, making it very hard to download the stream and save it for later.

There are some native, commercial, applications to do this on Windows like Replay Media Catcher but as far as I can tell, nothing on Linux. There has been a first-pass MythTV integration but this seems to use an embedded browser window.

snagged iplayer content

Red5 is a Java-based open-source Flash server which also comes bundled with some code for a sample client (download the tarball and then look in src/org/red5/samples/client). It might be possible to use this to download the stream; I haven’t yet tested. There also appear to be some code samples in Ruby and Python lying around, but these look very incomplete.

The URL for the stream is along the lines of this:

rtmp://217.243.192.52:1935/ondemand?_fcs_vhost=cp41752.edgefcs.net&auth=SECRET_KEY&aifp=v001&slist=STREAM_NAME

where both SECRET_KEY and STREAM_NAME come from a file of the name http://www.bbc.co.uk/mediaselector/3/stream/check/iplayer?pid=PID where PID is a BBC Programme ID such as b008h3zq as found on http://www.bbc.co.uk/programmes/ – the base URL for this address is found in the config.xml file linked to from every iPlayer page: http://www.bbc.co.uk/iplayer/emp/xml/config.xml

So this looks hackable, with the most promising chance probably being with Java at the moment.

15 thoughts on “Downloading from the BBC streaming iPlayer is hard

  1. OK I’m tempted to take the bait and have a look at doing this. I’d already got as far as working out the URL but was then kind of stumped for a way of downloading the rtmp stream (without using something like Orbit Downloader). Often you can just change the rtmp:// to http:// and the stream will work using the RTMPT protocol, but unfortunately this doesn’t seem to work.

    Hopefully Red5 will provide the missing link, although from a quick glance at the source code I’m not entirely sure how to do it.

    Java definitely seems the way to go though, as it would allow a signed applet to be added to the pages via greasemonkey which could provide a download button for the flv files.

    Now I just have to work out how Red5 works!

    Mark

  2. Hey there. I’m somewhat affiliated with RTMPy and although you are correct in that RTMP implemenation is incomplete yet, there is active work being done on it. So far, we have alpha support for AMF through [PyAMF](http://pyamf.org/) which is being used as the basis for the RTMP implementation.

    If you would be interesting in helping out with a client for downloading from BBC in the near future, that would be an intersting test for RTMP client functionality. Please stop by our mailing list if you feel so inclined :)

  3. Is there any kind of forum or wiki where people are working on this? I see MythTV working on this, and I was linked from a thread on the XBox Media Center forums: http://www.xboxmediacenter.com/forum/showthread.php?t=27063

    I’ve gotten as far as decompiling the flash player to see what it’s doing and deciding that it’s going to take considerably longer than a dull Saturday afternoon to hack something together.

    How are you getting the URLs? I’ve found the stream traffic was encrypted.

  4. I have been trying to do this as well, and got about as far as you did.

    A few extra bits I found out:

    The SECRET_KEY changes every time you request the mediaselector file.

    The reply when you request the mediaselector file includes the header ‘Vary: Cookie’, which indicates to caching proxies that they should get a fresh copy every time the cookies change.

  5. Sorry but none of the versions of the downloader work for me, on any platform. Resulting file is 0 bytes in size, finds MP4 file but dfails to download it. Given up trying to make it work.

  6. from a very quick casual look my understanding was that the iplayer-downloader only handled the RealAudio over RTSP streams (using mplayer), not the MP3 over RTMP streams?

  7. Yes this is true, now there is MP3 support
    see this clipping from the Authors Website:

    Paul Battley Wrote at 2008-09-23 21:06 UTC (6 days ago)
    using Firefox 3.0.1 on Linux:

    I’ve just uploaded a new version with proper radio download support—which just meant detecting radio programmes and changing the extension to mp3!

    That’s version 0.1.12 of the library and command-line tools, and version 1.8 of the GUI.

    Hurra !

  8. I had this problem today. Since i use mplayer in firefox, i just typed “ps lax | grep mplayer” in a shell and got this:

    0 0 17665 17464 20 0 174336 8488 – Ss ? 0:00 mplayer -wid 0x220080a -vf scale=640:-3 -osdlevel 1 -nojoystick -noconsolecontrols -cookies -softvol -slave -user-agent NSPlayer -nomouseinput -cache 512 rtsp://rmv8.bbc.net.uk:554/radio4fmcoyopa/radio_4_fm_-_monday_2100.ra?timestamp=1237240768&pid=b00j3vn9&BBC-UID=64998765964d0de8f7947d6ad1740198c717267800d0205bdc6dc842b46f86d8&SSO2-UID=

    Then just take the URL and download it with mplayer. Here is the result of the nice radio program
    http://www.bbc.co.uk/iplayer/episode/b00j3w8k/Sound_Architecture_The_Spaces_That_Speak/
    i got today:

    http://andersson.is-a-geek.org/aac/

    Video seems to be harder since you probably need to find a nice proxy in the UK.
    If you succed – put the result on http://thepiratebay.org

    Greetings from Uppsala, Sweden

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>