How to export Firefox's history to a text file
04 January, 2005Note that this doesn’t currently work in Firefox 1.5
Specifically, serialised as RDF/XML.
Early last year, Jamie Zawinski asked about how to extract information from Firefox’s history.dat file and ended up having to write a Perl script to do it.
He actually got further than most – most people who try to access the data not via Firefox itself just give up. This is because Firefox’s history is stored in a Mork file, which is an incredibly over-complicated and under-documented proprietary file format. You’ll learn more about it by reading the comments on Jamie’s post that you would by reading the docs.
Mork is used as a file format all over Mozilla, in both Firefox and Thunderbird in various points, but there are plans to replace it with SQLite in Mozilla 2.0 (see Unified Storage on the official Mozilla wiki) possibly with some interfaces made public via RDF (but I’m not 100% sure on this).
As it stands, there’s an open bug asking for history.dat to be easier to parse (which will probably be fixed by the move to SQLite rather than anything else) which actually has a very useful piece of Javascript attached which will convert the Mork history.dat into an RDF/XML history file, which I personally use as a bookmarklet from my toolbar: [export history.dat](<javascript:(function(){netscape.security.PrivilegeManager.enablePrivilege(%22UniversalXPConnect%22);var rdfhis =Components.classes[%22@mozilla.org/rdf/rdf-service;1%22].getService(Components.interfaces.nsIRDFService).GetDataSource(%22rdf:history%22);var serializer=Components.classes[%22@mozilla.org/rdf/xml-serializer;1%22] .createInstance(Components.interfaces.nsIRDFXMLSerializer);serializer.init(rdfhis);var outputStream = { data: %22%22, close : function(){}, flush : function(){},write: function (buffer,count){this.data += buffer; return count; }, writeFrom : function (stream,count){}, isNonBlocking: false};serializer.QueryInterface(Components.interfaces.nsIRDFXMLSource).Serialize(outputStream);outputStream.close();var summ=outputStream.data;var fp =Components.classes[%22@mozilla.org/filepicker;1%22].createInstance(Components.interfaces.nsIFilePicker);fp.init(window,%22%22,Components.interfaces.nsIFilePicker.modeSave);fp.defaultString = %22history.rdf.txt%22;fp.show();var filoutputStream = Components.classes[%22@mozilla.org/network/file-output-stream;1%22].createInstance( Components.interfaces.nsIFileOutputStream );filoutputStream.init( fp.file, 0x04 | 0x08, 420, 0 );filoutputStream.write(summ, summ.length);filoutputStream.close();})() >). Just drag it to your toolbar, load a local file in Firefox to ensure you’re in a trusted context and click it like you would any other bookmarklet. It’ll give you a security warning, which you can click “yes” to (if you trust my link – if not, you can get it from the bug linked to above, or look at the code yourself). It’ll take a few seconds to kick in (or longer, depending on the size of your history) but will eventually ask you to specify a location to save the file to. Give it a name, and you’re free to parse it with whatever you like (although preferably an actual RDF parser 🙂
Comments
Acting True
04 March, 2005 at 20:24
Pip
04 March, 2005 at 23:49
Anonymous
17 May, 2005 at 06:52
Pip
17 May, 2005 at 07:07
Anonymous
12 June, 2005 at 13:56
Pip
12 June, 2005 at 14:05
GJ
07 July, 2005 at 12:00
Pip
07 July, 2005 at 13:09
Jimmy Carlyle
01 September, 2005 at 13:28
Pip
01 September, 2005 at 13:38
wom
03 September, 2005 at 14:50
Pip
03 September, 2005 at 15:44
Anonymous
10 November, 2005 at 22:10
Anonymous
13 December, 2005 at 12:55
Anonymous
13 December, 2005 at 12:57
Pip
13 December, 2005 at 14:14
Anonymous
13 December, 2005 at 16:43
sari gordon
22 December, 2005 at 17:39
Pip
22 December, 2005 at 18:58
Anonymous
25 January, 2006 at 19:39
Anonymous
03 February, 2006 at 05:46
leo4must
06 February, 2006 at 20:32
Pip
07 February, 2006 at 12:32
Anonymous
15 February, 2006 at 01:30
Anonymous
08 March, 2006 at 22:41
Anonymous
17 March, 2006 at 18:14
Anonymous
09 June, 2006 at 08:28
Pip
09 June, 2006 at 09:52
Anonymous
09 June, 2006 at 11:02
Anonymous
27 June, 2006 at 07:21
Anonymous
28 June, 2006 at 03:55
Anonymous
05 July, 2006 at 13:53
Pip
05 July, 2006 at 16:22
Anonymous
24 July, 2006 at 01:14
Anonymous
16 November, 2006 at 19:28
Ben
09 March, 2007 at 22:22
Phil Wilson
10 March, 2007 at 00:11
Ed Mullen
03 June, 2007 at 20:03
Phil Wilson
03 June, 2007 at 21:59
Jim-BobH
22 July, 2007 at 17:53
Anonymous
05 December, 2007 at 12:23
dglp
06 December, 2007 at 23:31
Anonymous
09 December, 2007 at 06:06
Hi, I’d like more information about this — I went to the link and it wasn’t clear or wasn’t there.
If you followed the link to bugzilla (i.e. ) then yes, it’s all very unclear.
What exactly would you like more information about?
I tried to open the export history.dat-link but it doesn’t work.
i get the following error in the firefox javascript-console:
Fehler: uncaught exception: Einem Skript von “http://philwilson.org” wurden UniversalXPConnect Sonderrechte verweigert.
in english this stands for, that firefox denied the priviliges to run this script.
anyone has an idea how to get this one running?
greetings
michael
Hi Michael.
Type “about:config” in your Firefox address bar and type “xpconnect” into the text box marked “filter” and check that the preference “security.xpconnect.plugin.unrestricted” is marked as “true”. I *think* that’s the problem you’re having.
I’m also having trouble with export history.dat-link on this page. The Java Console reports, “Error: uncaught exception: A script from “http://philwilson.org” was denied UniversalXPConnect privileges.”
security.xpconnect.plugin.unrestricted is true in about:config.
Any other ideas?
Yes – you need to viewing a local HTML page in Firefox before you click the bookmarklet.
Just view any webpage, save it to disk (or create a new page on your hard drive), load it in Firefox then click the link. Loading the local page puts Firefox into a lower security mode which lets the bookmarklet access the files on the hard drive.
Is it not possible to export firefox history when the URL is remote (not a local file) ? If yes, then what do I have to do extra?
I would like to export the history irrespective of the URL (local, remote or about:blank)
Please do help.
Regards,
Gautam Jain
I don’t think it’s possible to do it unless you’re in the local context because of security constraints. Obviously being able to dump a file to hard drive, named as you want it, via JavaScript, would be a massive hole.
Hi
I’ve spent some of today looking for something like this and it worked great. Only trouble is how can I import the file onto another FireFox installation.
The user got a new PC is the reason.
Cheers
Jimmy
If you need to move the Firefox history between machines, you should just be able to copy the history.dat file from the profile directory:
* close Firefox on both computers
* find the profile directory on each computer (on Windows this will be something like C:Documents and SettingsUsernameApplication DataMozillaFirefoxProfilessdfkjsh4.default)
* copy the history.dat from the source profile directory to the destination profile directory
I believe that should work just fine.
Hi, for a university project I need a way to track user surf paths. The history.dat as RDF is a already a big help. But it is incomplete in the way that it does not recognize back button actions and does not distiguish beetween link following and typing in an URL in the adress field. Is there a way to enrich the history with that data?
Hi wom. I seem to remember that the Mozilla suite’s history.dat is far richer in data than Firefox’s and does at least some (if not all of) the things you’re asking about.
WOM: Try writing a firefox extension.
hello Phil,
a few months ago i told you that i have problems to use the javascript. because of much work i had no time to look at your blog earlier. your advice to solve my problem did work very nice and now i have a full functional xml file.
thx for this phil.
now i am working on a script to convert the xml file to a clear listing with the visited sites.
i will finish this script until end of the week probably and post a link to my website where everyone is allowed use it.
would be nice if some of you guys would test my script.
greetings from germany
the prior comment was written by me (Michael) 🙂
Excellent news, Michael, I’m glad I could be of use. I look forward to seeing your script!
Writing the script was more easier than i had imagined.
i made a little webpage for this script. would be interesting if it works with other history files than mine.
in the future i will optimize the script and perhaps add some enhancements.
you can find the script here: http://www.universe-hero.de/kortas_net/projects/ffhistory/index.php
for comments, reportings bugs or some special ideas feel free to write me an email: m(.at.)kortas(.dot.)net
greetings
Michael
Wow, I’m a total dunce when it comes to any of the abbreviations you’re talking about, but I managed to export a list of the sites I’ve visited today. My plan is to blog my travels in an attempt to illustrate my goal of reading the entire web (). If you could develop this into an extension, I would be a happy blogger! Thanks for your efforts!
wow, I’m glad I could help! This stuff doesn’t work in the latest version of Firefox, so I’ll have to do something better anyway!
Phil, please help! I accessed your file about moving Firefox’s history to text, and it worked great. The only problem is, the history.dat file shows a whole bunch of files that the history.rtf.txt does not (the history.rtf.txt only shows the last nine days). I want to find out the dates accessed for all of these files in history.dat. Is there any way for me to do that?
Also, if a user goes into Firefox and goes to the history, and deletes a specific site (just right-clicks and hits delete, like that), will that site still show up in history.dat, or in the text version?
Please help – I’m really desperate to figure this out. Thanks in advance! I’m at ubergottin @ herzeleid . net if you want to email me directly, or post here. (I tried to follow the link on the “contact you” part of this site, but it didn’t work for me.)
Lindsay
Phil, Thank you so much for doing this! It worked great – until I updated to ver 1.5.
Hope to see your new version soon. I wish I could find an old version of FireFox to install so I could continue to use your bookmarklet for now.
Sival
The export history.dat is no longer working with FF 1.5.0.1 on Windows XP Pro SP2. It worked fine earlier (FF versions on WinXP SP2), but now Javascript console says:
uncaught exception: [Exception… “Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIRDFXMLSource.Serialize]” nsresult: “0x80004005 (NS_ERROR_FAILURE)” location: “JS frame :: javascript:(function(){netscape.security.PrivilegeManager.enablePrivilege(%22UniversalXPConnect%22);var rdfhis =Componens.classes (… too much to type)
Thanks leo – I did say at the beginning of the post “Note that this doesn’t currently work in Firefox 1.5”.
When I get a chance, I really will look at an alternative that works in FF 1.5!
There is an extension, Enhanced History Manager (https://addons.mozilla.org/extensions/moreinfo.php?id=420&application=firefox), that, while it doesn’t do a direct export of the history file, allows you to select entries in the history and copy them to the clipboard. From there, you can paste into a text editor.
I’m using it in Firefox 1.5
The extended history manager only lists the URLs when you copy and paste — it does not include page titles or view date/times.
FYI There’s a utility at
http://www.sunturbine.com/dork/
that exports the HISTORY.DAT file to a tab delimited text file. Free/source code included.
Hi,
Does anybody know here if it’s possible to force history.dat outside the profile like the brower.bookmarks.file staff?
I try to have this for my network users whom can’t get the whole profile roaming bu I could’nt find any clue.
Thanks.
Well, you can force the entire Firefox profile to a network drive, for example, using something along the lines of:
browser.cache.disk.parent_directory = X:wherever
Other than that, no, I don’t think so.
I guess I have no choice but doing this 🙁
Thank for your help.
Thank you very much for your efforts, I was able to extract my foxfire history into a text file: it worked Great 🙂
By using the warning you gave about the file being overwritten without a notice, I renamed the file before running another dat file through and was happy to not lose anything..
Thanks again
rhage2
I know most people haven’t heard of Google, but Google Browser-Sync is out for FF. (Check in the labs section). It is supposed to synchronize Passwords, Bookmarks, & History. I think there is a bug in the initial release, as when I try and sync one of my PCs with a large history or bookmark file, it quits. BUT, it’s a step in the right direction.
I would like and many of my customers would like, a way to remove all sexual terms and links from the history file, without clearing the history.
People get so embarrassed when you type a l in the browser and get lesbian something.. most just have me turn off history all together
You and me both!
Hi, I’m just a blue collar guy who likes to turn his history off for the same reason as mentioned before. I don’t no how though. Can someone help me with this? Much appreciated
Can someone send a solution for a mac user. I have students playing in bad sites and do not want to accuse the wrong one. You can email an answer to mmorse@hotmail.com
http://www.sunturbine.com/dork/ now redirects to http://www.purescience.com/dork/, which is 404. The “Dork” conversion utility is now hard to find; a copy of it is on Bug 242438
Hi Ben, thanks very much for that useful update!
I’ve made DORK available at:
http://edmullen.net/utility/dork.zip
Ace, thanks for that Ed.
Thanks to Ed Mullen for enabling me to access Dork and to its author Keith Anderson.
Suggestions to Keith and other users:
1. Every time you run it it will OVERWRITE any history.txt file it previously created, so if you don’t want to lose your “history history” you had better move or rename the existing one.
(The above fact is in the Dork readme.txt but the following warning is not.)
2. If you run it with Firefox running you will get:
ERROR: Could not open file. Error 32
which apparently means the file is in use by another application (in this case, I closed Firefox and Dork seems to work perfectly).
IHTH
Jim
if you get the “uncaught exception: Einem Skript von “http://philwilson.org” wurden UniversalXPConnect Sonderrechte verweigert.” error, this is a security issue. Try adding the following lines to your user.js file located in your profile dir (create it, if it does not exist):
user_pref(“capability.principal.philwilson.id”, “http://philwilson.org”);
user_pref(“capability.principal.philwilson.granted”, “UniversalXPConnect”);
Save the file, restart Firefox. This should work. But be careful with granting rights to pages you don’t know and trust!
Interesting stuff here. I am looking for a way to graph my browsing history and haven’t had any joy finding explanations of history.dat. And here we are at Firefox 2.0 and Mork is still in place.
Are there any straightforward ways of converting the mork file into a format that will preserve the number of visits per site?
In the latter case you don’t have to shutdown Firefox but you can just create a copy of the history.dat file and use the copy with DORK.
Thanks for this wonderful tool.