Automating web testing
10 November, 2006We’re just starting to ramp up the amount of testing we do on our web apps at work, mostly via the use of JUnit and some code coverage tools which are automated via CruiseControl.
I’m reasonably interested in getting some automated user acceptance testing going, and have been looking at Selenium, because it seems a great way to generate tests from the real UI, rather than, say, writing code for them in HttpUnit or JMeter (although I know that’s really a load testing tool, it could do the job at a push).
I’d like to be able to generate the tests using the Selenium IDE which is a Firefox extension. My problem with Selenium seems to be that it validates against real browsers, rather than against the HTTP requests and responses and so won’t fit in to our automated testing.
It’s possible to write Selenium tests as Java code (amongst others) but again this runs against the actual browser, and the whole point is that with the IDE you should’t actually have to be a developer to write acceptance tests.
Our CruiseControl machine is a headless server somewhere, so does this really mean that we can’t use Selenium?
I think that actually I’m just baffled that a tool might work this way, and not allow automation without a browser to hand. It rather looks as though we’re going to have to forget the idea of generating tests using the Selenium IDE and just code them in with HttpUnit (or Cactus or HtmlUnit or whatever)
Comments
Jason Huggins
14 November, 2006 at 07:14
Pip
14 November, 2006 at 10:35
Jason Porritt
17 November, 2006 at 15:32
Pip
17 November, 2006 at 16:29
Hi, Jason Huggins, the inventor of Selenium here. :-)… Though you see it as a liability, my co-workers amd I at ThoughtWorks created Selenium to test precisely what HTTP protocol drivers like HTTPUnit *can’t* test. Namely, applications that use JavaScript (aka “Ajax”) and for testing cross-browser compatibility. The very first test case for Selenium was for a fancy Ajaxy pop-up window that sometimes worked in in IE and other times worked in Mozilla. (I needed it to work in both browsers all the time!) I wrote Selenium so that I could certify on every build that my app worked in the actual browsers that my users used day-to-day. Protocol drivers like HTTPUnit don’t and can’t give me that level of confidence. On the other hand, there is a cost to driving the browser — it’s slow, because launching and driving a browser (and really, any UI– web or not– is slow). So, out of the box, Selenium’s not an effective load testing tool without lots of resources, but then again it wasn’t designed to be. 🙂 In those cases, HTTPUnit or JMeter are what you’re looking for.
Hi Jason, thanks for commenting!
Yes, you’re right of course, and I do really like Selenium, I just wish I could run it in some way on our server machine which doesn’t have any kind of windowing system installed, let alone a browser.
Just wishful thinking – really it goes to show how good Selenium is!
We are running Selenium RC on a Linux server with Firefox, but without using a “real” display. We use Xvfb to satisfy Firefox’s need for a display and it all works very well. A simple google search for “selenium xvfb” will turn up a few good articles to get you started.
I’d be happy to answer any questions (jasonmtu–at–yahoo.com).
Wow, thanks for that Jason. We’re think we’re running Solaris on our CI server so I’ll pass this info onto our sysadmins and see what they think of it. Definitely sounds like a great place to go though.