New automated regression test framework
Robert O'Callahan writes in bug 230697:
I have been developing a new regression test framework, primarily for layout, but since it's end-to-end it also tests HTTP, Gfx/Widget, parser, content, views, etc. It's inspired by Hixie's test engine for Opera. It's fairly straightforward and requires only very minor changes to the Mozilla codebase. Basically we set up a special local Web server that feeds Mozilla a XUL app to drive the tests and a set of testcases. The XUL app loads each testcase and then signals the server (by requesting a magic URL) that the testcase is loaded. The server then takes a screenshot of the Mozilla window and replies to the URL request, causing Mozilla to move to the next testcase. The result is a set of PNGs, one per testcase. Regression testing consists of building a baseline set of PNGs and then rerunning the tests with a modified Mozilla, and comparing the PNGs.