A Touch of Class


Creating the Future of CSS Testing (2011 Edition)

In an earlier article I explained the role CSS2.1 and how powerful and useful a CSS2.1 test suite could be for the Web.

In this article I'm going to explain the current situation in CSS testing at W3C, what my goals are for each aspect of the testing process, and what's missing to get us there. It's an update of my 2009 edition.

Submitting Tests

The goal here is to allow multiple people to submit and update tests in ways that are straightforward and easy for them, and to store the tests in a way that both works for W3C and is compatible with what our contributors want to do with the tests elsewhere.

We currently have:

What we're missing:

Publishing Tests

The goal here is to present the tests for each module on an official W3C server in the formats needed to run on all CSS implementations; to have the tests indexed so that it is easy for a developer to find the tests he is interested in running; and to have a coverage report that shows which parts of the spec are tested and which tests are missing.

What we have:

What we're missing:

Reviewing Tests

The goal here is to make it straightforward and easy for reviewers to review and comment on the tests they're interested in reviewing and to keep track of the required next steps for each test.

What we have:

What we're missing:

Ideally we want it to be:

What we're thinking:

Melinda Grant (HP), Eira Monstad, Peter Linss (HP / CSSWG Chair) and I spent some time working out the requirements and design for a system. One of the key ideas was to integrate with the Subversion repository so that changes there automatically show up in the test's record in the system. Another idea was to detect and cache the metadata in the tests to make it possible to search for tests on a particular topic.

We have a half-finished prototype of such a system, but it hasn't been getting much priority.

Collecting and Reporting Test Results

The goal for collecting is to make it easy and efficient to collect accurate test results for as many implementations as we can, both for the human-verified tests and the reftests.

The goal for reporting is to generate up-to-date reports from the test results that are useful and insightful for implementors, web designers, and W3C.

What we have:

What we're missing:

Get Involved

The CSS Working Group's wiki has more information on CSS testing and how you can get involved.