A Touch of Class

fantasai
2009-11/12

Creating the Future of CSS Testing (2009 Edition)

This article has been updated for 2011

In the previous article I explained the role CSS2.1 and how powerful and useful a CSS2.1 test suite could be for the Web.

In this article I'm going to explain the current situation in CSS testing at W3C, what my goals are for each aspect of the testing process, and what's missing to get us there.

Submitting Tests

The goal here is to allow multiple people to submit and update tests in ways that are straightforward and easy for them, and to store the tests in a way that both works for W3C and is compatible with what our contributors want to do with the tests elsewhere.

We currently have:

What we're missing:

Publishing Tests

The goal here is to present the tests for each module on an official W3C server in the formats needed to run on all CSS implementations; to have the tests indexed so that it is easy for a developer to find the tests he is interested in running; and to have a coverage report that shows which parts of the spec are tested and which tests are missing.

What we have:

What we're missing:

Reviewing Tests

The goal here is to make it straightforward and easy for reviewers to review and comment on the tests they're interested in reviewing and to keep track of the required next steps for each test.

What we have:

What we're missing:

Ideally we want it to be:

What we're thinking:

Melinda Grant (HP), Eira Monstad, Peter Linss (HP / CSSWG Chair) and I spent some time last year working out the requirements and design for a system. One of the key ideas was to integrate with the Subversion repository so that changes there automatically show up in the test's record in the system. Another idea was to detect and cache the metadata in the tests to make it possible to search for tests on a particular topic.

I started prototyping a system in Python+Django to address these concerns during the W3C test hackathon last September, but did not manage to complete the Subversion integration in that timeframe. If working on the design and implementation (front-end or back-end) of a system like this interests you, drop me a line. :)

Collecting and Reporting Test Results

The goal for collecting is to make it easy and efficient to collect accurate test results for as many implementations as we can, both for the human-verified tests and the reftests.

The goal for reporting is to generate up-to-date reports from the test results that are useful and insightful for implementors, web designers, and W3C.

What we have:

What we're missing:

Get Involved

The CSS Working Group's wiki has more information on CSS testing and how you can get involved.