CSS2.1 Test Suite: How to Contribute

CSS2.1 Conformance Test Suite: How to Contribute

Elika J. Etemad (fantasai) W3C CSS Working Group W3C Invited Experts
Agenda
- Introduce Zakim, #css-test, public-css-testsuite, csswg.inkedblade.net
- Introduce each other
- Introduce the CSS2.1 Test Suite template
- Review CSS Test Suite Guidelines
- Explain CSS2.1 Test Suite harness
- Go over contribution process
- Q&A throughout: feel free to ask questions anytime
Technological Support
- Zakim Conference Bridge
  - Mute: 61# on keypad or "Zakim, mute me" on IRC
  - Unmute: 60# on keypad or "Zakim, unmute me" on IRC
- IRC
  - Server: irc.w3.org
  - Port: 6665 (This is not the default.)
  - Channel: #css-test
- Testing Wiki: http://csswg.inkedblade.net/test
- Testing Mailing List: public-css-testsuite@w3.org

Test Suite Format

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: XHTML 1.1

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: XHTML 1.1 and PNG
- Source format is XHTML 1.1 Strict
- Image format is PNG
- Scripts generate:
  - HTML 4.01 Strict / PNG
  - XHTML 1.1 Strict / PNG
  - XHTML 1.1 Strict / JPEG plus @page headers (for printers)
- We can add more export formats as needed.
- Validate your tests before submission

Test Suite Format: Title

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Title

Title appears in generated index
Make it unique and descriptive

Bad:

<title>CSS 2.1 Test Suite: Border Conflict Resolution</title>

We have 100 tests on "Border Conflict Resolution".

Good:

<title>CSS 2.1 Test Suite: Border Conflict Resolution (width) - hidden/double</title>

Test Suite Format: Credits

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Credits

Example:

  <link rel="author" title="Boris Zbarsky"
        href="mailto:bzbarsky@mit.edu"/>

Example:

  <link rel="author" title="Bert Bos"
        href="http://www.w3.org/People/Bos/"/>

Example:

  <link rel="author" title="Microsoft"
        href="http://microsoft.com/"/>

Test Suite Format: Spec Links

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Spec Links

Link to all relevant sections of the test suite
- Don't link to common incidental features, e.g. 'color' when using color: green in a selectors test
Use anchors from table of contents
Primary section listed first
Order most rare/specific to least rare/specific.

Example:

  <link rel="help" href="http://www.w3.org/TR/CSS21/text.html#alignment-prop"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/visudet.html#q7"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/visudet.html#line-height"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/colors.html#background-properties"/>

Test Suite Format: Content

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
          "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS 2.1 Test Suite: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Content
- Best to use: <div>, <span>, <p>, <img>
  - Can assume div {display:block} and span {display:inline;}
  - Beware! <p> has margins by default! (Usually p {display: block: margin: 1em 0;})
- Use other elements only to test interaction with HTML features (or other namespaces' features).
- Test table features with HTML table elements and <div> + display: table-*.
- No style attributes unless testing them specifically.
- style attributes, non-UTF-8 encoding, external style sheets, scripting: check guidelines
- Indent nicely!
Test Suite Format: Support Files
- Standard images provided in support/ directory
  - 1x1 color swatches
  - 15x15 color swatches
  - 15x15 bordered color swatches
  - assorted rulers and red/green grids
  - a cat
  - a 4-part picture
- Test-specific files named after the test and put in support/ directory
Test Suite Format: Ahem Font
- Many tests rely on the Ahem font
- All glyphs 1em wide
- Baseline 0.2em above bottom / 0.8em below top
- x-height = 0.8em
- Great for testing inline box model
- Must flag Ahem requirement

Test Suite Format: Ahem Font Glyphs

X	U+0058 (X)	A square exactly 1em in height and width. Most US-ASCII characters have this glyph.
p	U+0070 (p)	A rectangle exactly 0.2em high, 1em wide, and aligned so that its top is flush with the baseline.
É	U+00C9 (É)	A rectangle exactly 0.8em high, 1em wide, and aligned so that its bottom is flush with the baseline.
	U+0020 ( )	A transparent space exactly 1em high and wide.

Test Suite Format: Flagging Requirements
- Existing requirement flags:
  (a) Ahem font (g) bitmap graphics (h) session history (i) interaction (m) MathML (n) XML Namespaces (o) DOM (v) SVG
- Upcoming flags:
  - Continuous (scrolling) media
  - Paged media
- Add to end of filename.
- Upcoming change: move flags into a <meta> element.
Test Suite Format: File Name Format
- Current format: tAABBCC-test-topic-##-k-req.ext
- Examples: t040105-import-01-b.xht, t0402-syntax-06-f.xht, t170602-bdr-conflct-w-54-d.xht
- Proposed new format: test-topic-###.ext (source) / test-topic-###-flags.ext (final)
  - Drop tAABBCC: test section is already in <link>
  - Drop k (kind of test) marker
  - Add failure (f) kind of test marker to req flags
Test Suite Format: Question Check
- Any questions so far?
Writing a Good Test: Guidelines
- Obvious pass/fail result - easy to interpret result
- Quick to determine the result - a few seconds sufficient to determine result
- Self explanatory - no spec knowledge necessary to run tests
- Short - no scrolling even on short screens
- Valid Markup and CSS - (except failure tests)
- See techniques for tips and examples
- Please adhere to color code
CVS
- Test suite is stored in dev.w3.org CVS, a publicly-accessible versioning system
- Browse the source over HTTP
- Use command-line CVS to check out the test suite and/or make a patch
- Anonymous read-only access available
- Test suite owners and peers have write access
Contributing tests
- Write a test that conforms to the guidelines.
- Submit it to public-css-testsuite.
- Get it reviewed and approved.
- Ask someone with write access to check it in.
Write Access and Review Process
- Module ownership model
- fantasai is the module owner
- Anyone can review except
  - You can't review your own test
  - You can't review your company's tests unless you're an owner/peer
- Review checks for
  - Adherence to guidelines (required)
  - Correctness (recommended)
- Meritocracy: individuals, not corporations, must earn module peer status
That's it!
- Please subscribe to public-css-testsuite
- Ask questions! I'm there to help.
- This outline/slideshow is licensed under a Creative Commons Attribution 3.0 License .

Contributing to the CSS2.1 Test Suite

June 26, 2007

CSS2.1 Conformance Test Suite: How to Contribute

Agenda

Technological Support

Test Suite Format

Test Suite Format: XHTML 1.1

Test Suite Format: XHTML 1.1 and PNG

Test Suite Format: Title

Test Suite Format: Title

Test Suite Format: Credits

Test Suite Format: Credits

Test Suite Format: Spec Links

Test Suite Format: Spec Links

Test Suite Format: Content

Test Suite Format: Content

Test Suite Format: Support Files

Test Suite Format: Ahem Font

Test Suite Format: Ahem Font Glyphs

Test Suite Format: Flagging Requirements

Test Suite Format: File Name Format

Test Suite Format: Question Check

Writing a Good Test: Guidelines

CVS

Contributing tests

Write Access and Review Process

That's it!