CSS2.1 Test Suite: How to Contribute

CSS2.1 Conformance Test Suite: How to Contribute

Elika J. Etemad (fantasai) W3C CSS Working Group W3C Invited Experts
Agenda
- Introduce Zakim, #css-test, public-css-testsuite, csswg.org
- Introduce each other
- Introduce the CSS2.1 Test Suite template
- Review CSS Test Suite Guidelines
- Explain CSS2.1 Test Suite harness
- Go over contribution process
- Q&A throughout: feel free to ask questions anytime
Technological Support
- Zakim Conference Bridge
  - Mute: 61# on keypad or "Zakim, mute me" on IRC
  - Unmute: 60# on keypad or "Zakim, unmute me" on IRC
- IRC
  - Server: irc.w3.org
  - Port: 6665 (This is not the default.)
  - Channel: #css-test
- Testing Wiki: http://wiki.csswg.org/test
- Testing Mailing List: public-css-testsuite@w3.org

Test Suite Format

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: XHTML 1.1

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: XHTML and PNG
- Source format is XHTML 1.1 Strict
- Image format is PNG
- Scripts generate:
  - HTML 4.01 Strict / PNG
  - XHTML 1.1 Strict / PNG
  - XHTML 1.1 Strict / JPEG plus @page headers (for printers)
- We can add more export formats as needed.
- Validate your tests before submission

Test Suite Format: Title

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Title
- Title appears in generated index
- Used to evaluate coverage and avoid duplicate work
- Make it unique and descriptive
- Bad:
```
<title>CSS Test: Border Conflict Resolution</title>
```
  We have 100 tests on "Border Conflict Resolution".
- Good:
```
<title>CSS Test: Border Conflict Resolution (width) - hidden/double</title>
```

Test Suite Format: Credits

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Credits

Example:

  <link rel="author" title="Boris Zbarsky"
        href="mailto:bzbarsky@mit.edu"/>

Example:

  <link rel="author" title="Bert Bos"
        href="http://www.w3.org/People/Bos/"/>

Example:

  <link rel="author" title="Microsoft"
        href="http://microsoft.com/"/>

Test Suite Format: Spec Links

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
    <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Spec Links

Link to all relevant sections of the test suite
- Don't link to common incidental features, e.g. 'color' when using color: green in a selectors test
Use anchors from table of contents
Primary section listed first
Order most rare/specific to least rare/specific.

Example:

  <link rel="help" href="http://www.w3.org/TR/CSS21/text.html#alignment-prop"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/visudet.html#q7"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/visudet.html#line-height"/>
  <link rel="help" href="http://www.w3.org/TR/CSS21/colors.html#background-properties"/>

Test Suite Format: Content

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Content
- Best to use: <div>, <span>, <p>, <img>
  - Can assume div {display:block} and span {display:inline;}
  - Beware! <p> has margins by default! (Usually p {display: block: margin: 1em 0;})
- Use other elements only to test interaction with HTML features (or other namespaces' features).
- Test table features with HTML table elements and <div> + display: table-*.
- No style attributes unless testing them specifically.
- style attributes, non-UTF-8 encoding, external style sheets, scripting: check guidelines
- Indent nicely!
Test Suite Format: Support Files
- Standard images provided in support/ directory
  - 1x1 color swatches
  - 15x15 color swatches
  - 15x15 bordered color swatches
  - assorted rulers and red/green grids
  - a cat
  - a 4-part picture
- Test-specific files named after the test and put in support/ directory
Test Suite Format: Ahem Font
- Many tests rely on the Ahem font
- All glyphs 1em wide
- Baseline 0.2em above bottom / 0.8em below top
- x-height = 0.8em
- Great for testing inline box model
- Must flag Ahem requirement
```
<meta name="flags" value="ahem"/>
```

Test Suite Format: Ahem Font Glyphs

X	U+0058 (X)	A square exactly 1em in height and width. Most US-ASCII characters have this glyph.
p	U+0070 (p)	A rectangle exactly 0.2em high, 1em wide, and aligned so that its top is flush with the baseline.
É	U+00C9 (É)	A rectangle exactly 0.8em high, 1em wide, and aligned so that its bottom is flush with the baseline.
	U+0020 ( )	A transparent space exactly 1em high and wide.

Test Suite Format: Requirement Flags

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>CSS Test: DESCRIPTION OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR"
        href="mailto:EMAIL OR http://CONTACT_PAGE"/>
  <link rel="help" href="RELEVANT_SPEC_SECTION"/>
  <meta name="flags" value="REQUIREMENT_FLAGS"/>
  <style type="text/css"><![CDATA[
   CSS FOR TEST
  ]]></style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

Test Suite Format: Requirement Flags
- Use a meta tag to flag requirements and special processing:
```
<meta name="flags" value="ahem image"/>
```
- Common requirement flags:
  
  ahem
  Requires Ahem font
  dom
  Requires DOM scripting support
  image
  Requires bitmap graphics support
  interact
  Requires human interaction
  invalid
  Tests handling of invalid CSS
- Other flags described in guidelines
Test Suite Format: File Name Format
- test-topic-###.ext where ### is a 3-digit zero-padded number (e.g. 001)
- Filenames must be unique throughout the test suite.
- Keep them concise.
Test Suite Format: Question Check
- Any questions so far?
Writing a Good Test: Guidelines
- Obvious pass/fail result - easy to interpret result
- Quick to determine the result - a few seconds sufficient to determine result
- Self explanatory - no spec knowledge necessary to run tests
- Short - no scrolling even on short screens
- Valid Markup and CSS - (except failure tests)
- See techniques for tips and examples
- Please adhere to color code
Subversion
- Test suite is stored in test.csswg.org SVN, a publicly-accessible versioning system
- Browse the source over HTTP
- Use Subversion tools to check out the test suite and/or make a patch
- Anonymous read-only access available
- Write access available by request from fantasai
  - Each contributor gets their own directory in contributors/
  - incoming/ is your workspace
  - submitted/ is for submitting tests already in the above format
  - Be sure to rename and move files through Subversion so we can track history.
Contributing tests
- Write a test that conforms to the guidelines.
- Submit it to public-css-testsuite.
- Get it reviewed and approved.
- Ask someone with write access to check it in.
Write Access and Review Process
- Module ownership model
- fantasai is the module owner
- Anyone can review except
  - You can't review your own test
  - You can't review your company's tests unless you're an owner/peer
- Review checks for
  - Adherence to guidelines
  - Correctness
- Meritocracy: individuals, not corporations, must earn module peer status
That's it!
- Please subscribe to public-css-testsuite
- Ask questions! I'm there to help.
- This outline/slideshow is licensed under a Creative Commons Attribution 3.0 License .

Contributing to the CSS2.1 Test Suite

June 26, 2007

CSS2.1 Conformance Test Suite: How to Contribute

Agenda

Technological Support

Test Suite Format

Test Suite Format: XHTML 1.1

Test Suite Format: XHTML and PNG

Test Suite Format: Title

Test Suite Format: Title

Test Suite Format: Credits

Test Suite Format: Credits

Test Suite Format: Spec Links

Test Suite Format: Spec Links

Test Suite Format: Content

Test Suite Format: Content

Test Suite Format: Support Files

Test Suite Format: Ahem Font

Test Suite Format: Ahem Font Glyphs

Test Suite Format: Requirement Flags

Test Suite Format: Requirement Flags

Test Suite Format: File Name Format

Test Suite Format: Question Check

Writing a Good Test: Guidelines

Subversion

Contributing tests

Write Access and Review Process

That's it!