A Tale of Opposing Directions: Bidirectional Text in HTML and CSS

A Tale of Opposing Directions:

Bidirectional Text in HTML and CSS

Elika J. Etemad (fantasai) Mozilla Project W3C CSS Working Group

Introduction to Bidirectional Text

What is bidirectional text?

Mixing LTR/RTL text

Today we watched a presentation
1—————————————————————————————►
by Elika Etemad (الیکا اعتماد) on
2—————————————►  ◄——————————3  4►
the bidi algorithm.
5—————————————————►

Introduction to Bidirectional Text

What is bidirectional text?
- RTL with numbers
```
ال ۱۳۹۱ مان ایران نرفتم.
◄——————————————3 2——► ◄1
```
As soon as you expect to have any RTL, you must deal with bidi.

Introduction to Bidirectional Text

Text is stored in logical order, not visual order.

by Elika Etemad (الیکا اعتماد) on bidi.
1    2      3      5      4     6   7

Introduction to Bidirectional Text

Visual order depends on line-breaking.

by Elika Etemad (الیکا اعتماد) on bidi.
1    2      3      5      4     6   7

by Elika Etemad (الیکا
1    2      3      4
اعتماد) on bidi.
   5     6   7

The (Vastly Simplified) Unicode Bidirectional Algorithm
1. Classify all neutrals
2. Assign embedding levels
3. Reorder by number
Resolving Neutrals: Intrinsic Directionality

All Unicode characters are either
- strong left-to-right (L), e.g. Latin, 中文
- strong right-to-left (R), e.g. العربية‎, עִבְרִית
- neutral (N), e.g. spaces, punctuation
Note: Numbers are special. We'll pretend they're L for now.
Resolving Neutrals: Sandwich Rule

Surrounded neutrals take surrounding directionality.
```
L N N N L
    ⟱
L L L L L
```
Resolving Neutrals: Sandwich Rule Failures
- If the two sides conflict?
```
L N N N R
    ⟱
L ? ? ? R
```
- Neutrals at the start/end of a paragraph?
```
.This is my sentence
```
```
این جمله است‎.
```
Resolving Neutrals: Base Direction

Each paragraph has a base direction.
- In HTML: per block element, by the dir attribute.
- In plaintext: per paragraph, by first strong character.
Use the base direction for initial/final/conflicted neutrals.
Embedding Levels

L ⇒ evens R ⇒ odds
1. Assign all characters to base level.
2. Increase by one as required by directionality.

Embedding Levels: LTR Base Direction

Fox پرید over Dog.
LLLNRRRRNLLLLNLLLN
LLLLRRRRLLLLLLLLLL
000011110000000000

abc DEF ghij
000 111 0000

Embedding Levels: RTL Base Direction

رروباه از رو Dog پرید.
NRRRRNLLLNRRNRRNRRRRRR
RRRRRRLLLRRRRRRRRRRRRR
1111112221111111111111

ABC def GHIJ
111 222 1111

Reordering

For each level run:
1. Reverse the order of chars.
2. Reduce their level by one.

Reordering: LTR Base Direction

abc DEF ghij
000 111 0000

abc FED ghij
000 000 0000

Reordering: RTL Base Direction

ABC def GHIJ
111 222 1111

ABC fed GHIJ
111 111 1111

JIHG def CBA
0000 000 000

Embeddings: Why Embed?

The fox پرید over the lazy dog.
1—————► ◄——2 3————————————————►
——————————————————————————————►

The fox پرید over the تنبل dog.
1—————► ◄——2 3——————► ◄——2 1——►
——————————————————————————————►

Embeddings: Why Embed?

الیکا ”...“ نوشت.
◄———3  2—►  ◄———1
◄———————————————

Fox پرید over Dog.
1—► ◄——2 3———————►
—————————————————►

Embeddings: Why Embed?

الیکا ”Fox پرید over Dog“ نوشت.
◄———5  4——————► ◄——3 2———►  ◄———1

الیکا ”Fox پرید over Dog“ نوشت.
◄———3  2———————————————►  ◄———1
◄———5  2—► ◄——3 4——————►  ◄———1

Embeddings: Mechanics

Embeddings are markup that creates a "paragraph" within a paragraph.

Increase embedding level by at least 1 wrt surrounding text.
Set base direction explicitly within that run.

الیکا ”Fox پرید over Dog!“ نوشت.
◄—1—►  ◄——————2—————————►  ◄—1—►
11111  222 3333 222222222  11111
ZYXWV  stu OPQR nmlkjihgf  EDCBA

Embeddings: Reordering

ABCDE »fghijklmn OPQR stu« VWXYZ
◄—1—►  ◄——————2—————————►  ◄—1—►
11111  222222222 3333 222  11111

ABCDE fghijklmn RQPO stu VWXYZ
11111 222222222 2222 222 11111

ABCDE uts OPQR nmlkjihgf VWXYZ
11111 111 1111 111111111 11111

ZYXWV stu OPQR nmlkjihgf EDCBA
00000 000 0000 000000000 00000

Intermission

End of Part I
Bidi in HTML 4: Controls

Global dir attribute
- dir="ltr" | dir="rtl"
- Sets base direction on blocks (, <div>, <html>)
- Creates an embedding on inlines (<a>, , )
Bidi in HTML 4: Usage

Global dir attribute
- Directionality inherits (embeddingness does not)
- Set on ancestor to set base direction of descendants
- Default is ltr
Bidi in CSS 2: Effects

Directionality affects:
- Default text alignment
- Ordering of table columns
- Alignment of overflowing blocks
- Ordering of text
Bidi in CSS 2: Do Not Use
- Represented in CSS model by direction and unicode-bidi properties
- DO NOT USE: auto-set by HTML style sheet based on markup
- Use markup!
Bidi Basics: Document Base Direction
```
<html dir=rtl>
```

Bidi Basics: Embedding Blocks

<blockquote dir=rtl>...</blockquote>

<div dir=rtl>...</div>

Bidi Basics: Embedding Inline
```
... <q dir=rtl>...</q> ...
```
Some Problems... and How to Fix Them

Most problems in bidi are due to either
- Insufficient combination ⇒ use an embedding
- Insufficient separation ⇒ use an &rlm;/&lrm;
Creating Separation: Neutrals
```
L N N N L
    ⟱
L L L L L
```
strong right-to-left (R), e.g. العربية‎, עִבְרִית
strong right-to-left (R), e.g. العربية, עִבְרִית
strong right-to-left (R), e.g. العربية&lrm;, עִבְרִית
Embedding Woes
- Adjacent embeddings glom together:
```
 ...
ABC
DEF ...
```
 FED CBA not CBA FED !
- Numbers are weird.
 - They can, in certain circumstances, jump two levels.
 - Therefore, they can glom onto or even flip into an embedding
Short story: embeddings suck.
Dealing with User Input: HTML4 Edition
- If you know the user string's directionality, embed it
  - always
  - only when different from surrounding text
- If two or more user strings appear in a bidi paragraph, separate them.
```
title — author:
AUTHOR — TITLE:
```
- Beware numbers in your UI!
  - Embed number-punctuation sequences, e.g. serial numbers, MAC address, phone numbers, product numbers, etc.
  - Separate numbers from any user input.
- Strip trailing spaces in an element
Auto-Detection
- various heuristics, e.g. %RTL above threshold
- first-strong
  - Simple
  - Predictable
  - Controllable
  - Compatible
No heuristic is always right! It's just a heuristic.
Intermission

End of Part II
HTML5: Isolates
- Fixes suckiness of embeddings:
 - Absolutely no leaking
 - Treated as neutral
- Tree-structure-friendly: just wrap it and go
- Eliminates most (all?) need for RLM/RLM tricks
```
e.g. <bdi dir=rtl>العربية</bdi>,
 <bdi dir=rtl>עִבְרִית</bdi>
```
HTML5: Auto-Detection

dir=auto provides browser-side first-strong auto-detection
- Great for phrases, single paragraphs
- Triggers isolation
- <pre> and <textarea> are special
 - Per-paragraph checking
 - Automatic alignment per plaintext paragraph
 - Compatible with plaintext rules
CSS3: Direction-based Selectors
```
:dir(rtl) { ... CSS for rtl elements ... }
:dir(ltr) { ... CSS for ltr elements ... }
```
- Triggers on HTML dir attribute
- Processed after dir=auto resolution
CSS3: Matching Alignment
```
text-align: start | end | match-parent
```
Acknowledgements and References
- Unicode Standard Annex #9
- Additional Requirements for Bidi in HTML
- Bidi Tutorial
- www-international

A Tale of Opposing Directions

SF Globalization • 14 January 2013

A Tale of Opposing Directions:

Bidirectional Text in HTML and CSS

Introduction to Bidirectional Text

Introduction to Bidirectional Text

Introduction to Bidirectional Text

Introduction to Bidirectional Text

The (Vastly Simplified) Unicode Bidirectional Algorithm

Resolving Neutrals: Intrinsic Directionality

Resolving Neutrals: Sandwich Rule

Resolving Neutrals: Sandwich Rule Failures

Resolving Neutrals: Base Direction

Embedding Levels

Embedding Levels: LTR Base Direction

Embedding Levels: RTL Base Direction

Reordering

Reordering: LTR Base Direction

Reordering: RTL Base Direction

Embeddings: Why Embed?

Embeddings: Why Embed?

Embeddings: Why Embed?

Embeddings: Mechanics

Embeddings: Reordering

Intermission

Bidi in HTML 4: Controls

Global dir attribute

Bidi in HTML 4: Usage

Global dir attribute

Bidi in CSS 2: Effects

Bidi in CSS 2: Do Not Use

Bidi Basics: Document Base Direction

Bidi Basics: Embedding Blocks

Bidi Basics: Embedding Inline

Some Problems... and How to Fix Them

Creating Separation: Neutrals

Embedding Woes

Dealing with User Input: HTML4 Edition

Auto-Detection

Intermission

HTML5: Isolates

HTML5: Auto-Detection

CSS3: Direction-based Selectors

CSS3: Matching Alignment

Acknowledgements and References

Global `dir` attribute

Global `dir` attribute