Emoji width: the motorway was one cell all along
a585a74 listed 🛣 (U+1F6E3) in the wide set on the reading that the font drew a two-cell colour glyph where both authorities measured one. It had the direction backwards, and the entry is what JP is looking at now: the Trophy case's Longest-distance heading reserves a cell the terminal never draws, so its row is 24 characters where its neighbours are 25 and the right border lands a column in. The arithmetic is unambiguous — that heading emits one character fewer than the rows around it while measuring the same — and the control was one board down the whole time. 🕸 (U+1F578) is the identical Unicode class, used bare, never listed, and framed flush throughout. The rule the mistake missed: outside Emoji_Presentation, a *bare* codepoint has no variation selector asking for emoji presentation, so the font draws a one-cell text glyph — which is exactly what Rich and wcwidth said. Both authorities already agreeing is not a bug to correct. The wide set is for the one thing it was built for, a VS16 sequence the narrow verdict gets wrong (🛩), and 🛣 needed no entry at all. Pinned by a test naming both glyphs, and the skill that would have recommended the same mistake an hour ago now argues against it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
09bae4cf43
commit
574f1303a7
4 files changed
+80
-57
No files matched your search
@@ -23,9 +23,9 @@ An emoji in a row breaks its layout, in either of two ways:
|
||||
the right of the glyph — the terminal draws it in *fewer* cells than the authorities
|
||||
reserve. → the narrow allowlist (`_DEFAULT_NARROW_LONE`).
|
||||
- **The row overruns**, pushing what follows a column right — the terminal draws it in
|
||||
*more* cells than the authorities reserve. → the wide set (`_DEFAULT_WIDE_BASE`). This is
|
||||
the case for an emoji outside Emoji_Presentation, which wcwidth and Rich both call one
|
||||
cell while the font paints a two-cell colour glyph.
|
||||
*more* cells than the authorities reserve. → the wide set (`_DEFAULT_WIDE_BASE`). In
|
||||
practice this only happens to a **VS16 sequence** the narrow verdict got wrong; a *bare*
|
||||
codepoint both authorities call one cell is measured correctly and must be left alone.
|
||||
|
||||
Step 1 tells you which; it also tells you when the glyph is **already listed**, which is
|
||||
the commonest answer to "please add this one".
|
||||
@@ -52,11 +52,14 @@ the commonest answer to "please add this one".
|
||||
`_DEFAULT_NARROW_LONE` in [emoji_width.py](../../..//meshterm/ui/tui/emoji_width.py):
|
||||
`_DEFAULT_NARROW_LONE = "👋👍"`. (For a throwaway session test instead of a code
|
||||
change, set `MESHTERM_NARROW_EMOJI="👋👍"` in the environment.)
|
||||
- **lone narrow codepoint** (e.g. `🛣` U+1F6E3): both authorities already say one. If
|
||||
the font nonetheless paints a two-cell colour glyph — an emoji outside
|
||||
Emoji_Presentation whose East-Asian width is Neutral — it *overruns* every row that
|
||||
pads to a width; nothing narrowed it, it was never measured right. Add it to
|
||||
`_DEFAULT_WIDE_BASE` (or `MESHTERM_WIDE_EMOJI` for a session test). Otherwise no change.
|
||||
- **lone narrow codepoint, bare** (e.g. `🛣` U+1F6E3, `🕸` U+1F578): **no change**, and
|
||||
resist the pull to "fix" it. Both authorities say one because the codepoint is outside
|
||||
Emoji_Presentation, and with no VS16 asking for emoji presentation the font draws a
|
||||
one-cell *text* glyph — they are right. Forcing it into `_DEFAULT_WIDE_BASE` reserves a
|
||||
cell the terminal never draws and pulls the row's border a column **in**; that is
|
||||
exactly what listing `🛣` did to the Trophy case, while `🕸` — same class, one board
|
||||
down, never listed — framed flush throughout. Both authorities already agreeing is not
|
||||
a bug to correct.
|
||||
- **VS16 emoji-presentation sequence** (e.g. `☀️`): **handled by default** (the variation
|
||||
selector is skipped, so the base is measured alone). But the renderer is not uniform —
|
||||
if *this* one paints two cells wide, add its **base** codepoint to `_DEFAULT_WIDE_BASE`,
|
||||
|
||||
@@ -92,11 +92,13 @@ def _classify(emoji: str) -> tuple[str, str]:
|
||||
"If your terminal draws it TWO wide, leave it — narrowing would break its border.",
|
||||
)
|
||||
return (
|
||||
"lone narrow codepoint",
|
||||
"Both authorities measure it ONE. If your terminal draws it that way, no change is "
|
||||
"needed. If the font draws it as a two-cell colour glyph (an emoji outside "
|
||||
"Emoji_Presentation, e.g. U+1F6E3 🛣), it overruns every row that pads to a width — "
|
||||
"add it to _DEFAULT_WIDE_BASE (or MESHTERM_WIDE_EMOJI for a session test).",
|
||||
"lone narrow codepoint (bare, text presentation)",
|
||||
"NO CHANGE — and resist the pull to 'fix' it. Both authorities measure it one "
|
||||
"because it sits outside Emoji_Presentation, and with no VS16 asking for emoji "
|
||||
"presentation the font draws a one-cell text glyph: they are right. Forcing it "
|
||||
"to two reserves a cell the terminal never draws and pulls the row's border a "
|
||||
"column IN — the mistake U+1F6E3 🛣 cost the Trophy case. If this row really is "
|
||||
"misaligned, another glyph on it is the culprit; classify that one.",
|
||||
)
|
||||
if base and _VS16 in emoji:
|
||||
return (
|
||||
|
||||
@@ -52,20 +52,22 @@ as *four* (a two-column notch, worse than a lone emoji's one). Since the indicat
|
||||
shared across flags, they can only be handled as a whole category, never one country at a
|
||||
time; narrowing every indicator to one cell makes any flag sum to two in both authorities.
|
||||
|
||||
Some glyphs run the other way — this terminal draws them **two** cells where the authorities say
|
||||
one — and they are carved back out into a curated *wide* set (:data:`_DEFAULT_WIDE_BASE`, extended
|
||||
without a code change via ``MESHTERM_WIDE_EMOJI``) counted two in **both** authorities. A VS16
|
||||
sequence lands here when the "width 1" verdict is wrong for it: the renderer is not uniform — the
|
||||
probe ``☀️`` paints in one cell, but ``🛩️`` (a small airplane, U+1F6E9 U+FE0F) paints in two, and
|
||||
narrowing it along with the rest pulls its chat-row border a column short and smears the line. A
|
||||
*lone* codepoint lands here for a plainer reason: an emoji outside Emoji_Presentation whose
|
||||
East-Asian width is Neutral (``🛣``, U+1F6E3) is measured as one cell by Rich and wcwidth alike,
|
||||
yet the font draws it as a two-cell colour glyph — never narrowed, just never measured right, so
|
||||
it overruns its row by a column wherever the row pads to a width. Either way it is the mirror image
|
||||
of the lone-narrow allowlist, curated for the same reason: which glyph a terminal draws wide is the
|
||||
font's business, not the codepoint's, and a cursor probe reads the PTY, not the renderer. Keyed on
|
||||
the *base* codepoint (the trailing ``U+FE0F`` is skipped regardless), so a bare base airplane rides
|
||||
along with its VS16 form.
|
||||
Some VS16 sequences run the other way — this terminal draws them **two** cells where the "width 1"
|
||||
verdict would narrow them — and they are carved back out into a curated *wide* set
|
||||
(:data:`_DEFAULT_WIDE_BASE`, extended without a code change via ``MESHTERM_WIDE_EMOJI``) counted two
|
||||
in **both** authorities. The renderer is simply not uniform: the probe ``☀️`` paints in one cell,
|
||||
but ``🛩️`` (a small airplane, U+1F6E9 U+FE0F) paints in two, and narrowing it along with the rest
|
||||
pulls its chat-row border a column short and smears the line. It is the mirror image of the
|
||||
lone-narrow allowlist, curated for the same reason: which glyph a terminal draws wide is the font's
|
||||
business, not the codepoint's, and a cursor probe reads the PTY, not the renderer. Keyed on the
|
||||
*base* codepoint (the trailing ``U+FE0F`` is skipped regardless).
|
||||
|
||||
A **bare** codepoint the authorities *already agree* measures one is not a candidate for that set,
|
||||
however wide the glyph looks in a font book: without a variation selector asking for emoji
|
||||
presentation, an emoji outside Emoji_Presentation (``🛣`` U+1F6E3, ``🕸`` U+1F578) draws as a
|
||||
one-cell text glyph, which is what both authorities said. Forcing one to two reserves a cell the
|
||||
terminal never draws and pulls the row's border a column *in* — see :data:`_DEFAULT_WIDE_BASE` for
|
||||
the case that proved it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -93,28 +95,28 @@ _PROBE = "☀️"
|
||||
#: the confirmed glyph and extended — no code change — through ``MESHTERM_NARROW_EMOJI``.
|
||||
_DEFAULT_NARROW_LONE = "👋"
|
||||
|
||||
#: Base codepoints this terminal draws *two* cells wide where the width authorities say one.
|
||||
#: Two sources feed it, and both are the same font story:
|
||||
#: Base codepoints this terminal draws *two* cells wide where the width authorities say one —
|
||||
#: the mirror of :data:`_DEFAULT_NARROW_LONE`, and populated for one reason only: a **VS16
|
||||
#: sequence** the narrow-VS16 verdict gets wrong. The renderer is not uniform (see the module
|
||||
#: docstring): ``☀️``, the calibration probe, paints in one cell, but ``🛩️`` (U+1F6E9 U+FE0F)
|
||||
#: paints in two, so blanket-narrowing it would pull that row's border a column short.
|
||||
#:
|
||||
#: * a **VS16 sequence** the narrow-VS16 verdict gets wrong — the renderer is not uniform (see the
|
||||
#: module docstring): ``☀️``, the calibration probe, paints in one cell, but ``🛩️``
|
||||
#: (U+1F6E9 U+FE0F) paints in two, so blanket-narrowing it pulls its chat-row border a column
|
||||
#: short and smears the row;
|
||||
#: * a **lone codepoint** Rich and wcwidth both already call one — the emoji outside
|
||||
#: Emoji_Presentation, whose East-Asian width is Neutral (``🛣`` U+1F6E3, the Trophy case's
|
||||
#: Longest-distance icon) — that the font nonetheless draws as a two-cell colour glyph. Nothing
|
||||
#: narrowed these; they were never measured right in the first place, so they overrun their row
|
||||
#: by a column on every screen that pads to a width.
|
||||
#: A **bare** codepoint does not belong here, and the attempt to put one here is worth recording
|
||||
#: because the mistake is inviting. An emoji outside Emoji_Presentation — ``🛣`` U+1F6E3, ``🕸``
|
||||
#: U+1F578 — has East-Asian width Neutral, so Rich and wcwidth both measure it one; without a
|
||||
#: variation selector asking for emoji presentation the font draws it as a one-cell text glyph,
|
||||
#: which is exactly what they said. Listing ``🛣`` to force it to two made the Trophy case's
|
||||
#: Longest-distance heading reserve a cell the terminal never drew, pulling that row's right
|
||||
#: border one column in (JP, 2026-08-09) — while ``🕸``, the same Unicode class one board down and
|
||||
#: never listed, framed flush the whole time. **Both authorities already agreeing at one is not a
|
||||
#: bug to correct**; when a row misaligns, check whether the glyph carries U+FE0F before assuming
|
||||
#: the font overrode the measurement.
|
||||
#:
|
||||
#: There is no rule separating either from its correctly-measured neighbours — it is the font's
|
||||
#: glyph, not the codepoint, and a probe reads the PTY, not the renderer — so, the mirror of
|
||||
#: :data:`_DEFAULT_NARROW_LONE`, this is a curated set, seeded with the confirmed glyphs and
|
||||
#: extended, no code change, through ``MESHTERM_WIDE_EMOJI``. Keyed on the base codepoint,
|
||||
#: Which VS16 sequence a terminal paints wide is still the font's business rather than the
|
||||
#: codepoint's, and a probe reads the PTY and not the renderer, so this stays a curated set,
|
||||
#: extended without a code change through ``MESHTERM_WIDE_EMOJI``. Keyed on the base codepoint,
|
||||
#: because the trailing selector is skipped by both authorities either way.
|
||||
_DEFAULT_WIDE_BASE = (
|
||||
"\U0001f6e9" # 🛩 airplane (U+1F6E9), drawn two-wide with its VS16 selector
|
||||
"\U0001f6e3" # 🛣 motorway (U+1F6E3), drawn two-wide though both authorities measure one
|
||||
)
|
||||
_DEFAULT_WIDE_BASE = "\U0001f6e9" # 🛩 airplane, drawn two-wide with its VS16 selector
|
||||
|
||||
#: Set once :func:`calibrate` has run so repeated calls are cheap no-ops.
|
||||
_CALIBRATED = False
|
||||
|
||||
+27
-11
@@ -25,7 +25,8 @@ _CA = "🇨🇦" # a flag: two Regional Indicators; prompt_toolkit miscounts it
|
||||
_CN = "🇨🇳" # a second flag sharing the "C" indicator — the whole category must be handled
|
||||
_PLANE = "🛩️" # a VS16 sequence this terminal draws two wide despite the narrow-VS16 verdict
|
||||
_PLANE_BASE = "\U0001f6e9" # its base codepoint (no selector) — what the wide set is keyed on
|
||||
_ROAD = "\U0001f6e3" # 🛣 a lone codepoint both authorities measure one, drawn two wide here
|
||||
_ROAD = "\U0001f6e3" # 🛣 a bare text-default emoji: one cell, and both authorities agree
|
||||
_WEB = "\U0001f578" # 🕸 its untouched sibling one Trophy board down — the control
|
||||
|
||||
|
||||
def test_narrow_lone_set_defaults_extends_and_disables(monkeypatch) -> None:
|
||||
@@ -44,9 +45,10 @@ def test_narrow_lone_set_defaults_extends_and_disables(monkeypatch) -> None:
|
||||
def test_wide_base_set_defaults_extends_and_disables(monkeypatch) -> None:
|
||||
"""The wide set seeds to the confirmed bases; the env var overrides it outright."""
|
||||
monkeypatch.delenv("MESHTERM_WIDE_EMOJI", raising=False)
|
||||
# Both sources of a wide base: a VS16 sequence the narrow verdict gets wrong (the airplane)
|
||||
# and a lone codepoint both authorities under-measure at one cell (the motorway).
|
||||
assert {_PLANE_BASE, _ROAD} <= ew._wide_base_set()
|
||||
# One source only: a VS16 sequence the narrow verdict gets wrong. A *bare* codepoint both
|
||||
# authorities already measure at one is not a candidate — see test_bare_text_default_emoji.
|
||||
assert _PLANE_BASE in ew._wide_base_set()
|
||||
assert not {_ROAD, _WEB} & ew._wide_base_set()
|
||||
|
||||
# ✈ (U+2708) is a second VS16 airplane base; both list cleanly.
|
||||
monkeypatch.setenv("MESHTERM_WIDE_EMOJI", "\U0001f6e9✈")
|
||||
@@ -77,9 +79,6 @@ def test_rich_cell_len_narrows_only_allowlisted_lone_emoji() -> None:
|
||||
# frames flush instead of collapsing to one and smearing the row.
|
||||
assert cell_len(_PLANE) == 2
|
||||
assert cell_len("hi 🛩️") == len("hi ") + 2
|
||||
# A listed *lone* base is the same fix for a glyph nothing narrowed: Rich reads U+1F6E3 as
|
||||
# one cell on its own, so a heading carrying it would overrun its row by a column.
|
||||
assert ew._make_cell_len(frozenset(), frozenset(_ROAD))(_ROAD) == 2
|
||||
|
||||
|
||||
def test_pt_cache_narrows_only_allowlisted_lone_emoji() -> None:
|
||||
@@ -95,8 +94,24 @@ def test_pt_cache_narrows_only_allowlisted_lone_emoji() -> None:
|
||||
# zero, so the whole glyph (and any line holding it) keeps the terminal's two cells.
|
||||
assert cache[_PLANE] == 2
|
||||
assert cache["🛩️ hi"] == 2 + len(" hi")
|
||||
# And a listed lone base, which wcwidth also under-measures at one, lands on two as well.
|
||||
assert ew._make_pt_cache(frozenset(), frozenset(_ROAD))[_ROAD] == 2
|
||||
|
||||
|
||||
def test_bare_text_default_emoji_are_left_exactly_as_measured() -> None:
|
||||
"""An emoji outside Emoji_Presentation, drawn *bare*, is one cell — and stays one.
|
||||
|
||||
``🛣`` and ``🕸`` have East-Asian width Neutral, so Rich and wcwidth both measure them at
|
||||
one; with no variation selector asking for emoji presentation, the font draws a one-cell
|
||||
text glyph, which is exactly what they said. Both authorities already agreeing is not a
|
||||
bug to correct: listing ``🛣`` in the wide set made the Trophy case's Longest-distance
|
||||
heading reserve a cell the terminal never drew and pulled that row's border a column in,
|
||||
while ``🕸`` — same class, one board down, never listed — framed flush throughout.
|
||||
"""
|
||||
cell_len = ew._make_cell_len(frozenset(_WAVE), ew._wide_base_set())
|
||||
cache = ew._make_pt_cache(frozenset(_WAVE), ew._wide_base_set())
|
||||
for bare in (_ROAD, _WEB):
|
||||
assert cell_len(bare) == 1 and cache[bare] == 1
|
||||
# A heading built around one costs its own cells and no more, so the row frames flush.
|
||||
assert cell_len(f"── {_ROAD} Longest distance ──") == len("── ") + 1 + len(" Longest distance ──")
|
||||
|
||||
|
||||
def test_flags_measure_two_cells_in_both_authorities() -> None:
|
||||
@@ -142,12 +157,13 @@ def test_calibrate_width1_narrows_the_wave_in_both_authorities(monkeypatch) -> N
|
||||
assert cells.cell_len(_DISH) == 2 # unlisted icon stays wide
|
||||
assert cells.cell_len(_CA) == 2 # flag handled by category, no allowlist entry
|
||||
assert cells.cell_len(_PLANE) == 2 # wide-VS16 exception carved back out of the narrowing
|
||||
assert cells.cell_len(_ROAD) == 2 # a lone base the authorities under-measured at one
|
||||
assert cells.cell_len(_ROAD) == 1 # bare text-default emoji: left exactly as measured
|
||||
assert cells.cell_len(_WEB) == 1
|
||||
assert get_cwidth(_WAVE) == 1 # pt now places the border a cell earlier
|
||||
assert get_cwidth(_DISH) == 2
|
||||
assert get_cwidth(_CA) == 2 # was 4 unpatched
|
||||
assert get_cwidth(_PLANE) == 2 # was 1 unpatched: the airplane smeared a cell short
|
||||
assert get_cwidth(_ROAD) == 2 # likewise for the motorway, in the border authority
|
||||
assert get_cwidth(_ROAD) == 1 and get_cwidth(_WEB) == 1 # untouched, in both authorities
|
||||
finally:
|
||||
_restore(snap)
|
||||
|
||||
|
||||
Reference in new issue
Block a user