Chat ragged border: fix country flags, add-narrow-emoji skill
A country flag (🇨🇦) is a Regional Indicator pair, not a lone codepoint: Rich and the terminal draw the pair as one two-cell glyph, but prompt_toolkit counts each indicator as two (wcwidth), measuring the flag as four and placing the border two columns short — a worse notch than a lone emoji's one. Handle flags as a category, not per-country: an indicator is measured as one cell, so any flag sums to two in both authorities. The indicators are shared across flags (🇨🇦 and 🇨🇳 both start with C), so listing a single flag would half-fix its neighbours and skip the rest — the block is the only correct unit. prompt_toolkit's cache is now always swapped when the renderer draws emoji narrow (not only when the lone allowlist is non-empty), since flags need it even with an empty MESHTERM_NARROW_EMOJI. Add the add-narrow-emoji skill: a classify.py that reports an emoji's codepoints and its width under wcwidth/Rich/pt (now and fixed) and names the class + action, plus a SKILL.md procedure. It steers a lone wide codepoint into the allowlist, recognises flags and VS16 as already handled, and refuses ZWJ / keycap / modifier clusters (their parts would misalign) — encoding the per-glyph, confirm-visually, never-blanket discipline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
c155aa4216
commit
0b8ec9f75f
4 files changed
+251
-15
No files matched your search
@@ -0,0 +1,79 @@
|
||||
---
|
||||
name: add-narrow-emoji
|
||||
description: Fix a ragged/short right border on a MeshTerm chat row that contains an emoji the terminal draws narrower than it measures. Classifies the emoji (lone codepoint, country flag, VS16 sequence, or ZWJ/other cluster) and adds it to the terminal-aligned width handling the correct way — a lone codepoint joins the allowlist; flags and VS16 are already handled; ZWJ clusters need a maintainer. Use when a chat message with an emoji notches the panel's right border.
|
||||
---
|
||||
|
||||
# Adding a narrow emoji to MeshTerm's border-alignment allowlist
|
||||
|
||||
MeshTerm aligns Rich's and prompt_toolkit's glyph widths with how the terminal
|
||||
*actually draws* an emoji, so a chat row frames flush instead of notching its right
|
||||
border a column (or more) short. The machinery lives in
|
||||
[meshterm/ui/tui/emoji_width.py](../../..//meshterm/ui/tui/emoji_width.py); the design
|
||||
is in that module's docstring and in the `glyph-width-alignment` memory. Background:
|
||||
the terminal's narrow/wide split is **per-glyph and font-driven** — there is no rule and
|
||||
no way to probe it — so lone emoji are a hand-curated allowlist, extended one glyph at a
|
||||
time as they are confirmed. This skill does that one addition correctly.
|
||||
|
||||
## When to run
|
||||
|
||||
A chat message containing an emoji renders with its right border pulled in (a blank sits
|
||||
between the border and the frame edge), or a smear to the right of the glyph. That is the
|
||||
symptom of the terminal drawing the glyph in fewer cells than Rich/prompt_toolkit reserve.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Classify the emoji.** Run, from the repo root:
|
||||
|
||||
```
|
||||
$DEV_VENV_PYTHON .claude\skills\add-narrow-emoji\classify.py <emoji>
|
||||
```
|
||||
|
||||
It prints the codepoints, how each authority measures it now (`Rich now`, `pt now`)
|
||||
and after the fix (`Rich fixed`, `pt fixed`), the class, and the action. `pt now` is
|
||||
the number that matters — prompt_toolkit places the panel border.
|
||||
|
||||
2. **Act on the class:**
|
||||
- **lone wide codepoint** (e.g. `👋`, `👍`): the allowlist case. Only add it **if the
|
||||
terminal draws it in one cell** — confirm visually first. Append the character to
|
||||
`_DEFAULT_NARROW_LONE` in [emoji_width.py](../../..//meshterm/ui/tui/emoji_width.py):
|
||||
`_DEFAULT_NARROW_LONE = "👋👍"`. (For a throwaway session test instead of a code
|
||||
change, set `MESHTERM_NARROW_EMOJI="👋👍"` in the environment.)
|
||||
- **flag (Regional Indicator pair)** (e.g. `🇨🇦`): **already handled** as a whole
|
||||
category — no entry. Confirm `pt fixed` is 2 and stop. Never list an individual flag:
|
||||
indicators are shared, so it would half-fix that flag's neighbours and skip the rest.
|
||||
- **VS16 emoji-presentation sequence** (e.g. `☀️`): **already handled** (the variation
|
||||
selector is skipped). No entry.
|
||||
- **ZWJ sequence / keycap / skin-tone / other cluster** (e.g. `👨👩👧`, `1️⃣`, `👋🏽`):
|
||||
**not covered** by the lone/flag mechanism — narrowing its parts individually would
|
||||
misalign it. Do not force it into the allowlist; note it for bespoke handling.
|
||||
|
||||
3. **Extend the tests.** In [tests/test_emoji_width.py](../../..//tests/test_emoji_width.py),
|
||||
add the new glyph to `test_rich_cell_len_narrows_only_allowlisted_lone_emoji` and
|
||||
`test_pt_cache_narrows_only_allowlisted_lone_emoji` (both authorities → one cell), and to
|
||||
the width-1 calibrate test. Keep a known-wide emoji (`📡`) asserted at two, so a future
|
||||
over-broad change is caught.
|
||||
|
||||
4. **Run the tests:**
|
||||
|
||||
```
|
||||
$DEV_VENV_PYTHON -m pytest tests/test_emoji_width.py -q
|
||||
```
|
||||
|
||||
5. **Confirm visually.** The fix can't be reproduced headlessly — a non-interactive run
|
||||
keeps Rich's defaults, and CI can't see the renderer. Ask the person on the affected
|
||||
terminal to send a chat message with the glyph and confirm the border is now flush and
|
||||
nothing shifted. Only then is the addition trustworthy.
|
||||
|
||||
6. **Commit** with a message naming the glyph and why (see the repo's commit style).
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Never narrow a glyph the terminal draws two wide** (a menu icon like `📡`, `💬`). The
|
||||
fix only helps a glyph drawn *narrower* than measured; narrowing a correctly-wide glyph
|
||||
introduces the notch instead of removing it. When unsure, leave it out.
|
||||
- **One glyph, confirmed, at a time.** An unlisted narrow emoji is no worse than before
|
||||
(the `erase_down` full-repaint still prevents its smear); a wrongly-listed one breaks a
|
||||
border. Bias to omission.
|
||||
- **Do not touch Rich/prompt_toolkit width globally.** A blanket emoji remap has been tried
|
||||
and reverted twice — it wrecks the menu icons the terminal genuinely draws two wide. The
|
||||
curated allowlist and the flag category are the only sanctioned levers.
|
||||
@@ -0,0 +1,110 @@
|
||||
"""Diagnose one emoji for the MeshTerm terminal-aligned width machinery.
|
||||
|
||||
Given an emoji on the command line, report its codepoints and how the three width
|
||||
authorities measure it (wcwidth, Rich, prompt_toolkit), then classify it and print the
|
||||
exact action the `add-narrow-emoji` skill should take. Run from the repo root:
|
||||
|
||||
d:\\vibe\\MeshTerm\\.venv\\Scripts\\python.exe .claude\\skills\\add-narrow-emoji\\classify.py 👋
|
||||
|
||||
The point is to remove the guesswork the skill's judgement call depends on: a lone wide
|
||||
codepoint gets an allowlist entry, a flag is already handled by category, and a ZWJ /
|
||||
keycap / modifier cluster is *not* covered and must not be shoehorned in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
# The chat border fix only ever engages on a terminal that draws emoji narrow; measure
|
||||
# there or trust the maintainer's eyes. This script just classifies — it never patches.
|
||||
sys.stdout.reconfigure(encoding="utf-8") # print emoji safely on a cp1252 console
|
||||
# Run from anywhere: put the repo root (four levels up) on the path so `meshterm` imports.
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[3]))
|
||||
|
||||
from wcwidth import wcwidth # noqa: E402
|
||||
|
||||
import rich.cells as rc # noqa: E402
|
||||
from prompt_toolkit.utils import get_cwidth # noqa: E402
|
||||
|
||||
from meshterm.ui.tui.emoji_width import ( # noqa: E402
|
||||
_DEFAULT_NARROW_LONE,
|
||||
_is_regional_indicator,
|
||||
_make_cell_len,
|
||||
_make_pt_cache,
|
||||
)
|
||||
|
||||
_ZWJ = ""
|
||||
_VS16 = "️"
|
||||
|
||||
|
||||
def _classify(emoji: str) -> tuple[str, str]:
|
||||
"""Return ``(label, action)`` for ``emoji`` — what it is and what the skill should do."""
|
||||
cps = [ord(c) for c in emoji]
|
||||
if not cps:
|
||||
return "empty", "Nothing to classify — pass an emoji as the argument."
|
||||
if all(_is_regional_indicator(c) for c in emoji) and len(emoji) == 2:
|
||||
return (
|
||||
"flag (Regional Indicator pair)",
|
||||
"ALREADY HANDLED as a category — no allowlist entry. Verify pt width is 2 below; "
|
||||
"if so there is nothing to add. Never list a single flag (it half-fixes its "
|
||||
"neighbours that share an indicator).",
|
||||
)
|
||||
if _ZWJ in emoji:
|
||||
return (
|
||||
"ZWJ sequence (multi-glyph cluster)",
|
||||
"NOT covered by the lone/flag mechanism (a family, a profession, a flag-with-tag). "
|
||||
"Do not add it to the allowlist — the parts would be narrowed individually and "
|
||||
"misalign. Flag this to a maintainer for bespoke handling.",
|
||||
)
|
||||
if len(emoji) == 1:
|
||||
if wcwidth(emoji) == 2:
|
||||
return (
|
||||
"lone wide codepoint",
|
||||
"IF your terminal draws it in ONE cell, add it to _DEFAULT_NARROW_LONE in "
|
||||
"meshterm/ui/tui/emoji_width.py (or MESHTERM_NARROW_EMOJI for a session test). "
|
||||
"If your terminal draws it TWO wide, leave it — narrowing would break its border.",
|
||||
)
|
||||
return (
|
||||
"lone narrow codepoint",
|
||||
"Already one cell everywhere — it never notches a border. No change needed.",
|
||||
)
|
||||
others = [c for c in emoji if c not in (_VS16, _ZWJ)]
|
||||
if len(others) == 1 and _VS16 in emoji:
|
||||
return (
|
||||
"VS16 emoji-presentation sequence",
|
||||
"ALREADY HANDLED (the variation selector is skipped so the base is measured alone). "
|
||||
"No allowlist entry needed.",
|
||||
)
|
||||
return (
|
||||
"multi-codepoint cluster (keycap / skin-tone / other)",
|
||||
"NOT a plain lone codepoint — the lone/flag mechanism does not cover it. Flag this to a "
|
||||
"maintainer rather than adding parts to the allowlist.",
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if len(argv) != 2:
|
||||
print("usage: classify.py <emoji>")
|
||||
return 2
|
||||
emoji = argv[1]
|
||||
narrow = frozenset(_DEFAULT_NARROW_LONE)
|
||||
label, action = _classify(emoji)
|
||||
|
||||
print(f"emoji : {emoji}")
|
||||
print("codepoints : " + " ".join(f"U+{ord(c):04X}" for c in emoji))
|
||||
print(f"grapheme : {len(emoji)} codepoint(s)")
|
||||
print(f"wcwidth : {sum(max(0, wcwidth(c)) for c in emoji)} (per char: "
|
||||
+ ", ".join(str(wcwidth(c)) for c in emoji) + ")")
|
||||
print(f"Rich now : {rc.cell_len(emoji)}")
|
||||
print(f"pt now : {get_cwidth(emoji)} <- the authority that places the panel border")
|
||||
print(f"Rich fixed : {_make_cell_len(narrow)(emoji)}")
|
||||
print(f"pt fixed : {_make_pt_cache(narrow)[emoji]}")
|
||||
print()
|
||||
print(f"class : {label}")
|
||||
print(f"action : {action}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
@@ -43,6 +43,14 @@ is an explicit allowlist (:data:`_DEFAULT_NARROW_LONE`, extended without a code
|
||||
``MESHTERM_NARROW_EMOJI``), and — because it is the panel border *prompt_toolkit* places,
|
||||
not Rich — narrowing a lone emoji patches *both* width authorities, where the VS16 fix only
|
||||
needed Rich (prompt_toolkit never promoted a VS16 sequence in the first place).
|
||||
|
||||
Country **flags** (``🇨🇦``, ``🇺🇸``) are the same "width 1" story but need no allowlist,
|
||||
because for them there *is* a rule: a flag is a grapheme pair of Regional Indicator Symbols
|
||||
(see :func:`_is_regional_indicator`), which the terminal — and Rich — draw as one two-cell
|
||||
glyph, while prompt_toolkit's wcwidth counts each indicator as two and so measures the flag
|
||||
as *four* (a two-column notch, worse than a lone emoji's one). Since the indicators are
|
||||
shared across flags, they can only be handled as a whole category, never one country at a
|
||||
time; narrowing every indicator to one cell makes any flag sum to two in both authorities.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -74,6 +82,20 @@ _DEFAULT_NARROW_LONE = "👋"
|
||||
_CALIBRATED = False
|
||||
|
||||
|
||||
def _is_regional_indicator(char: str) -> bool:
|
||||
"""Whether ``char`` is a single Regional Indicator Symbol — a country flag's building block.
|
||||
|
||||
The two-letter flags (``🇨🇦``, ``🇺🇸``) are a grapheme pair drawn from this block
|
||||
(U+1F1E6–U+1F1FF). wcwidth counts each indicator as two, so prompt_toolkit measures a flag
|
||||
as *four* cells, while Rich and the terminal draw the pair as one two-cell glyph. Narrowing
|
||||
every indicator to one makes any flag sum to two in both authorities, so a chat row with a
|
||||
flag frames flush. This is a whole category, not a per-country entry: the indicators are
|
||||
shared across flags (``🇨🇦`` and ``🇨🇳`` both start with ``C``), so they can only be
|
||||
handled as a block — listing one flag would half-fix its neighbours and skip the rest.
|
||||
"""
|
||||
return len(char) == 1 and 0x1F1E6 <= ord(char) <= 0x1F1FF
|
||||
|
||||
|
||||
def _narrow_lone_set() -> frozenset[str]:
|
||||
"""The lone-codepoint emoji to measure as one cell (see :data:`_DEFAULT_NARROW_LONE`).
|
||||
|
||||
@@ -263,6 +285,8 @@ def _make_cell_len(narrow: frozenset[str]) -> Callable[[str, str], int]:
|
||||
* **Curated lone emoji** — a single codepoint in ``narrow`` (``👋`` and friends) that Rich
|
||||
would call two but this terminal draws in one is counted as one, so a chat row carrying
|
||||
it pads to a flush right border instead of a column-short notch.
|
||||
* **Flag indicators** — a Regional Indicator (:func:`_is_regional_indicator`) counts as one,
|
||||
so a country flag sums to two (Rich already agrees here; this keeps the two loops in step).
|
||||
|
||||
Everything else keeps its ``get_character_cell_size`` value, so a lone emoji the terminal
|
||||
*does* draw two wide (a menu icon like ``📡``, never placed in ``narrow``) is left alone.
|
||||
@@ -276,7 +300,10 @@ def _make_cell_len(narrow: frozenset[str]) -> Callable[[str, str], int]:
|
||||
for char in text:
|
||||
if char in _ZERO_WIDTH:
|
||||
continue
|
||||
total += 1 if char in narrow else get_size(char, unicode_version)
|
||||
if char in narrow or _is_regional_indicator(char):
|
||||
total += 1
|
||||
else:
|
||||
total += get_size(char, unicode_version)
|
||||
return total
|
||||
|
||||
return _cell_len
|
||||
@@ -289,9 +316,10 @@ def _make_pt_cache(narrow: frozenset[str]) -> "object":
|
||||
composed frame into its screen buffer stepping a cursor by ``get_cwidth`` (backed by
|
||||
``_CHAR_SIZES_CACHE``), so unless it too measures ``👋`` as one, it positions the border a
|
||||
column past where the terminal draws the narrow glyph — the notch survives even with Rich
|
||||
corrected. The subclass returns one for a listed lone glyph and defers everything else to
|
||||
the stock ``wcwidth`` logic; because the base ``__missing__`` sums per character through
|
||||
the cache, a whole chat line ``"Bob 👋 hi"`` inherits the one-cell ``👋`` for free.
|
||||
corrected. The subclass returns one for a listed lone glyph (or a flag's Regional Indicator)
|
||||
and defers everything else to the stock ``wcwidth`` logic; because the base ``__missing__``
|
||||
sums per character through the cache, a whole chat line ``"Bob 👋 hi"`` inherits the one-cell
|
||||
``👋`` for free, and a flag ``"🇨🇦"`` sums its two one-cell indicators to a flush two.
|
||||
"""
|
||||
import prompt_toolkit.utils as ptu
|
||||
|
||||
@@ -299,7 +327,7 @@ def _make_pt_cache(narrow: frozenset[str]) -> "object":
|
||||
|
||||
class _NarrowLoneCache(base_cache_cls): # type: ignore[valid-type, misc]
|
||||
def __missing__(self, string: str) -> int:
|
||||
if string in narrow:
|
||||
if string in narrow or _is_regional_indicator(string):
|
||||
self[string] = 1
|
||||
return 1
|
||||
return super().__missing__(string)
|
||||
@@ -312,21 +340,18 @@ def _install_terminal_widths(narrow: frozenset[str]) -> None:
|
||||
|
||||
Rich's ``cell_len`` delegates to the module-level ``_cell_len``; replacing that (and
|
||||
clearing the memoised wrapper) steers every downstream caller — panels, tables, text —
|
||||
through :func:`_make_cell_len`. prompt_toolkit is patched *only* when there are lone emoji
|
||||
to narrow: a VS16-only correction needs nothing from it (it never promoted the sequence),
|
||||
but a lone-emoji correction does, because pt is what places the border (see
|
||||
:func:`_make_pt_cache`). Both run at :func:`calibrate` time, before a frame is drawn, so no
|
||||
stale two-cell measurement or ``Char`` is ever painted.
|
||||
through :func:`_make_cell_len`. prompt_toolkit is patched too, because it is what places the
|
||||
border (see :func:`_make_pt_cache`): a lone emoji or a flag both need it, and even an empty
|
||||
``narrow`` still leaves the flag-indicator category to correct, so the cache is always
|
||||
swapped. Both run at :func:`calibrate` time, before a frame is drawn, so no stale two-cell
|
||||
measurement or ``Char`` is ever painted.
|
||||
"""
|
||||
import prompt_toolkit.utils as ptu
|
||||
import rich.cells as cells
|
||||
|
||||
cells.cached_cell_len.cache_clear()
|
||||
cells._cell_len = _make_cell_len(narrow)
|
||||
|
||||
if narrow:
|
||||
import prompt_toolkit.utils as ptu
|
||||
|
||||
ptu._CHAR_SIZES_CACHE = _make_pt_cache(narrow)
|
||||
ptu._CHAR_SIZES_CACHE = _make_pt_cache(narrow)
|
||||
|
||||
|
||||
__all__ = ["calibrate"]
|
||||
@@ -21,6 +21,8 @@ from meshterm.ui.tui import emoji_width as ew
|
||||
# One confirmed narrow glyph, plus one the same terminal draws two wide, used throughout.
|
||||
_WAVE = "👋" # the confirmed lone-codepoint emoji this terminal draws in one cell
|
||||
_DISH = "📡" # a menu icon the terminal draws two wide — must never be narrowed
|
||||
_CA = "🇨🇦" # a flag: two Regional Indicators; prompt_toolkit miscounts it as four cells
|
||||
_CN = "🇨🇳" # a second flag sharing the "C" indicator — the whole category must be handled
|
||||
|
||||
|
||||
def test_narrow_lone_set_defaults_extends_and_disables(monkeypatch) -> None:
|
||||
@@ -59,6 +61,24 @@ def test_pt_cache_narrows_only_allowlisted_lone_emoji() -> None:
|
||||
assert cache["Bob 👋"] == 5
|
||||
|
||||
|
||||
def test_flags_measure_two_cells_in_both_authorities() -> None:
|
||||
"""A country flag is a Regional Indicator pair: prompt_toolkit's wcwidth calls it four
|
||||
cells, Rich and the terminal draw it as one two-cell glyph. Narrowing the indicators as a
|
||||
category makes every flag sum to two in both authorities — no per-country allowlist entry,
|
||||
and flags sharing an indicator (🇨🇦 / 🇨🇳) are all fixed at once."""
|
||||
# Only the lone-emoji allowlist is passed; flags are handled by category, not by listing.
|
||||
cell_len = ew._make_cell_len(frozenset(_WAVE))
|
||||
cache = ew._make_pt_cache(frozenset(_WAVE))
|
||||
|
||||
for flag in (_CA, _CN):
|
||||
assert cell_len(flag) == 2
|
||||
assert cache[flag] == 2 # was 4 unpatched: two indicators at wcwidth 2 each
|
||||
|
||||
# A flag rides a chat line without dragging the border: "eh? 🇨🇦" measures its plain
|
||||
# cells plus the flag's two.
|
||||
assert cell_len("eh 🇨🇦") == len("eh ") + 2
|
||||
|
||||
|
||||
def _snapshot() -> tuple:
|
||||
"""Capture the mutable width state :func:`calibrate` patches, to restore afterwards."""
|
||||
return (cells._cell_len, ptu._CHAR_SIZES_CACHE, ew._CALIBRATED)
|
||||
@@ -81,8 +101,10 @@ def test_calibrate_width1_narrows_the_wave_in_both_authorities(monkeypatch) -> N
|
||||
ew.calibrate(force_width=1)
|
||||
assert cells.cell_len(_WAVE) == 1
|
||||
assert cells.cell_len(_DISH) == 2 # unlisted icon stays wide
|
||||
assert cells.cell_len(_CA) == 2 # flag handled by category, no allowlist entry
|
||||
assert get_cwidth(_WAVE) == 1 # pt now places the border a cell earlier
|
||||
assert get_cwidth(_DISH) == 2
|
||||
assert get_cwidth(_CA) == 2 # was 4 unpatched
|
||||
finally:
|
||||
_restore(snap)
|
||||
|
||||
|
||||
Reference in new issue
Block a user