Every glyph that may be drawn as an emoji is reserved two cells

The curated narrow and wide sets are gone, with the probe that decided when
to apply them and the add-narrow-emoji skill that fed them. They lined up
the rows somebody had already looked at and none of the rows nobody had,
and a contact's name arrives off the air carrying whatever its owner typed.

In their place, one structural rule: a glyph built from several codepoints
(a selector sequence, a ZWJ sequence, a keycap, a flag) or a lone
text-default emoji or Regional Indicator is reserved two cells, the most a
single glyph draws. Pinned into two cells, a glyph the font draws in two
fills them and one it draws in one leaves a blank beside it, so the text
after any emoji starts in the same column on every row. Pinning alone could
only hold the lanes after it.

Two exceptions keep their one cell: Rich's single-cell ranges, which its
fast path measures without asking, and the nine text-default emoji the app
draws itself as marks. That list is closed by a test over the package's
source, so a new mark cannot slip into the reserve unnoticed.

Rich measures in three places and prompt_toolkit in a fourth, and all four
are patched from the same rule at once: Rich's segment splitter walks a
string until a whole-string width meets its cut, and a character measured
differently from the string around it is a cut that walk never meets. Every
multi-codepoint glyph is also merged into one fragment before layout, so a
mark followed by a selector is not folded into a one-cell slot.

Installed only by the full-screen app on a platform that draws emoji. The
command line keeps the stock tables, because nothing pins its output.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jean-Pierre MartineauandClaude Opus 5 committed 2026-09-16 20:29:02 -04:00
1 parent 68c528938c
commit 38d9ffacc5
8 files changed
+622 -1219

No files matched your search

-107
View File
@@ -1,107 +0,0 @@
---
name: add-narrow-emoji
description: Fix a MeshTerm row whose layout an emoji breaks — a right border pulled a column short, or a glyph overrunning its lane — because the terminal draws it in a different number of cells than Rich/prompt_toolkit measure. Works in both directions: a glyph drawn narrower than measured joins the narrow allowlist, one drawn wider joins the wide set. Classifies the emoji (lone codepoint, country flag, VS16 sequence, or ZWJ/other cluster), says whether it is already listed, and adds it the correct way. Use whenever an emoji "fucks up the layout", notches a panel border, or smears a row.
---
# Adding an emoji to MeshTerm's terminal-aligned width sets
MeshTerm aligns Rich's and prompt_toolkit's glyph widths with how the terminal
*actually draws* an emoji, so a row frames flush instead of notching its right border a
column short or overrunning its lane. The machinery lives in
[emoji_width.py](../../..//meshterm/ui/tui/emoji_width.py); the design is in that module's
docstring and in the `glyph-width-alignment` memory. Background: the terminal's
narrow/wide split is **per-glyph and font-driven** — there is no rule and no way to probe
it (a cursor probe reads the PTY, not the renderer) — so the exceptions are two
hand-curated sets, extended one confirmed glyph at a time. This skill does that one
addition correctly.
## When to run
An emoji in a row breaks its layout, in either of two ways:
- **Border pulled in** (a blank sits between the border and the frame edge), or a smear to
the right of the glyph — the terminal draws it in *fewer* cells than the authorities
reserve. → the narrow allowlist (`_DEFAULT_NARROW_LONE`).
- **The row overruns**, pushing what follows a column right — the terminal draws it in
*more* cells than the authorities reserve. → the wide set (`_DEFAULT_WIDE_BASE`). In
practice this only happens to a **VS16 sequence** the narrow verdict got wrong; a *bare*
codepoint both authorities call one cell is measured correctly and must be left alone.
Step 1 tells you which; it also tells you when the glyph is **already listed**, which is
the commonest answer to "please add this one".
## Steps
1. **Classify the emoji.** Run, from the repo root:
The interpreter below is `$DEV_VENV_PYTHON` from `.dev.env` at the repo root (gitignored;
`.dev.env.example` shows the shape) — read it first.
```
$DEV_VENV_PYTHON .claude/skills/add-narrow-emoji/classify.py <emoji>
```
It prints the codepoints, how each authority measures it now (`Rich now`, `pt now`)
and after the fix (`Rich fixed`, `pt fixed`), the class, and the action. `pt now` is
the number that matters — prompt_toolkit places the panel border.
2. **Act on the class.** The machinery runs in *both* directions — a glyph drawn narrower
than measured joins `_DEFAULT_NARROW_LONE`, one drawn wider joins `_DEFAULT_WIDE_BASE`
— and the classifier names which, including "already listed", the commonest answer:
- **already listed** (either set): nothing to add. If a row still misaligns, some
*other* glyph on it is the culprit — classify that one.
- **lone wide codepoint** (e.g. `👋`, `👍`): the narrow-allowlist case. Only add it **if
the terminal draws it in one cell** — confirm visually first. Append the character to
`_DEFAULT_NARROW_LONE` in [emoji_width.py](../../..//meshterm/ui/tui/emoji_width.py):
`_DEFAULT_NARROW_LONE = "👋👍"`. (For a throwaway session test instead of a code
change, set `MESHTERM_NARROW_EMOJI="👋👍"` in the environment.)
- **lone narrow codepoint, bare** (e.g. `🛣` U+1F6E3, `🕸` U+1F578): **no change**, and
resist the pull to "fix" it. Both authorities say one because the codepoint is outside
Emoji_Presentation, and with no VS16 asking for emoji presentation the font draws a
one-cell *text* glyph — they are right. Forcing it into `_DEFAULT_WIDE_BASE` reserves a
cell the terminal never draws and pulls the row's border a column **in**; that is
exactly what listing `🛣` did to the Trophy case, while `🕸` — same class, one board
down, never listed — framed flush throughout. Both authorities already agreeing is not
a bug to correct.
- **VS16 emoji-presentation sequence** (e.g. `☀️`): **handled by default** (the variation
selector is skipped, so the base is measured alone). But the renderer is not uniform —
if *this* one paints two cells wide, add its **base** codepoint to `_DEFAULT_WIDE_BASE`,
as `🛩️` (U+1F6E9) needed. Keying on the base makes the bare form ride along.
- **flag (Regional Indicator pair)** (e.g. `🇨🇦`): **already handled** as a whole
category — no entry. Confirm `pt fixed` is 2 and stop. Never list an individual flag:
indicators are shared, so it would half-fix that flag's neighbours and skip the rest.
- **ZWJ sequence / keycap / skin-tone / other cluster** (e.g. `👨‍👩‍👧`, `1️⃣`, `👋🏽`):
**not covered** by the lone/flag mechanism — resizing its parts individually would
misalign it. Do not force it into either set; note it for bespoke handling.
3. **Extend the tests.** In [tests/test_emoji_width.py](../../..//tests/test_emoji_width.py),
add the new glyph to `test_rich_cell_len_narrows_only_allowlisted_lone_emoji` and
`test_pt_cache_narrows_only_allowlisted_lone_emoji` (both authorities → one cell for a
narrow addition, two for a wide one), and to the width-1 calibrate test. Keep a
known-wide emoji (`📡`) asserted at two, so a future over-broad change is caught.
4. **Run the tests:**
```
$DEV_VENV_PYTHON -m pytest tests/test_emoji_width.py -q
```
5. **Confirm visually.** The fix can't be reproduced headlessly — a non-interactive run
keeps Rich's defaults, and CI can't see the renderer. Ask the person on the affected
terminal to send a chat message with the glyph and confirm the border is now flush and
nothing shifted. Only then is the addition trustworthy.
6. **Commit** with a message naming the glyph and why (see the repo's commit style).
## Guardrails
- **Never narrow a glyph the terminal draws two wide** (a menu icon like `📡`, `💬`), and
never widen one it draws in one cell. Each set only helps glyphs mismeasured in its own
direction; a wrong entry *introduces* the misalignment it was meant to remove. When
unsure, leave it out.
- **One glyph, confirmed, at a time.** An unlisted emoji is no worse than before (the
`erase_down` full-repaint still prevents a narrow one's smear); a wrongly-listed one
breaks a border. Bias to omission.
- **Do not touch Rich/prompt_toolkit width globally.** A blanket emoji remap has been tried
and reverted twice — it wrecks the menu icons the terminal genuinely draws two wide. The
curated allowlist and the flag category are the only sanctioned levers.
-144
View File
@@ -1,144 +0,0 @@
r"""Diagnose one emoji for the MeshTerm terminal-aligned width machinery.
Given an emoji on the command line, report its codepoints and how the three width
authorities measure it (wcwidth, Rich, prompt_toolkit), then classify it and print the
exact action the `add-narrow-emoji` skill should take. Run from the repo root:
$DEV_VENV_PYTHON .claude/skills/add-narrow-emoji/classify.py 👋
(``$DEV_VENV_PYTHON`` is the interpreter named in ``.dev.env``; ``.dev.env.example`` shows
the shape.)
The point is to remove the guesswork the skill's judgement call depends on: a lone wide
codepoint gets an allowlist entry, a flag is already handled by category, and a ZWJ /
keycap / modifier cluster is *not* covered and must not be shoehorned in.
"""
from __future__ import annotations
import pathlib
import sys
# The chat border fix only ever engages on a terminal that draws emoji narrow; measure
# there or trust the maintainer's eyes. This script just classifies — it never patches.
sys.stdout.reconfigure(encoding="utf-8") # print emoji safely on a cp1252 console
# Run from anywhere: put the repo root (four levels up) on the path so `meshterm` imports.
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[3]))
import rich.cells as rc # noqa: E402
from prompt_toolkit.utils import get_cwidth # noqa: E402
from wcwidth import wcwidth # noqa: E402
from meshterm.ui.tui.emoji_width import ( # noqa: E402
_DEFAULT_NARROW_LONE,
_DEFAULT_WIDE_BASE,
_is_regional_indicator,
_make_cell_len,
_make_pt_cache,
)
_ZWJ = "‍"
_VS16 = "️"
_NARROW = frozenset(_DEFAULT_NARROW_LONE)
_WIDE = frozenset(_DEFAULT_WIDE_BASE) - frozenset((_ZWJ, _VS16))
def _base(emoji: str) -> str:
"""The emoji's base codepoint — what the wide set is keyed on (selectors dropped)."""
others = [c for c in emoji if c not in (_VS16, _ZWJ)]
return others[0] if len(others) == 1 else ""
def _classify(emoji: str) -> tuple[str, str]:
"""Return ``(label, action)`` for ``emoji`` — what it is and what the skill should do."""
if not emoji:
return "empty", "Nothing to classify — pass an emoji as the argument."
# Already curated? Say so first — it is the commonest answer to "please add this one",
# and every branch below would otherwise re-derive advice for a decision already made.
base = _base(emoji)
if base and base in _WIDE:
return (
"lone codepoint drawn WIDE — already listed",
"ALREADY IN _DEFAULT_WIDE_BASE (both authorities measure it 2, bare and with its "
"VS16 selector). Nothing to add. If a row still misaligns, the culprit is another "
"glyph on it — classify that one instead.",
)
if len(emoji) == 1 and emoji in _NARROW:
return (
"lone codepoint drawn NARROW — already listed",
"ALREADY IN _DEFAULT_NARROW_LONE (both authorities measure it 1). Nothing to add.",
)
if len(emoji) == 2 and all(_is_regional_indicator(c) for c in emoji):
return (
"flag (Regional Indicator pair)",
"ALREADY HANDLED as a category — no allowlist entry. Verify pt width is 2 below; "
"if so there is nothing to add. Never list a single flag (it half-fixes its "
"neighbours that share an indicator).",
)
if _ZWJ in emoji:
return (
"ZWJ sequence (multi-glyph cluster)",
"NOT covered by the lone/flag mechanism (a family, a profession, a flag-with-tag). "
"Do not add it to either set — the parts would be resized individually and "
"misalign. Flag this to a maintainer for bespoke handling.",
)
if len(emoji) == 1:
if wcwidth(emoji) == 2:
return (
"lone wide codepoint",
"IF your terminal draws it in ONE cell, add it to _DEFAULT_NARROW_LONE in "
"meshterm/ui/tui/emoji_width.py (or MESHTERM_NARROW_EMOJI for a session test). "
"If your terminal draws it TWO wide, leave it — narrowing would break its border.",
)
return (
"lone narrow codepoint (bare, text presentation)",
"NO CHANGE — and resist the pull to 'fix' it. Both authorities measure it one "
"because it sits outside Emoji_Presentation, and with no VS16 asking for emoji "
"presentation the font draws a one-cell text glyph: they are right. Forcing it "
"to two reserves a cell the terminal never draws and pulls the row's border a "
"column IN — the mistake U+1F6E3 🛣 cost the Trophy case. If this row really is "
"misaligned, another glyph on it is the culprit; classify that one.",
)
if base and _VS16 in emoji:
return (
"VS16 emoji-presentation sequence",
"HANDLED BY DEFAULT (the variation selector is skipped, so the base is measured "
"alone and the narrow-VS16 verdict applies). If your terminal draws THIS one two "
"cells wide anyway — the renderer is not uniform — add its BASE codepoint to "
"_DEFAULT_WIDE_BASE (or MESHTERM_WIDE_EMOJI), as U+1F6E9 🛩️ needed.",
)
return (
"multi-codepoint cluster (keycap / skin-tone / other)",
"NOT a plain lone codepoint — the lone/flag mechanism does not cover it. Flag this to a "
"maintainer rather than adding parts to either set.",
)
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: classify.py <emoji>")
return 2
emoji = argv[1]
label, action = _classify(emoji)
print(f"emoji : {emoji}")
print("codepoints : " + " ".join(f"U+{ord(c):04X}" for c in emoji))
print(f"grapheme : {len(emoji)} codepoint(s)")
print(f"wcwidth : {sum(max(0, wcwidth(c)) for c in emoji)} (per char: "
+ ", ".join(str(wcwidth(c)) for c in emoji) + ")")
print(f"Rich now : {rc.cell_len(emoji)}")
print(f"pt now : {get_cwidth(emoji)} <- the authority that places the panel border")
# "fixed" = what the width-1 calibration path already produces today, both sets applied.
print(f"Rich fixed : {_make_cell_len(_NARROW, _WIDE)(emoji)}")
print(f"pt fixed : {_make_pt_cache(_NARROW, _WIDE)[emoji]}")
print()
print(f"class : {label}")
print(f"action : {action}")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+5 -4
View File
@@ -75,10 +75,11 @@ class Platform:
this is ``False`` there. ``MESHTERM_FULL_WIDTH`` remains an explicit override
on top of this default.
emoji: Whether emoji icons render at all. ``False`` means every icon funnel routes
through a compact single-BMP-glyph table instead (P3), and the emoji-width
calibration probe (:func:`~meshterm.ui.tui.emoji_width.calibrate`) is skipped
outright — there is nothing to calibrate once no emoji are ever drawn, and the
probe writes escape sequences the console font can't shape anyway.
through a compact single-BMP-glyph table instead (P3), and neither half of the
emoji alignment runs — the reserve-two measurement
(:func:`~meshterm.ui.tui.emoji_width.install`) or the column pinning
(:mod:`~meshterm.ui.tui.colsnap`). Once no emoji are ever drawn, every glyph on
screen is one the console font has verified, and the stock widths are exact.
ascii_fold: Whether names and message bodies are NFKD-folded (accents stripped) at
the render boundary before display. Storage is never touched. Wired in P3.
truecolor: Whether the theme may use arbitrary 24-bit SGR colour. ``False`` selects
+7 -7
View File
@@ -48,7 +48,7 @@ from .tui import (
Separator,
TuiSession,
)
from .tui.emoji_width import calibrate as calibrate_emoji_width
from .tui.emoji_width import install as install_emoji_widths
from .tui.spinner import spinner_interval
from .widgets import battery_cell
@@ -379,13 +379,13 @@ async def run_menu(ctx: AppContext) -> None:
Args:
ctx: The shared application context.
"""
# Measure how this terminal renders emoji and align Rich to it, before
# prompt_toolkit takes over the screen. This keeps every panel border — and
# every chat bubble — aligned regardless of the terminal's emoji widths. Skipped
# outright on a platform that never draws emoji (PicoCalc): there is nothing to
# calibrate, and the probe writes escape sequences the console font can't shape.
# Reserve two cells for every glyph that may be drawn as an emoji, in every width
# authority, before prompt_toolkit lays out its first frame: the session pins each such
# glyph into its reservation, so rows line up whatever the terminal's font does with it.
# Skipped outright on a platform that never draws emoji (PicoCalc), where every glyph is
# one its own font has verified and the stock widths are already exact.
if get_platform().emoji:
calibrate_emoji_width()
install_emoji_widths()
header_cache: dict = {}
session = TuiSession(header=lambda cols: _header(ctx, header_cache, cols))
+15 -16
View File
@@ -1,10 +1,9 @@
# SPDX-License-Identifier: Apache-2.0
"""Pin a written row's columns to the terminal's own grid, so no glyph can shift it.
Every other correction in this package answers *how many cells does this glyph take* —
:mod:`~meshterm.ui.tui.emoji_width` is that question's whole home, two width authorities and
two hand-curated exception sets deep. This module answers the one that actually decides
whether a row lines up: **which column does the next glyph land in?**
:mod:`~meshterm.ui.tui.emoji_width` answers *how many cells does this glyph take*. This
module answers the question that actually decides whether a row lines up: **which column does
the next glyph land in?**
A row reaches the terminal as one run of characters, and the terminal advances its own cursor
by its own idea of each glyph's width. Where that idea differs from ours by a cell,
@@ -18,13 +17,12 @@ Whether a font draws a given emoji in one cell or two is the font's business rat
codepoint's; there is no table that is right on every terminal; and no escape sequence reports
a renderer's width back to the program, because on a split terminal the component that tracks
the cursor and the component that paints the glyph are different programs that disagree with
each other (a cursor probe reads the PTY, not the painter — :mod:`~meshterm.ui.tui.emoji_width`
carries that reasoning in full, along with the two curated sets it leaves us with). Curating
exceptions one confirmed glyph at a time lines up the rows somebody has already looked at, and
leaves every name that arrives off the air — written by a stranger, carrying whatever they felt
like typing — exactly as broken as it was. A contact list is the worst case in the app for
precisely that reason: its content is the one thing on screen that nobody can put on a list in
advance.
each other (a cursor probe reads the PTY, not the painter). Curating exceptions one confirmed
glyph at a time — which is what the app used to do — lines up the rows somebody has already
looked at, and leaves every name that arrives off the air, written by a stranger and carrying
whatever they felt like typing, exactly as broken as it was. A contact list is the worst case
in the app for precisely that reason: its content is the one thing on screen that nobody can
put on a list in advance.
So this module stops asking the terminal to agree. After any glyph whose drawn width is not
certain, the row carries an **absolute column address** — ``CSI n G``, naming the one-based
@@ -40,9 +38,10 @@ move anything but itself:
* Drawn **wider**, its overhang is written over by whatever the next column holds.
Both are one cell of cosmetic damage inside the glyph's own lane, where a shift was a whole row
of it. The curated width sets keep their job — they are what makes the *reservation* right, so
a listed glyph gets neither the gap nor the overwrite — but they stop being what holds the row
together. A glyph nobody has ever classified now costs a cell instead of a row.
of it, and the second never happens in practice: :mod:`~meshterm.ui.tui.emoji_width` reserves
every glyph that may be drawn as an emoji two cells, the most a single glyph draws. Pinning holds
the lanes after a glyph wherever the terminal puts it; the reservation is what makes the text
beside it line up from row to row as well, since a pin can only land where the glyph was measured.
**What is certain.** Pinning after a glyph that did not need it is harmless (it addresses the
column the cursor is already in) but it is not free: five bytes on the wire and a cluster walk
@@ -55,8 +54,8 @@ screen; the walk is paid only where an emoji actually is.
Getting an entry in that list wrong costs exactly what today costs anyway — one glyph's row —
so the list is a performance claim and never the thing correctness rests on. That is the whole
point of the inversion: the old exception sets had to be *complete* to keep the app aligned,
and this one only has to be *cheap*.
point of the inversion: the exception sets this replaced had to be *complete* to keep the app
aligned, and this one only has to be *cheap*.
**Two writers, two spellings of the same pin.** A frame reaches the terminal one of two ways,
and each knows something different about where its cursor is:
File diff suppressed because it is too large. Load diff
+293 -324
View File
@@ -1,330 +1,322 @@
# SPDX-License-Identifier: Apache-2.0
"""Terminal-aligned emoji width: VS16 sequences and curated lone-codepoint emoji.
"""Reserve-two emoji widths: one rule in every width authority, and every glyph kept whole.
The border-alignment story lives in :mod:`meshterm.ui.tui.emoji_width`. These tests pin the
two things that make a chat row with ``👋`` frame flush: Rich must *measure* the glyph as one
cell (so it pads the row to the right width) and prompt_toolkit must *count* it as one (so it
places the panel's right border where the terminal actually draws it). An emoji the terminal
draws two wide — a menu icon like ``📡``, never on the allowlist — must stay two in both.
The measuring story lives in :mod:`meshterm.ui.tui.emoji_width`, and the placement half in
:mod:`meshterm.ui.tui.colsnap`. What these tests pin:
The factory helpers (``_make_cell_len``, ``_make_pt_cache``) are exercised in isolation so no
process-global width table is touched; the two :func:`calibrate` tests that do patch the real
tables snapshot and restore them in a ``finally`` so nothing leaks into a later test.
* every glyph that may be drawn as an emoji is reserved two cells, in Rich and in prompt_toolkit;
* the app's own marks, Rich's single-cell ranges and ordinary text keep their stock widths;
* Rich's three measurements and prompt_toolkit's agree about every string, which is what keeps
Rich's segment splitter from walking past a cut it can never meet;
* the app's source draws no text-default emoji its vocabulary does not name;
* a glyph built from several codepoints is split, cut and laid out as the one glyph it is.
:func:`~meshterm.ui.tui.emoji_width.install` patches process-global tables, so every test that
needs it goes through the ``installed`` fixture, which puts every table and cache back afterwards.
"""
from __future__ import annotations
import ast
import pathlib
import threading
from collections.abc import Iterator
import prompt_toolkit.utils as ptu
import pytest
import rich.cells as cells
import rich.segment as segment
from prompt_toolkit.utils import get_cwidth
from meshterm.ui.tui import emoji_width as ew
# One confirmed narrow glyph, plus one the same terminal draws two wide, used throughout.
_WAVE = "👋" # the confirmed lone-codepoint emoji this terminal draws in one cell
_DISH = "📡" # a menu icon the terminal draws two wide — must never be narrowed
_CA = "🇨🇦" # a flag: two Regional Indicators; prompt_toolkit miscounts it as four cells
_CN = "🇨🇳" # a second flag sharing the "C" indicator — the whole category must be handled
_PLANE = "🛩️" # a VS16 sequence this terminal draws two wide despite the narrow-VS16 verdict
_PLANE_BASE = "\U0001f6e9" # its base codepoint (no selector) — what the wide set is keyed on
_ROAD = "\U0001f6e3" # 🛣 a bare text-default emoji: one cell, and both authorities agree
_WEB = "\U0001f578" # 🕸 its untouched sibling one Trophy board down — the control
_SHRUG = "\U0001f937‍♂️" # 🤷‍♂️ a ZWJ sequence: base, joiner, male sign, selector — one glyph
_FAMILY = "\U0001f468‍\U0001f469‍\U0001f467" # 👨‍👩‍👧 three joined people, still one glyph
_ZWJ = "‍" # the joiner itself: the tell that its neighbours are a single glyph
_THUMB = "\U0001f44d\U0001f3fd" # 👍🏽 a base and its skin tone, no joiner — still one glyph
_TECHIE = "\U0001f468\U0001f3fb‍\U0001f4bb" # 👨🏻‍💻 a toned base, then a joined laptop
_FLAG = "\U0001f3f4" # 🏴 a black flag, the base of the pirate flag below
_STRANDED = f"{_FLAG}‍" # 🏴‍☠️ cut after its joiner by a name's byte limit: joins nothing
_ZWJ = "\u200d" # the joiner: the tell that its neighbours are a single glyph
_VS16 = "\ufe0f" # the request for emoji presentation
_WAVE = "\U0001f44b" # 👋 emoji presentation by default: two cells by the stock tables already
_DISH = "\U0001f4e1" # 📡 a menu icon, the same
_CA = "\U0001f1e8\U0001f1e6" # 🇨🇦 a flag: two Regional Indicators, one glyph
_CN = "\U0001f1e8\U0001f1f3" # 🇨🇳 a second flag sharing the "C" indicator
_SUN = "☀" # ☀ a text-default emoji, alone
_PLANE = "\U0001f6e9" # 🛩 another, which a font draws either way
_HEART = "❤" # ❤ another
_ROAD = "\U0001f6e3" # 🛣 text-default too, but one of the app's own marks
_WEB = "\U0001f578" # 🕸 the same
_BIN = "\U0001f5d1" # 🗑 the same
_WARN = "⚠" # ⚠ the same
_SHRUG = f"\U0001f937{_ZWJ}♂{_VS16}" # the shrug: base, joiner, sign, selector
_FAMILY = f"\U0001f468{_ZWJ}\U0001f469{_ZWJ}\U0001f467" # the family: three joined people
_THUMB = "\U0001f44d\U0001f3fd" # a thumb and its skin tone, no joiner: still one glyph
_TECHIE = f"\U0001f468\U0001f3fb{_ZWJ}\U0001f4bb" # the technologist: toned, then joined
_KEYCAP = f"1{_VS16}\u20e3" # a keycap: a digit, a selector and the enclosing mark
_FLAG = "\U0001f3f4" # a black flag, the base of a pirate flag
_STRANDED = f"{_FLAG}{_ZWJ}" # a pirate flag cut after its joiner by a name's byte limit
#: Strings every authority has to agree about: emoji of every shape, the app's own marks, and the
#: scripts and marks that must come through the rule untouched.
_SAMPLES = (
"plain ascii",
f"Bob {_FAMILY} x",
f"{_CA}{_CN}",
f"{_SUN}{_VS16}{_SHRUG}",
f"{_SUN} bare, {_PLANE} bare, {_HEART} bare",
f"a{_ZWJ}b",
f"{_STRANDED} x",
f"{_KEYCAP}!",
f"│ {_WARN}{_VS16} warn │ {_WARN} mark │",
f"Tech {_TECHIE} x {_THUMB}{_THUMB}",
"漢字 names",
"ष\u094d\u200dक", # a Devanagari conjunct: a joiner between letters, not pictographs
f"{_CA[0]} a lone indicator",
"e\u0301 a combining accent",
f"{_ROAD}{_WEB}{_BIN} ↔ ↕",
)
def test_narrow_lone_set_defaults_extends_and_disables(monkeypatch) -> None:
"""The allowlist seeds to the confirmed glyph; the env var overrides it outright."""
monkeypatch.delenv("MESHTERM_NARROW_EMOJI", raising=False)
assert _WAVE in ew._narrow_lone_set()
@pytest.fixture
def installed() -> Iterator[None]:
"""Install the reserve-two rule for one test, and put every table and cache back after."""
from meshterm.ui.tui import colsnap
from meshterm.ui.tui.render import _ANSI_CACHE
monkeypatch.setenv("MESHTERM_NARROW_EMOJI", "👋🤙")
assert {"👋", "🤙"} <= ew._narrow_lone_set()
# Empty string is the off switch — a way back to Rich/pt defaults with no code change.
monkeypatch.setenv("MESHTERM_NARROW_EMOJI", "")
assert ew._narrow_lone_set() == frozenset()
def test_wide_base_set_defaults_extends_and_disables(monkeypatch) -> None:
"""The wide set seeds to the confirmed bases; the env var overrides it outright."""
monkeypatch.delenv("MESHTERM_WIDE_EMOJI", raising=False)
# One source only: a VS16 sequence the narrow verdict gets wrong. A *bare* codepoint both
# authorities already measure at one is not a candidate — see test_bare_text_default_emoji.
assert _PLANE_BASE in ew._wide_base_set()
assert not {_ROAD, _WEB} & ew._wide_base_set()
# ✈ (U+2708) is a second VS16 airplane base; both list cleanly.
monkeypatch.setenv("MESHTERM_WIDE_EMOJI", "\U0001f6e9✈")
assert {"\U0001f6e9", "✈"} <= ew._wide_base_set()
# Pasting the whole rendered glyph keeps the base but strips the zero-width selector, so
# the selector is never itself counted as a wide cell.
monkeypatch.setenv("MESHTERM_WIDE_EMOJI", _PLANE)
wide = ew._wide_base_set()
assert _PLANE_BASE in wide and "️" not in wide
# Empty string trusts the narrow-VS16 verdict for every sequence.
monkeypatch.setenv("MESHTERM_WIDE_EMOJI", "")
assert ew._wide_base_set() == frozenset()
def test_rich_cell_len_narrows_only_allowlisted_lone_emoji() -> None:
"""Rich's replacement measures a listed lone emoji as one, an unlisted one still as two."""
cell_len = ew._make_cell_len(frozenset(_WAVE), frozenset(_PLANE_BASE))
assert cell_len(_WAVE) == 1
assert cell_len(_DISH) == 2 # unlisted: the terminal draws it wide, so leave it wide
assert cell_len("Bob 👋 hi") == 8 # the narrowed glyph flows through a whole line
# The pre-existing VS16 handling still applies: a narrow base is measured alone, its
# variation selector skipped, so "☀️" stays one cell rather than being promoted to two.
assert cell_len("☀️") == 1
# A wide-VS16 exception is forced back to two, selector still skipped, so the airplane
# frames flush instead of collapsing to one and smearing the row.
assert cell_len(_PLANE) == 2
assert cell_len("hi 🛩️") == len("hi ") + 2
# A ZWJ sequence is one glyph, measured as its base: the codepoints the joiner folds in
# cost nothing, however many of them there are. Rich's stock loop already does this, and
# the selector-skipping replacement must not lose it — counting the male sign as a cell
# of its own is what pulls the row's right border a column in.
assert cell_len(_SHRUG) == 2
assert cell_len(_FAMILY) == 2
assert cell_len(f"Bob {_FAMILY} hi") == len("Bob hi") + 2
def test_pt_cache_narrows_only_allowlisted_lone_emoji() -> None:
"""prompt_toolkit's cache — the authority that places the border — matches Rich."""
cache = ew._make_pt_cache(frozenset(_WAVE), frozenset(_PLANE_BASE))
assert cache[_WAVE] == 1
assert cache[_DISH] == 2
# The base cache sums a multi-char string per character through the cache, so the
# one-cell wave is inherited by any line that contains it.
assert cache["Bob 👋"] == 5
# The wide-VS16 airplane is the reverse: its base is forced to two, the selector stays
# zero, so the whole glyph (and any line holding it) keeps the terminal's two cells.
assert cache[_PLANE] == 2
assert cache["🛩️ hi"] == 2 + len(" hi")
# The sequences prompt_toolkit added up part by part: three cells for the shrug and six
# for the family, each of them one two-cell glyph, each pulling a border in by the excess.
assert cache[_SHRUG] == 2
assert cache[_FAMILY] == 2
assert cache[f"Bob {_FAMILY} hi"] == len("Bob hi") + 2
# A lone joiner is still zero, and a codepoint a sequence joins keeps its own width when
# it stands alone — ``↕`` is the reorder icon, not part of anything.
assert cache[_ZWJ] == 0
assert cache["↕"] == 1
def test_bare_text_default_emoji_are_left_exactly_as_measured() -> None:
"""An emoji outside Emoji_Presentation, drawn *bare*, is one cell — and stays one.
``🛣`` and ``🕸`` have East-Asian width Neutral, so Rich and wcwidth both measure them at
one; with no variation selector asking for emoji presentation, the font draws a one-cell
text glyph, which is exactly what they said. Both authorities already agreeing is not a
bug to correct: listing ``🛣`` in the wide set made the Trophy case's Longest-distance
heading reserve a cell the terminal never drew and pulled that row's border a column in,
while ``🕸`` — same class, one board down, never listed — framed flush throughout.
"""
cell_len = ew._make_cell_len(frozenset(_WAVE), ew._wide_base_set())
cache = ew._make_pt_cache(frozenset(_WAVE), ew._wide_base_set())
for bare in (_ROAD, _WEB):
assert cell_len(bare) == 1 and cache[bare] == 1
# A heading built around one costs its own cells and no more, so the row frames flush.
caption = cell_len(f"── {_ROAD} Longest distance ──")
assert caption == len("── ") + 1 + len(" Longest distance ──")
# The Trophy case's *visible* gap after the road is not this module's business and must
# not become it: the heading pads the mark out to the widest of the seven disciplines
# (records_screen.discipline_label), which costs a real cell the terminal advances over
# — where widening the glyph here would claim one it does not.
def test_flags_measure_two_cells_in_both_authorities() -> None:
"""A country flag measures two cells in both width authorities.
A flag is a Regional Indicator pair: prompt_toolkit's wcwidth calls it four cells,
while Rich and the terminal draw it as one two-cell glyph. Narrowing the
indicators as a category makes every flag sum to two in both — no per-country
allowlist entry, and flags sharing an indicator (🇨🇦 / 🇨🇳) are fixed at once.
"""
# Only the lone-emoji allowlist is passed; flags are handled by category, not by listing.
cell_len = ew._make_cell_len(frozenset(_WAVE), frozenset())
cache = ew._make_pt_cache(frozenset(_WAVE), frozenset())
for flag in (_CA, _CN):
assert cell_len(flag) == 2
assert cache[flag] == 2 # was 4 unpatched: two indicators at wcwidth 2 each
# A flag rides a chat line without dragging the border: "eh? 🇨🇦" measures its plain
# cells plus the flag's two.
assert cell_len("eh 🇨🇦") == len("eh ") + 2
def _snapshot() -> tuple:
"""Capture the mutable width state :func:`calibrate` patches, to restore afterwards."""
return (cells._cell_len, ptu._CHAR_SIZES_CACHE, ew._CALIBRATED, ew._CLUSTERS)
def _restore(snap: tuple) -> None:
"""Put the width authorities (and the once-only flag) back, clearing Rich's memo cache."""
cells._cell_len, ptu._CHAR_SIZES_CACHE, ew._CALIBRATED, ew._CLUSTERS = snap
cells.cached_cell_len.cache_clear()
def test_calibrate_width1_narrows_the_wave_in_both_authorities(monkeypatch) -> None:
"""A renderer that draws emoji narrow gets both Rich and pt aligned to the terminal."""
monkeypatch.delenv("MESHTERM_NARROW_EMOJI", raising=False)
monkeypatch.delenv("MESHTERM_WIDE_EMOJI", raising=False)
from prompt_toolkit.utils import get_cwidth
snap = _snapshot()
saved = (
cells._cell_len,
cells.get_character_cell_size,
cells.split_graphemes,
segment.get_character_cell_size,
ptu._CHAR_SIZES_CACHE,
ew._INSTALLED,
ew._CLUSTERS,
)
ew._INSTALLED = False
ew.install()
try:
ew._CALIBRATED = False
ew.calibrate(force_width=1)
assert cells.cell_len(_WAVE) == 1
assert cells.cell_len(_DISH) == 2 # unlisted icon stays wide
assert cells.cell_len(_CA) == 2 # flag handled by category, no allowlist entry
assert cells.cell_len(_PLANE) == 2 # wide-VS16 exception carved back out of the narrowing
assert cells.cell_len(_ROAD) == 1 # bare text-default emoji: left exactly as measured
assert cells.cell_len(_WEB) == 1
assert get_cwidth(_WAVE) == 1 # pt now places the border a cell earlier
assert get_cwidth(_DISH) == 2
assert get_cwidth(_CA) == 2 # was 4 unpatched
assert get_cwidth(_PLANE) == 2 # was 1 unpatched: the airplane smeared a cell short
assert get_cwidth(_ROAD) == 1 and get_cwidth(_WEB) == 1 # untouched, in both authorities
assert cells.cell_len(_SHRUG) == 2 and get_cwidth(_SHRUG) == 2 # pt counted three
assert cells.cell_len(_FAMILY) == 2 and get_cwidth(_FAMILY) == 2 # pt counted six
yield
finally:
_restore(snap)
(
cells._cell_len,
cells.get_character_cell_size,
cells.split_graphemes,
segment.get_character_cell_size,
ptu._CHAR_SIZES_CACHE,
ew._INSTALLED,
ew._CLUSTERS,
) = saved
cells.cached_cell_len.cache_clear()
_ANSI_CACHE.clear()
colsnap._CACHE.clear()
def test_calibrate_width2_narrows_nothing_but_still_joins_clusters(monkeypatch) -> None:
"""A width-2 terminal narrows nothing, but still gets the cluster join.
def test_whatever_may_be_drawn_as_an_emoji_is_reserved_two_cells(installed) -> None: # noqa: ANN001
"""Every shape of emoji measures two in both authorities, whatever the stock tables said.
Narrowing would itself break the border there — but a joined sequence is
over-measured at *every* width, so that one correction still lands.
prompt_toolkit sums a ZWJ sequence's codepoints wherever it runs, which no terminal draws:
the family is one two-cell glyph on the widest terminal as surely as on the narrowest. So
the cluster rule is installed on its own here, and nothing else is — the curated sets and
the flag category stay unconfirmed on this renderer, and Rich is not touched at all.
The stock answers were all over the place — prompt_toolkit gave a flag four, a sun asking
for emoji presentation one, a toned thumb four and a family six — and no answer below two
is safe for a glyph a font may draw in two: the pin after it would overwrite its right half.
"""
monkeypatch.delenv("MESHTERM_NARROW_EMOJI", raising=False)
from prompt_toolkit.utils import get_cwidth
snap = _snapshot()
try:
ew._CALIBRATED = False
ew.calibrate(force_width=2)
assert cells.cell_len(_WAVE) == 2
assert get_cwidth(_WAVE) == 2
# Rich untouched: it still promotes a VS16 sequence to the two cells this terminal draws.
assert cells.cell_len(_PLANE) == 2 and cells.cell_len("☀️") == 2
# Neither curated set nor the flag category applies here — pt keeps its stock answers.
assert get_cwidth("☀️") == 1
assert get_cwidth(_CA) == 4
# The one correction that holds at either width.
assert get_cwidth(_SHRUG) == 2 # was 3
assert get_cwidth(_FAMILY) == 2 # was 6
assert get_cwidth(_TECHIE) == 2 # was 6: a skin tone is no cell, at either width
finally:
_restore(snap)
for glyph in (
_WAVE,
_DISH,
_SUN,
f"{_SUN}{_VS16}",
_PLANE,
f"{_PLANE}{_VS16}",
_HEART,
_CA,
_CA[0],
_SHRUG,
_FAMILY,
_THUMB,
_TECHIE,
_KEYCAP,
f"{_WARN}{_VS16}", # a mark asking for emoji presentation is an emoji like any other
):
assert cells.cell_len(glyph) == 2, f"Rich: {glyph!r}"
assert get_cwidth(glyph) == 2, f"prompt_toolkit: {glyph!r}"
def test_a_skin_tone_is_part_of_its_glyph_in_both_authorities() -> None:
"""A skin-tone modifier recolours the emoji before it and takes no cell of its own.
def test_text_the_app_draws_keeps_its_stock_width(installed) -> None: # noqa: ANN001
"""The rule reserves nothing it has no reason to: text, chrome and the app's own marks.
wcwidth counts each modifier as two, so prompt_toolkit reserved four cells for ``👍🏽`` and
for the joined ``👨🏻‍💻`` — a two-cell notch in every row naming such a node, and every lane
after the name drawn two columns off. Rich's table already says zero; the cache now agrees,
on both calibration paths.
The marks are text-default emoji too, but they are the app's own vocabulary, drawn in one
cell on the terminals it is used on, so they keep the one cell every screen was laid out
with. Rich's single-cell ranges are measured by a fast path no patch reaches, so everything
has to agree with it there.
"""
for narrow, flags in ((frozenset(_WAVE), True), (frozenset(), False)):
cache = ew._make_pt_cache(narrow, frozenset(), flags=flags)
assert cache[_THUMB] == 2
assert cache[_TECHIE] == 2
assert cache[f"Tech {_TECHIE} x"] == len("Tech ") + 2 + len(" x")
assert ew._make_cell_len(frozenset(_WAVE), frozenset())(_TECHIE) == 2
for text, width in (
("a", 1),
("#", 1),
("1", 1),
("é", 1),
("漢", 2),
("─", 1),
("⠿", 1),
("©", 1),
("▶", 1),
(_WARN, 1),
(_ROAD, 1),
(_WEB, 1),
(_BIN, 1),
("↕", 1),
("↔", 1),
("↩", 1),
("⌨", 1),
("⚙", 1),
("a\U0001f3fd", 1), # a stray skin tone on a letter leaves it a letter
):
assert cells.cell_len(text) == width, f"Rich: {text!r}"
assert get_cwidth(text) == width, f"prompt_toolkit: {text!r}"
def test_a_stranded_joiner_leaves_the_next_character_its_cell(monkeypatch) -> None:
"""A joiner with no pictograph after it joins nothing, in both authorities and both paths.
def test_every_authority_measures_every_string_the_same(installed) -> None: # noqa: ANN001
"""Rich's whole-string width, its grapheme spans and prompt_toolkit's cache all agree.
A MeshCore name is cut at a byte limit, so ``That's So Fetch 🏴‍☠️`` arrives as the flag and
a bare joiner. Both authorities folded whatever came next into the flag — the padding after
the name — so the name lane measured a cell short and every lane after it began a column
before the terminal drew it.
And every lone character measures the same in all of them. These are the four places a width
is read, and a row is only laid out once if they give it one answer.
"""
for text in _SAMPLES:
spans, total = cells.split_graphemes(text)
assert cells.cell_len(text) == total == get_cwidth(text), text
assert total == sum(width for _start, _end, width in spans), text
assert spans[0][0] == 0 and spans[-1][1] == len(text), f"spans cover {text!r}"
for char in text:
size = cells.get_character_cell_size(char)
assert size == segment.get_character_cell_size(char) == get_cwidth(char), repr(char)
def test_no_prefix_outgrows_the_character_that_ends_it(installed) -> None: # noqa: ANN001
"""The invariant Rich's segment splitter terminates on, then the splitter itself.
``Segment.split_cells`` steps a string a codepoint at a time until a whole-string width meets
the cut, stepping over a two-cell character only where that character measures two on its
own. A prefix that grew by two at a character measuring one would be a cut the walk steps past
in both directions forever — which is why the character width is patched alongside the string
width, and why this checks the invariant before trusting the splitter with it.
"""
for text in _SAMPLES:
for index, char in enumerate(text):
grew = cells.cell_len(text[: index + 1]) - cells.cell_len(text[:index])
assert 0 <= grew <= 2, f"{text!r} at {index}"
if grew == 2:
assert cells.get_character_cell_size(char) == 2, f"{text!r} at {index}"
def split_everywhere() -> None:
for text in _SAMPLES:
for cut in range(cells.cell_len(text) + 1):
segment.Segment(text).split_cells(cut)
worker = threading.Thread(target=split_everywhere, daemon=True)
worker.start()
worker.join(timeout=20)
assert not worker.is_alive(), "a segment split never met its cut"
def test_a_stranded_joiner_leaves_the_next_character_its_cell(installed) -> None: # noqa: ANN001
"""A joiner with no pictograph after it joins nothing, in both authorities.
A MeshCore name is cut at a byte limit, so a name ending on a pirate flag arrives as the
black flag and a bare joiner, followed by the lane's padding. Rich's stock loop folded the
padding into the flag, so the name lane measured a cell short of what the terminal drew.
"""
monkeypatch.delenv("MESHTERM_NARROW_EMOJI", raising=False)
padded = f"{_STRANDED} x" # the flag's two cells, two of padding, then the next lane
cell_len = ew._make_cell_len(frozenset(_WAVE), frozenset())
assert cell_len(padded) == 5
assert ew._make_pt_cache(frozenset(_WAVE), frozenset())[padded] == 5
assert cells.cell_len(padded) == 5 and get_cwidth(padded) == 5
# A real sequence still joins: the shrug's male sign is a pictograph, not a padding space.
assert cell_len(f"{_SHRUG} x") == 4
assert cells.cell_len(f"{_SHRUG} x") == 4 and get_cwidth(f"{_SHRUG} x") == 4
snap = _snapshot()
try:
ew._CALIBRATED = False
ew.calibrate(force_width=2)
# Rich's own loop, trusted on this path, skipped the space and counted four.
assert cells.cell_len(padded) == 5
assert cells.cell_len(_SHRUG) == 2 and cells.cell_len(_FAMILY) == 2
finally:
_restore(snap)
def test_install_happens_once(installed) -> None: # noqa: ANN001
"""A second call is a no-op: the tables patched by the first stay exactly as they are."""
patched = (cells._cell_len, ptu._CHAR_SIZES_CACHE)
ew.install()
assert (cells._cell_len, ptu._CHAR_SIZES_CACHE) == patched
def test_the_app_draws_no_text_default_emoji_its_vocabulary_does_not_name() -> None:
"""Every text-default emoji the package draws is one of its own marks, and every mark is used.
This is what keeps :data:`~meshterm.ui.tui.emoji_width._APP_TEXT_MARKS` a closed vocabulary
rather than a list kept by care: a new one-cell mark added to a screen without being named
there fails here, instead of quietly gaining a blank cell beside it. Prose is skipped
(docstrings and bare strings are read, not drawn), and so is the module that defines the list.
"""
from rich._unicode_data import load
text_default = {
char
for char in load("auto").narrow_to_wide
if not char.isascii() and char not in cells._SINGLE_CELLS
}
# Inputs to the PicoCalc's substitution table, which maps them to glyphs its font has. They
# are never drawn on a terminal that shows emoji, so they are not marks.
fold_inputs = {"↪", "✔", "✖"}
package = pathlib.Path(ew.__file__).parents[2]
drawn: set[str] = set()
for path in package.rglob("*.py"):
if path.name == "emoji_width.py":
continue
tree = ast.parse(path.read_text(encoding="utf-8"))
prose = {
id(node.value)
for node in ast.walk(tree)
if isinstance(node, ast.Expr) and isinstance(node.value, ast.Constant)
}
for node in ast.walk(tree):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
if id(node) not in prose:
drawn |= set(node.value) & text_default
assert drawn - fold_inputs <= ew._APP_TEXT_MARKS, (
f"name these in _APP_TEXT_MARKS: {sorted(drawn - fold_inputs - ew._APP_TEXT_MARKS)}"
)
assert ew._APP_TEXT_MARKS <= drawn, (
f"no longer drawn anywhere: {sorted(ew._APP_TEXT_MARKS - drawn)}"
)
def test_clusters_split_between_glyphs_and_never_inside_one() -> None:
"""The unit a lane may cut on: one entry per glyph the terminal actually draws.
"""The unit a lane may cut on and a width is measured by: one entry per glyph drawn.
A joined run, a flag's indicator *pair*, and a toned base each come back whole, because
each is one glyph — while a joiner that joins nothing stands on its own, as it does
everywhere else in this module (see :func:`~meshterm.ui.tui.emoji_width._joins`).
A joined run, a flag's indicator *pair*, a toned base and a keycap each come back whole,
because each is one glyph — while a joiner that joins nothing stands on its own (see
:func:`~meshterm.ui.tui.emoji_width._joins`), and so do letters around a joiner.
"""
assert list(ew.clusters(f"a{_FAMILY}{_CA}{_THUMB}b")) == ["a", _FAMILY, _CA, _THUMB, "b"]
text = f"a{_FAMILY}{_CA}{_THUMB}{_KEYCAP}b"
assert list(ew.clusters(text)) == ["a", _FAMILY, _CA, _THUMB, _KEYCAP, "b"]
# Two flags in a row pair up one at a time rather than running together into four.
assert list(ew.clusters(_CA + _CN)) == [_CA, _CN]
# A name cut at its byte limit just past a joiner: the joiner is not part of the flag.
assert list(ew.clusters(_STRANDED)) == [_FLAG, _ZWJ]
# A joiner between letters joins nothing a font draws as one emoji.
assert list(ew.clusters(f"a{_ZWJ}b")) == ["a", _ZWJ, "b"]
def test_cut_cells_keeps_every_glyph_whole_and_strands_no_joiner() -> None:
"""A lane's truncation lands between glyphs, so nothing measures what is not drawn.
The failure this exists for: cutting a codepoint at a time leaves a trailing joiner,
which folds whatever the caller appends — the lane's ellipsis — into the glyph before
it. The ellipsis then measures nothing while the terminal still draws it, and every
column right of the name starts a cell late.
"""
"""A lane's truncation lands between glyphs, so nothing is measured that is not drawn."""
assert ew.cut_cells(f"Bob {_FAMILY}", 6) == f"Bob {_FAMILY}" # it fits, so it is kept
assert ew.cut_cells(f"Bob {_FAMILY}", 5) == "Bob " # it doesn't: dropped whole
assert ew.cut_cells(_CA, 1) == "" # half a flag is a letter, not half a glyph
assert ew.cut_cells(_STRANDED, 4) == _FLAG # the joiner cannot swallow what follows
assert ew.cut_cells(_STRANDED, 4) == _FLAG # the joiner does not trail the cut
assert ew.cut_cells("plain name", 5) == "plain"
def test_drawable_folds_only_what_no_terminal_can_draw() -> None:
"""A newline, an escape or a bidi override in an advert name is folded to a space.
None of them is a width bug an allowlist could fix — they are text that must not reach
the terminal at all. The joiner is format-class too and is the one kept, because it is
what holds an emoji sequence together.
None of them is a width question — they are text that must not reach the terminal at all.
The joiner is format-class too and is the one kept, because it holds an emoji together.
"""
assert ew.drawable("line\nbreak") == "line break"
assert ew.drawable("esc\x1b[31mape") == "esc [31mape"
assert ew.drawable("flip‮me") == "flip me"
assert ew.drawable("flip\u202eme") == "flip me"
assert ew.drawable(f"Bob {_FAMILY}{_THUMB}") == f"Bob {_FAMILY}{_THUMB}"
def test_join_clusters_merges_only_joined_runs() -> None:
"""The fragment merge gathers a whole sequence and leaves everything else alone.
def test_join_clusters_merges_every_multi_codepoint_glyph() -> None:
"""The fragment merge gathers each glyph built from several codepoints into one fragment.
prompt_toolkit's ANSI text arrives one codepoint per fragment, so a sequence is a run:
a base, an optional selector, then *joiner + component + optional selector* groups.
prompt_toolkit's ANSI text arrives one codepoint per fragment, so a glyph is a run to be
gathered — and every such run is merged now, not only the joined ones, because a mark
measured one cell and followed by a selector must not be folded into a one-cell slot.
"""
def line(text: str) -> list:
@@ -333,52 +325,41 @@ def test_join_clusters_merges_only_joined_runs() -> None:
def texts(fragments: list) -> list:
return [text for _style, text in fragments]
# A line with no joiner is handed straight back — the same object, not a copy.
plain = line("hi 📡")
# A line with nothing to merge is handed straight back — the same object, not a copy.
plain = line(f"hi {_DISH} {_WARN}")
assert ew._join_clusters(plain) is plain
# The shrug's four fragments become one; the bars around it are untouched.
assert texts(ew._join_clusters(line(f"|{_SHRUG}|"))) == ["|", _SHRUG, "|"]
# Three joined people, five fragments, still one glyph.
assert texts(ew._join_clusters(line(_FAMILY))) == [_FAMILY]
# A selector on the *base*, before the joiner, belongs to the run: ❤️‍🔥.
burning = "❤️‍\U0001f525"
burning = f"❤{_VS16}{_ZWJ}\U0001f525" # ❤\ufe0f\u200d🔥 a selector on the base, then a join
assert texts(ew._join_clusters(line(burning))) == [burning]
# A bare VS16 pair is not a sequence — prompt_toolkit already folds it into the cell
# before it, so those fragments are left exactly as they came.
assert texts(ew._join_clusters(line(f"☀️{_SHRUG}"))) == ["☀", "️", _SHRUG]
# A trailing joiner with nothing to join is not a sequence either.
assert texts(ew._join_clusters(line(f"a{_ZWJ}"))) == ["a", _ZWJ]
# A skin tone on the base belongs to the run the way its selector does.
assert texts(ew._join_clusters(line(f"|{_TECHIE}|"))) == ["|", _TECHIE, "|"]
# A stranded joiner, followed by the name lane's padding, is not a sequence: the space
# after it keeps a fragment — and so a cell — of its own.
assert texts(ew._join_clusters(line(f"{_STRANDED} x"))) == [_FLAG, _ZWJ, " ", "x"]
# A flag's two indicators are one glyph too, written as one: a cursor pin between the
# halves would leave a terminal drawing two letters instead of a flag.
# A selector pair, a toned emoji and a keycap are one glyph each, merged like the rest.
assert texts(ew._join_clusters(line(f"{_SUN}{_VS16}{_SHRUG}"))) == [f"{_SUN}{_VS16}", _SHRUG]
assert texts(ew._join_clusters(line(f"{_WARN}{_VS16}|"))) == [f"{_WARN}{_VS16}", "|"]
assert texts(ew._join_clusters(line(f"{_THUMB}{_KEYCAP}"))) == [_THUMB, _KEYCAP]
# A flag's two indicators are one glyph, written as one: a cursor pin between the halves
# would leave a terminal drawing two letters instead of a flag.
assert texts(ew._join_clusters(line(f"|{_CA}|"))) == ["|", _CA, "|"]
assert texts(ew._join_clusters(line(f"{_CA}{_CN}"))) == [_CA, _CN]
# A lone indicator is no pair, and stays a fragment of its own.
# A lone indicator is no pair, and a trailing or stranded joiner joins nothing.
assert texts(ew._join_clusters(line(f"{_CA[0]} x"))) == [_CA[0], " ", "x"]
assert texts(ew._join_clusters(line(f"a{_ZWJ}"))) == ["a", _ZWJ]
assert texts(ew._join_clusters(line(f"{_STRANDED} x"))) == [_FLAG, _ZWJ, " ", "x"]
# A merged fragment iterates as the whole sequence, which is what makes prompt_toolkit
# build one Char of it instead of one per codepoint.
# A merged fragment iterates as the whole glyph, which is what makes prompt_toolkit build
# one Char of it instead of one per codepoint.
(merged,) = texts(ew._join_clusters(line(_SHRUG)))
assert list(merged) == [_SHRUG] and merged == _SHRUG
def test_cluster_control_lays_a_sequence_into_a_single_screen_cell(monkeypatch) -> None:
"""The delivery half: one ``Char`` per glyph, with every codepoint still written out.
def test_cluster_control_lays_each_glyph_into_a_single_screen_cell(installed) -> None: # noqa: ANN001
"""The delivery half: one two-cell ``Char`` per glyph, every codepoint still written out.
Measuring the sequence right is not enough on its own — prompt_toolkit lays out one
codepoint at a time, so an uncorrected screen puts the male sign in a cell of its own and
everything after it a column late. The control merges the run, so the window makes a
single two-cell ``Char`` of it and the text after lands where the terminal draws it. The
codepoints themselves are untouched: all four still reach the screen, in order, so the
terminal composes the same glyph.
Measuring a glyph right is not enough on its own — prompt_toolkit lays out one codepoint at a
time, so an unmerged shrug puts its male sign in a cell of its own, and a warning mark with a
selector becomes a two-cell glyph in a one-cell slot. Merged, each is one ``Char`` two cells
wide, and the text after lands where the pins will put it.
"""
monkeypatch.delenv("MESHTERM_NARROW_EMOJI", raising=False)
monkeypatch.delenv("MESHTERM_WIDE_EMOJI", raising=False)
from prompt_toolkit.application import Application
from prompt_toolkit.application.current import set_app
from prompt_toolkit.formatted_text import ANSI
@@ -388,30 +369,18 @@ def test_cluster_control_lays_a_sequence_into_a_single_screen_cell(monkeypatch)
from prompt_toolkit.layout.screen import Screen, WritePosition
from prompt_toolkit.output import DummyOutput
text = f"|{_SHRUG}|{_FAMILY}|end"
warn = f"{_WARN}{_VS16}"
text = f"|{_SHRUG}|{_FAMILY}|{warn}|end"
window = Window(ew.ClusterTextControl(lambda: ANSI(text)), always_hide_cursor=True)
app = Application(layout=Layout(window), input=DummyInput(), output=DummyOutput())
with set_app(app):
screen = Screen(default_char=None, initial_width=40, initial_height=1)
window.write_to_screen(screen, MouseHandlers(), WritePosition(0, 0, 40, 1), "", False, None)
row = screen.data_buffer[0]
def paint() -> list:
window = Window(ew.ClusterTextControl(lambda: ANSI(text)), always_hide_cursor=True)
app = Application(layout=Layout(window), input=DummyInput(), output=DummyOutput())
with set_app(app):
screen = Screen(default_char=None, initial_width=40, initial_height=1)
window.write_to_screen(
screen, MouseHandlers(), WritePosition(0, 0, 40, 1), "", False, None
)
return screen.data_buffer[0]
snap = _snapshot()
try:
ew._CALIBRATED = False
ew.calibrate(force_width=1)
row = paint()
# Each sequence is one cell-occupying Char, two cells wide, exactly as drawn.
assert (row[1].char, row[1].width) == (_SHRUG, 2)
assert (row[4].char, row[4].width) == (_FAMILY, 2)
# So the text after them starts at column 6 rather than 13 — seven cells of border
# drift removed (one for the shrug, four for the family, two from the second bar).
assert "".join(row[x].char for x in range(6, 10)) == "|end"
# Nothing was dropped on the way: the row still spells the source exactly.
assert "".join(row[x].char for x in range(40)).rstrip() == text
finally:
_restore(snap)
assert (row[1].char, row[1].width) == (_SHRUG, 2)
assert (row[4].char, row[4].width) == (_FAMILY, 2)
assert (row[7].char, row[7].width) == (warn, 2)
assert "".join(row[x].char for x in range(9, 13)) == "|end"
# Nothing was dropped on the way: the row still spells the source exactly.
assert "".join(row[x].char for x in range(40)).rstrip() == text
+9 -7
View File
@@ -66,17 +66,19 @@ def test_declaring_a_signature_does_not_reach_the_shared_one() -> None:
assert private.GetConsoleScreenBufferInfo is not shared
def test_the_probe_leaves_the_shared_signature_alone() -> None:
"""End to end: running the emoji probe must not disturb the process-wide function.
def test_the_console_probe_leaves_the_shared_signature_alone() -> None:
"""End to end: a real call site declaring the function must not disturb the shared one.
The narrow assertion above can pass while the real call site still reaches for
``ctypes.windll``, so this exercises the code that actually crashed.
The narrow assertion above can pass while a call site still reaches for
``ctypes.windll``, so this exercises one. The emoji-width probe that crashed is gone, but
the classic-console check declares the very same ``GetConsoleScreenBufferInfo`` with a
struct of its own, which is the same crash waiting in a different module.
"""
from meshterm.ui.tui import emoji_width
from meshterm.ui import termfont
shared = ctypes.windll.kernel32.GetConsoleScreenBufferInfo
before = shared.argtypes
# No console under pytest, so this returns None rather than measuring. That is fine:
# No console under pytest, so this returns None rather than answering. That is fine:
# the declarations happen before the first call either way, which is what matters.
emoji_width._win_probe_width("x") # noqa: SLF001 - the point is this private path
termfont._is_classic_console() # noqa: SLF001 - the point is this private path
assert ctypes.windll.kernel32.GetConsoleScreenBufferInfo.argtypes == before