No log keeps a password or a secret: the log file, the run log, or the remote history
Passwords, channel secrets, pairing PINs, and private keys reached the DEBUG log and the runs table in plain text: every chat run stored its channel secret inside the Conversation repr, and --password, --secret, channel links, and config set device_pin were stored as typed. meshterm.core.secrets now hides each one as <secret> by its shape (a secret-named key, record, set operation, or dataclass field; bytes; a Secret; a remote `password`/`set guest.password` command; name=value in text), and by value for each secret MeshTerm has read or sent. Both log handlers and the runs table go through it, always. The remote CLI history keeps `password ` without its value. Schema v20 hides the secrets that older versions stored. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
9e579424b4
commit
1d983013d7
13 files changed
+896
-60
No files matched your search
@@ -115,6 +115,14 @@ radios from two directories in the same shell.
|
||||
| `meshterm.log` | The log file. |
|
||||
| `.lock` | The lock for a single instance. |
|
||||
|
||||
**No log keeps a password or a secret.** The log file, the run log in the database (the
|
||||
arguments and the result of each command), and the remote command history of
|
||||
`remote.json` write `<secret>` in the place of a password, a channel secret, a pairing
|
||||
PIN, or a private key. This is always true, at each log level and with or without the
|
||||
**Anonymize** preference. The history keeps `password ` without its value, so ↑ still
|
||||
recalls the start of the command. `admin.json` keeps the admin passwords that you asked
|
||||
MeshTerm to remember, because that is its purpose.
|
||||
|
||||
**`--db` alone does not isolate a run.** It moves the database. The database is most of
|
||||
the data, but it does not hold the identity. The contact caches, the channel caches, the
|
||||
outbox, the stored admin passwords, and the remembered device are all in the config
|
||||
|
||||
@@ -281,6 +281,8 @@ and in quoted text.
|
||||
| save | Do an explicit save: the user's Save, or the device's save command | |
|
||||
| record | One stored data item | |
|
||||
| delete | Remove stored data permanently | purge (except the Purge feature) |
|
||||
| secret | A value that unlocks something: a password, a channel secret, a pairing PIN, a private key. No log keeps one: it is hidden as `<secret>` | credential (except in a name) |
|
||||
| hide (a secret) | Replace a secret with `<secret>` in a log or in the run log | redact, scrub |
|
||||
| personal detail | A value that identifies the user or the equipment of the user, but is not a secret: the account name in a home folder, the address of a radio, the address of a host | PII, private data |
|
||||
| mask | Replace most characters of a personal detail, and keep its first and last characters: `j*********u`, `DA:xx:xx:xx:7C:7C` | redact, scrub, obfuscate |
|
||||
| anonymize | Mask each personal detail in a text: the `anonymize` preference does it to the log and the diagnostics | |
|
||||
|
||||
@@ -21,7 +21,8 @@ MeshTerm also writes each failure to the log file, `~/.meshterm/meshterm.log` (r
|
||||
[Where MeshTerm keeps its state](../cli/README.md#where-meshterm-keeps-its-state)). Attach
|
||||
the log file to a bug report. To get more detail, set the **Log detail** preference to
|
||||
`DEBUG` (`meshterm preferences set log_level DEBUG`), and run the command again. The log
|
||||
can show your account name and the address of your radio. To mask them as MeshTerm writes
|
||||
never holds a password or a secret, at any level. It can show your account name and the
|
||||
address of your radio. To mask them as MeshTerm writes
|
||||
each line, turn on the **Anonymize** preference (`meshterm preferences set anonymize on`).
|
||||
|
||||
Sometimes MeshTerm meets a failure that it has no name for. Then the message is the error
|
||||
|
||||
@@ -58,6 +58,7 @@ from .regions import WILDCARD as REGION_WILDCARD
|
||||
from .regions import normalize as normalize_region
|
||||
from .regions import parse_region_list, region_key
|
||||
from .regions import validate as validate_region
|
||||
from .secrets import remember as remember_secret
|
||||
from .tracing import path_hash_flags, trace_timeout
|
||||
from .transmit_lock import TransmitLock
|
||||
|
||||
@@ -2105,6 +2106,9 @@ class MeshCoreDevice(Device):
|
||||
from .anonymize import remember_host
|
||||
|
||||
remember_host(host)
|
||||
# The pairing PIN is a secret. No log may repeat it (refer to
|
||||
# meshterm.core.secrets), so it is registered before the first connect.
|
||||
remember_secret(pin)
|
||||
self._mc = None # type: ignore[var-annotated] # meshcore.MeshCore
|
||||
#: The ``bleak.BleakClient`` of a BLE link. MeshTerm keeps it because ``meshcore``
|
||||
#: releases its own reference at the moment the peripheral disappears. Refer to
|
||||
@@ -3197,7 +3201,9 @@ class MeshCoreDevice(Device):
|
||||
result = await mc.commands.send_device_query()
|
||||
except Exception: # noqa: BLE001 - older firmware has no query: unknown, not fatal
|
||||
return {}
|
||||
return dict(getattr(result, "payload", {}) or {})
|
||||
payload = dict(getattr(result, "payload", {}) or {})
|
||||
remember_secret(payload.get("ble_pin")) # the PIN with which a phone pairs
|
||||
return payload
|
||||
|
||||
#: The maximum silence of the contacts stream before MeshTerm considers a read as
|
||||
#: failed (seconds). The library's own ``get_contacts`` arms one future for the *full*
|
||||
@@ -3467,6 +3473,9 @@ class MeshCoreDevice(Device):
|
||||
"""
|
||||
from meshcore import EventType
|
||||
|
||||
# Each password that MeshTerm sends is a secret that no log may repeat (refer to
|
||||
# meshterm.core.secrets): an admin password, a room password, or a guess.
|
||||
remember_secret(password)
|
||||
mc = self._require()
|
||||
pub = self._node_pubkey(node)
|
||||
loop = asyncio.get_running_loop()
|
||||
@@ -4384,6 +4393,7 @@ class MeshCoreDevice(Device):
|
||||
raise DeviceCommandError(f"channel read for slot {index} returned slot {got}")
|
||||
if not payload.get("channel_name"):
|
||||
return None
|
||||
remember_secret(payload.get("channel_secret"))
|
||||
return payload
|
||||
|
||||
async def _channel_event(self, index: int): # noqa: ANN202 - a meshcore Event
|
||||
@@ -4573,6 +4583,7 @@ class MeshCoreDevice(Device):
|
||||
key = payload.get("private_key")
|
||||
if key is None:
|
||||
raise RuntimeError("device did not return a private key (export may be disabled)")
|
||||
remember_secret(key)
|
||||
return key.hex() if isinstance(key, (bytes, bytearray)) else str(key)
|
||||
|
||||
async def import_private_key(self, key_hex: str) -> None: # noqa: D102
|
||||
|
||||
@@ -145,7 +145,7 @@ def _parse_typed(spec: SettingSpec, raw: Any, snapshot: dict | None) -> Any:
|
||||
return True
|
||||
if low in _FALSE:
|
||||
return False
|
||||
raise DeviceConfigError(f"{spec.key}: expected a boolean, got {raw!r}")
|
||||
raise DeviceConfigError(f"{spec.key}: expected a boolean{_got(spec, repr(raw))}")
|
||||
|
||||
# Numeric (int / enum / float): the try wraps only the conversion. Thus this handler
|
||||
# does not catch and wrap again a DeviceConfigError from the range and enum checks
|
||||
@@ -158,7 +158,9 @@ def _parse_typed(spec: SettingSpec, raw: Any, snapshot: dict | None) -> Any:
|
||||
else: # int or enum
|
||||
value = int(text, 0) if isinstance(raw, str) else int(raw)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise DeviceConfigError(f"{spec.key}: invalid {spec.value_type} value {raw!r}") from exc
|
||||
raise DeviceConfigError(
|
||||
f"{spec.key}: invalid {spec.value_type} value{_got(spec, repr(raw), before=' ')}"
|
||||
) from exc
|
||||
|
||||
if (
|
||||
spec.value_type == "enum"
|
||||
@@ -167,15 +169,35 @@ def _parse_typed(spec: SettingSpec, raw: Any, snapshot: dict | None) -> Any:
|
||||
and value not in spec.choices
|
||||
):
|
||||
allowed = ", ".join(str(k) for k in spec.choices)
|
||||
raise DeviceConfigError(f"{spec.key}: must be one of {allowed}, got {value}")
|
||||
raise DeviceConfigError(f"{spec.key}: must be one of {allowed}{_got(spec, value)}")
|
||||
if spec.minimum is not None and value < spec.minimum:
|
||||
raise DeviceConfigError(f"{spec.key}: must be >= {spec.minimum}, got {value}")
|
||||
raise DeviceConfigError(f"{spec.key}: must be >= {spec.minimum}{_got(spec, value)}")
|
||||
maximum = effective_maximum(spec, snapshot)
|
||||
if maximum is not None and value > maximum:
|
||||
raise DeviceConfigError(f"{spec.key}: must be <= {maximum:g}, got {value}")
|
||||
raise DeviceConfigError(f"{spec.key}: must be <= {maximum:g}{_got(spec, value)}")
|
||||
return value
|
||||
|
||||
|
||||
def _got(spec: SettingSpec, shown: Any, *, before: str = ", got ") -> str:
|
||||
"""The end of an error message that repeats the refused value, or nothing for a secret.
|
||||
|
||||
An error goes to the log and to the run log, and the value of a secret setting (the
|
||||
pairing PIN) must not (refer to :mod:`meshterm.core.secrets`). Thus the message of a
|
||||
secret setting states the rule and not the value.
|
||||
|
||||
Args:
|
||||
spec: The setting that refused the value.
|
||||
shown: The refused value, as the message writes it.
|
||||
before: The text before the value: ``", got "``, or ``" "`` for "value X".
|
||||
|
||||
Returns:
|
||||
``", got 1234"``, ``" '12a'"``, or ``""`` for a secret setting.
|
||||
"""
|
||||
from .secrets import names_secret
|
||||
|
||||
return "" if names_secret(spec.key) else f"{before}{shown}"
|
||||
|
||||
|
||||
def effective_maximum(spec: SettingSpec, snapshot: dict | None) -> float | None:
|
||||
"""The inclusive maximum for ``spec``: the device value if it is known, else the static one.
|
||||
|
||||
@@ -430,7 +452,8 @@ def _pin_rule(value: Any, snapshot: dict | None) -> str | None:
|
||||
"""The firmware accepts only a pairing PIN of zero (no PIN) or of exactly six digits."""
|
||||
if value == 0 or 100000 <= value <= 999999:
|
||||
return None
|
||||
return f"must be 0 (no PIN) or six digits, got {value}"
|
||||
# No "got": the PIN is a secret, and the message goes to the log (refer to _got).
|
||||
return "must be 0 (no PIN) or six digits"
|
||||
|
||||
|
||||
def _repeat_rule(value: Any, snapshot: dict | None) -> str | None:
|
||||
|
||||
@@ -28,6 +28,8 @@ from typing import ClassVar
|
||||
from .admin_store import admin_key
|
||||
from .atomicwrite import write_atomically
|
||||
from .models import Contact, utcnow
|
||||
from .secrets import names_secret, without_secret
|
||||
from .secrets import remember as remember_secret
|
||||
|
||||
#: How many command-line entries the store keeps for each node (newest last).
|
||||
HISTORY_CAP = 100
|
||||
@@ -121,7 +123,11 @@ class _RemoteNode:
|
||||
if value is not None:
|
||||
settings[str(key)] = value
|
||||
raw_history = record.get("history")
|
||||
history = [str(c) for c in raw_history] if isinstance(raw_history, list) else []
|
||||
# A history that an earlier version wrote can hold a password. The record keeps
|
||||
# the command without it, so the next write removes it from the file.
|
||||
history = (
|
||||
[without_secret(str(c)) for c in raw_history] if isinstance(raw_history, list) else []
|
||||
)
|
||||
return cls(settings=settings, history=history)
|
||||
|
||||
@property
|
||||
@@ -169,11 +175,20 @@ class RemoteStore:
|
||||
return dict(self._node(node).settings)
|
||||
|
||||
def remember_setting(self, node: Contact, key: str, value: str) -> None:
|
||||
"""Cache the value of one setting for ``node``, with the time now."""
|
||||
"""Cache the value of one setting for ``node``, with the time now.
|
||||
|
||||
The value of a setting that is a secret (``guest.password``, ``bridge.secret``) is
|
||||
also registered, so that no log can repeat it (refer to
|
||||
:mod:`meshterm.core.secrets`).
|
||||
"""
|
||||
if names_secret(key):
|
||||
remember_secret(value)
|
||||
self._put(node, key, CachedValue(value=value, read_at=utcnow()))
|
||||
|
||||
def remember_discovered(self, node: Contact, key: str, value: str) -> None:
|
||||
"""Cache a setting that is not in the catalog, which ``node`` proved that it has."""
|
||||
if names_secret(key):
|
||||
remember_secret(value)
|
||||
self._put(node, key, CachedValue(value=value, read_at=utcnow(), discovered=True))
|
||||
|
||||
def remember_unsupported(self, node: Contact, key: str) -> None:
|
||||
@@ -209,10 +224,17 @@ class RemoteStore:
|
||||
return list(self._node(node).history)
|
||||
|
||||
def append_history(self, node: Contact, command: str) -> None:
|
||||
"""Add one sent command to the history of the node (not a repeat of the last command)."""
|
||||
"""Add one sent command to the history of the node (not a repeat of the last command).
|
||||
|
||||
A command that sets a password or a secret is kept without its value
|
||||
(:func:`~meshterm.core.secrets.without_secret`): ``password hunter2`` is kept as
|
||||
``password``, with the space after it. Thus ↑ recalls the start of the command, and
|
||||
the file never holds the secret.
|
||||
"""
|
||||
command = command.strip()
|
||||
if not command:
|
||||
return
|
||||
command = without_secret(command)
|
||||
records = self._load()
|
||||
record = records.setdefault(admin_key(node), _RemoteNode())
|
||||
if record.history and record.history[-1] == command:
|
||||
|
||||
@@ -0,0 +1,383 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Keep each password and each secret out of each log: the log file and the run log.
|
||||
|
||||
MeshTerm keeps a record of what it does in two places: the log file (``meshterm.log``) and
|
||||
the ``runs`` table, which stores the parameters and the summary of each run. Neither place
|
||||
may ever hold a password, a channel secret, a pairing PIN, or a private key (JP,
|
||||
2026-10-18: "NEVER log passwords or secrets!"). Each one is written as :data:`SECRET_TEXT`
|
||||
instead. This rule does not depend on a preference, and the ``anonymize`` preference
|
||||
(refer to :mod:`meshterm.core.anonymize`) adds to it and does not replace it.
|
||||
|
||||
A secret gets into a record in many shapes, and this module has one rule for each shape.
|
||||
No rule depends on a list of the tools or of their parameters. Thus a tool that is added
|
||||
later is covered on the day that it arrives:
|
||||
|
||||
- **A named value.** A value is a secret when its name says so (:func:`names_secret`):
|
||||
a mapping key (``{"password": …}``), a dataclass field (``Conversation.secret``), the
|
||||
``key`` of a record (``{"key": "device_pin", "value": …}``), the name in an operation
|
||||
(``("set", "device_pin", "123456")``), or the name before ``=`` in a text
|
||||
(``…&secret=8f…``, ``secret=b'…'``).
|
||||
- **A wrapped value.** A tool can wrap a value in :class:`Secret` when nothing names it.
|
||||
- **Bytes.** The parameters of a run carry bytes only for keys and secrets. Thus each
|
||||
``bytes`` value is a secret.
|
||||
- **A remote command.** ``password <new>`` and ``set guest.password <new>`` carry the
|
||||
secret as their last word.
|
||||
- **A known value in free text.** Each value that a rule above hid, and each secret that
|
||||
MeshTerm reads (a stored password, a channel secret, a pairing PIN), is registered
|
||||
(:func:`remember`). :func:`hide` then finds it in any text: an error message that
|
||||
repeats it, or a frame that a library writes in hex.
|
||||
|
||||
:func:`loggable` applies the rules to a structure (the parameters and the summary of a
|
||||
run). :func:`hide` applies them to a text (a log record).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import threading
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import fields, is_dataclass
|
||||
from datetime import date, datetime
|
||||
from enum import Enum
|
||||
from pathlib import PurePath
|
||||
from typing import Any
|
||||
|
||||
#: What a log and the run log show in the place of a secret.
|
||||
SECRET_TEXT = "<secret>"
|
||||
|
||||
#: The words that make a name the name of a secret. A name matches when one of its words
|
||||
#: is in this set. The words of a name are the parts between the characters that are not
|
||||
#: letters or digits. Thus ``device_pin``, ``guest.password``, and ``bridge.secret`` match,
|
||||
#: and ``pinned`` and ``spin`` do not. ``prv`` is the private key of a repeater
|
||||
#: (``prv.key``). ``key`` alone is not a secret word, because most keys in MeshTerm are
|
||||
#: public keys.
|
||||
SECRET_WORDS = frozenset(
|
||||
{
|
||||
"password",
|
||||
"passwd",
|
||||
"passphrase",
|
||||
"secret",
|
||||
"pin",
|
||||
"private",
|
||||
"privkey",
|
||||
"prv",
|
||||
"psk",
|
||||
"token",
|
||||
"credential",
|
||||
"credentials",
|
||||
}
|
||||
)
|
||||
|
||||
#: The entries of a record whose ``key`` names a secret, and which hold its value.
|
||||
_RECORD_VALUES = frozenset({"value", "previous", "default", "old", "new"})
|
||||
|
||||
#: The entries of a record that name the thing that the record is about.
|
||||
_RECORD_NAMES = ("key", "setting")
|
||||
|
||||
#: The shortest text that :func:`remember` registers. A shorter value matches too much
|
||||
#: ordinary text in a log (``abc``, ``123``). The structural rules still hide it where it
|
||||
#: has a name.
|
||||
_MIN_REMEMBERED = 4
|
||||
|
||||
#: The shortest hex value that :func:`hide` finds inside a longer run of hex. A channel
|
||||
#: secret has 32 hex digits. With 16 or more, a match by chance is not probable.
|
||||
_MIN_HEX_SUBSTRING = 16
|
||||
|
||||
_WORD_SPLIT = re.compile(r"[^a-z0-9]+")
|
||||
_HEX = re.compile(r"[0-9a-fA-F]+")
|
||||
|
||||
#: ``name = value`` in a text: a URL query, a ``repr`` of an object, or a ``key=value``
|
||||
#: log line. A name with no quotes takes ``=``. A quoted name (a ``repr`` of a dict, or
|
||||
#: JSON) also takes ``:``. Prose ("device_pin: must be six digits") is not an assignment.
|
||||
#: The value is a quoted literal (also a ``bytes`` literal), or a run of characters up to
|
||||
#: the end of the value.
|
||||
_ASSIGNMENT = re.compile(
|
||||
r"(?P<name>(?<![\w.-])[A-Za-z_][\w.-]*)"
|
||||
r"(?P<sep>\s*=\s*|['\"]\s*[=:]\s*)"
|
||||
r"(?P<value>b?'(?:[^'\\]|\\.)*'|b?\"(?:[^\"\\]|\\.)*\"|<secret>|[^\s'\",;)}\]&<>]+)"
|
||||
)
|
||||
|
||||
#: A remote command that sets a password: ``password <new>``.
|
||||
_PASSWORD_COMMAND = re.compile(r"^(?P<head>\s*password\s+)(?P<value>\S.*)$", re.IGNORECASE)
|
||||
|
||||
#: A remote command that sets a setting: ``set <name> <value>``.
|
||||
_SET_COMMAND = re.compile(r"^(?P<head>\s*set\s+(?P<name>\S+)\s+)(?P<value>\S.*)$", re.IGNORECASE)
|
||||
|
||||
_known: set[str] = set()
|
||||
_known_pattern: re.Pattern[str] | None = None
|
||||
_known_lock = threading.Lock()
|
||||
|
||||
|
||||
class Secret:
|
||||
"""A value that no log may keep, for example a private key or a password.
|
||||
|
||||
:meth:`~meshterm.tools.base.Tool.execute` stores the parameters of each run in the
|
||||
``runs`` table, and logs them at debug level. A tool that has a secret that nothing
|
||||
names (a key in a tuple) puts it in this wrapper. :func:`loggable` and each text form
|
||||
of this object show :data:`SECRET_TEXT`, and the tool reads :attr:`value` (refer to
|
||||
:func:`reveal`).
|
||||
|
||||
The class does not derive from ``str``, because ``json.dumps`` writes the value of a
|
||||
``str`` subclass and never calls its ``default`` for it.
|
||||
|
||||
Attributes:
|
||||
value: The secret itself.
|
||||
"""
|
||||
|
||||
__slots__ = ("value",)
|
||||
|
||||
def __init__(self, value: str) -> None:
|
||||
"""Keep ``value`` out of each text form of this object, and register it."""
|
||||
self.value = value
|
||||
remember(value)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
""":data:`SECRET_TEXT`, never the value."""
|
||||
return SECRET_TEXT
|
||||
|
||||
__str__ = __repr__
|
||||
|
||||
|
||||
def reveal(value: Any) -> Any:
|
||||
"""The value inside a :class:`Secret`, or ``value`` itself if it is not a secret."""
|
||||
return value.value if isinstance(value, Secret) else value
|
||||
|
||||
|
||||
def names_secret(name: str) -> bool:
|
||||
"""Whether ``name`` is the name of a secret (refer to :data:`SECRET_WORDS`).
|
||||
|
||||
Args:
|
||||
name: A parameter name, a field name, a setting key, or a word of a text.
|
||||
|
||||
Returns:
|
||||
``True`` when one of the words of ``name`` is a secret word.
|
||||
"""
|
||||
return not SECRET_WORDS.isdisjoint(_WORD_SPLIT.split(name.lower()))
|
||||
|
||||
|
||||
def remember(value: Any) -> None:
|
||||
"""Register a secret, so that :func:`hide` finds it in any text after this call.
|
||||
|
||||
A value of bytes registers as hex, the form in which a log writes bytes. A value that
|
||||
is too short to find safely, or that is a secret word (a password of ``password``), is
|
||||
not registered: each mention of the word in the log would become the mark. Each other
|
||||
rule still hides such a value where it has a name.
|
||||
|
||||
Args:
|
||||
value: The secret: text, a number (a PIN), or bytes.
|
||||
"""
|
||||
global _known_pattern
|
||||
if value is None or isinstance(value, bool):
|
||||
return
|
||||
if isinstance(value, (bytes, bytearray, memoryview)):
|
||||
text = bytes(value).hex()
|
||||
else:
|
||||
text = str(value).strip()
|
||||
if len(text) < _MIN_REMEMBERED or text == SECRET_TEXT or text.lower() in SECRET_WORDS:
|
||||
return
|
||||
with _known_lock:
|
||||
if text in _known:
|
||||
return
|
||||
_known.add(text)
|
||||
_known_pattern = _compile_known(_known)
|
||||
|
||||
|
||||
def hide(text: str) -> str:
|
||||
"""Replace each secret in ``text`` with :data:`SECRET_TEXT`.
|
||||
|
||||
Args:
|
||||
text: A log record, an error message, a remote command, or a URL.
|
||||
|
||||
Returns:
|
||||
The text with each secret that a rule finds replaced.
|
||||
"""
|
||||
match = _PASSWORD_COMMAND.match(text)
|
||||
if match is not None:
|
||||
_remember_text(match.group("value"))
|
||||
return match.group("head") + SECRET_TEXT
|
||||
match = _SET_COMMAND.match(text)
|
||||
if match is not None and names_secret(match.group("name")):
|
||||
_remember_text(match.group("value"))
|
||||
return match.group("head") + SECRET_TEXT
|
||||
text = _ASSIGNMENT.sub(_hide_assignment, text)
|
||||
known = _known_pattern
|
||||
if known is not None:
|
||||
text = known.sub(SECRET_TEXT, text)
|
||||
return text
|
||||
|
||||
|
||||
def without_secret(command: str) -> str:
|
||||
"""A remote command with its secret value removed, for a history to recall.
|
||||
|
||||
The history of the remote command line keeps ``password `` for ``password hunter2``.
|
||||
Thus ↑ recalls the start of the command, and the user types the new value again. If
|
||||
the history kept :data:`SECRET_TEXT`, a recall and Enter would set the password to
|
||||
that text.
|
||||
|
||||
Args:
|
||||
command: A command as the user typed it.
|
||||
|
||||
Returns:
|
||||
The command up to its secret value, or the command itself when it has none.
|
||||
"""
|
||||
for pattern in (_PASSWORD_COMMAND, _SET_COMMAND):
|
||||
match = pattern.match(command)
|
||||
if match is not None and (pattern is _PASSWORD_COMMAND or names_secret(match["name"])):
|
||||
return match.group("head")
|
||||
return command
|
||||
|
||||
|
||||
def loggable(value: Any) -> Any:
|
||||
"""A copy of ``value`` that a log can keep: each secret in it is :data:`SECRET_TEXT`.
|
||||
|
||||
The copy has the same shape: mappings stay mappings, and lists and tuples keep their
|
||||
types. A dataclass becomes an object whose text form is the ``repr`` of the dataclass
|
||||
with its secret fields hidden, so the log line looks as it did. Each text goes through
|
||||
:func:`hide`. Each value that a rule hides is registered (:func:`remember`). Thus the
|
||||
same value is also hidden when an error message repeats it later in the run.
|
||||
|
||||
Args:
|
||||
value: The parameters or the summary of a run, or any part of them.
|
||||
|
||||
Returns:
|
||||
The copy, ready for a log line or for ``json.dumps(..., default=str)``.
|
||||
"""
|
||||
if isinstance(value, Secret):
|
||||
remember(value.value)
|
||||
return SECRET_TEXT
|
||||
if isinstance(value, (bytes, bytearray, memoryview)):
|
||||
return _hidden(value)
|
||||
if isinstance(value, str):
|
||||
return hide(value)
|
||||
if value is None or isinstance(value, (bool, int, float, Enum, PurePath, date, datetime)):
|
||||
return value
|
||||
if isinstance(value, Mapping):
|
||||
return _loggable_mapping(value)
|
||||
if is_dataclass(value) and not isinstance(value, type):
|
||||
items = []
|
||||
for f in fields(value):
|
||||
item = getattr(value, f.name)
|
||||
items.append((f.name, _hidden(item) if names_secret(f.name) else loggable(item)))
|
||||
return _Shown(type(value).__name__, items)
|
||||
if isinstance(value, (list, tuple, set, frozenset)):
|
||||
items = _loggable_sequence(list(value))
|
||||
return tuple(items) if isinstance(value, tuple) else items
|
||||
return _Shown(hide(repr(value)))
|
||||
|
||||
|
||||
class _Shown:
|
||||
"""An object in a log, with its secrets hidden: its ``repr`` and its ``str`` are the same.
|
||||
|
||||
``json.dumps(..., default=str)`` writes an object as its ``str``, and a log line writes
|
||||
a value in a dict as its ``repr``. Both forms of the original object can hold the
|
||||
secret, so both forms of this object are the hidden form.
|
||||
"""
|
||||
|
||||
__slots__ = ("_text",)
|
||||
|
||||
def __init__(self, name: str, items: list[tuple[str, Any]] | None = None) -> None:
|
||||
"""Build the form ``Name(a=1, b=<secret>)``, or keep ``name`` as the whole text."""
|
||||
if items is None:
|
||||
self._text = name
|
||||
else:
|
||||
inner = ", ".join(f"{field}={_repr(value)}" for field, value in items)
|
||||
self._text = f"{name}({inner})"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
"""The hidden form."""
|
||||
return self._text
|
||||
|
||||
__str__ = __repr__
|
||||
|
||||
|
||||
def _repr(value: Any) -> str:
|
||||
"""The ``repr`` of a value in a hidden form, with :data:`SECRET_TEXT` not in quotes."""
|
||||
return SECRET_TEXT if value == SECRET_TEXT else repr(value)
|
||||
|
||||
|
||||
def _hidden(value: Any) -> Any:
|
||||
""":data:`SECRET_TEXT` for a value that a name or a type makes secret, after it is registered.
|
||||
|
||||
An absent value (``None``, or an empty text) stays as it is. That a password was not
|
||||
given is a fact that a maintainer can use, and it hides nothing.
|
||||
"""
|
||||
if value is None or value == "" or value == b"":
|
||||
return value
|
||||
remember(reveal(value))
|
||||
return SECRET_TEXT
|
||||
|
||||
|
||||
def _loggable_mapping(value: Mapping) -> dict:
|
||||
""":func:`loggable` for a mapping: hide each value that its key or its record names secret."""
|
||||
record_names = [value.get(name) for name in _RECORD_NAMES]
|
||||
secret_record = any(isinstance(name, str) and names_secret(name) for name in record_names)
|
||||
shown: dict = {}
|
||||
for key, item in value.items():
|
||||
named = names_secret(str(key)) or (secret_record and key in _RECORD_VALUES)
|
||||
shown[key] = _hidden(item) if named else loggable(item)
|
||||
return shown
|
||||
|
||||
|
||||
def _loggable_sequence(items: list) -> list:
|
||||
""":func:`loggable` for a sequence, and for an operation that sets a secret.
|
||||
|
||||
An operation that sets a value names the setting before the value:
|
||||
``("set", "device_pin", "123456")``. Thus, in a sequence of three items that starts
|
||||
with ``set`` and whose second item names a secret, the third item is the secret. No
|
||||
other sequence has a value after a name. A list of setting names
|
||||
(``["name", "device_pin", "radio_freq"]``) stays as it is.
|
||||
"""
|
||||
shown = [loggable(item) for item in items]
|
||||
verb, name = (items[0], items[1]) if len(items) == 3 else (None, None)
|
||||
if isinstance(verb, str) and verb.lower() == "set" and isinstance(name, str):
|
||||
if names_secret(name):
|
||||
shown[2] = _hidden(items[2])
|
||||
return shown
|
||||
|
||||
|
||||
def _hide_assignment(match: re.Match[str]) -> str:
|
||||
"""Hide the value of ``name = value`` when the name is the name of a secret."""
|
||||
value = match.group("value")
|
||||
if not names_secret(match.group("name")) or value in (SECRET_TEXT, "None", "null"):
|
||||
return match.group(0)
|
||||
if value in ("''", '""', "b''", 'b""'):
|
||||
return match.group(0)
|
||||
_remember_text(value)
|
||||
return match.group("name") + match.group("sep") + SECRET_TEXT
|
||||
|
||||
|
||||
def _remember_text(value: str) -> None:
|
||||
"""Register a value that a text rule found, without the quotes of a literal."""
|
||||
text = value.strip()
|
||||
if len(text) >= 2 and text[-1] in "'\"" and text.lstrip("b")[:1] == text[-1]:
|
||||
text = text.lstrip("b")[1:-1]
|
||||
remember(text)
|
||||
|
||||
|
||||
def _compile_known(known: set[str]) -> re.Pattern[str]:
|
||||
"""The pattern that finds each registered secret.
|
||||
|
||||
A long hex value matches also inside a longer run of hex, in any letter case, because a
|
||||
library writes a whole frame as one run of hex. Each other value matches only as a
|
||||
whole word, so a PIN of ``123456`` does not hide a part of ``71234567``.
|
||||
"""
|
||||
parts: list[str] = []
|
||||
for text in sorted(known, key=len, reverse=True):
|
||||
if len(text) >= _MIN_HEX_SUBSTRING and _HEX.fullmatch(text):
|
||||
parts.append(f"(?i:{re.escape(text)})")
|
||||
else:
|
||||
parts.append(rf"(?<![\w]){re.escape(text)}(?![\w])")
|
||||
return re.compile("|".join(parts))
|
||||
|
||||
|
||||
__all__ = [
|
||||
"SECRET_TEXT",
|
||||
"SECRET_WORDS",
|
||||
"Secret",
|
||||
"hide",
|
||||
"loggable",
|
||||
"names_secret",
|
||||
"remember",
|
||||
"reveal",
|
||||
"without_secret",
|
||||
]
|
||||
@@ -12,7 +12,7 @@ import json
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
SCHEMA_VERSION = 19
|
||||
SCHEMA_VERSION = 20
|
||||
|
||||
_SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS schema_meta (
|
||||
@@ -404,6 +404,37 @@ def _migrate(conn: sqlite3.Connection) -> None:
|
||||
# that the run log writes as "<secret>" (refer to meshterm.tools.base.Secret).
|
||||
# This step removes the keys that the earlier versions stored, one time.
|
||||
_forget_imported_keys(conn)
|
||||
if stored is not None and int(stored["value"]) < 20:
|
||||
# v19 -> v20: no secret of any kind in the run log. The parameters and summaries
|
||||
# of the runs held channel secrets (in each chat run, inside its Conversation),
|
||||
# room passwords, admin passwords, and pairing PINs. The run log now stores each
|
||||
# one as "<secret>" (refer to meshterm.core.secrets). This step hides the secrets
|
||||
# that the earlier versions stored, one time.
|
||||
_hide_stored_secrets(conn)
|
||||
|
||||
|
||||
def _hide_stored_secrets(conn: sqlite3.Connection) -> None:
|
||||
"""Hide each secret in the stored arguments and summaries of the runs.
|
||||
|
||||
The rules are the rules that the run log applies to a new run
|
||||
(:func:`~meshterm.core.secrets.loggable`). The stored JSON has lost the types of the
|
||||
objects: a ``Conversation`` is the text of its ``repr``. The text rules find the
|
||||
secret in that text (``secret=b'...'``). A row that does not change is not written.
|
||||
"""
|
||||
from ..core.secrets import loggable
|
||||
|
||||
rows = conn.execute("SELECT id, args_json, summary_json FROM runs").fetchall()
|
||||
for row in rows:
|
||||
for column in ("args_json", "summary_json"):
|
||||
stored = row[column]
|
||||
if not stored:
|
||||
continue
|
||||
try:
|
||||
hidden = json.dumps(loggable(json.loads(stored)), default=str)
|
||||
except ValueError:
|
||||
continue
|
||||
if hidden != json.dumps(json.loads(stored)):
|
||||
conn.execute(f"UPDATE runs SET {column} = ? WHERE id = ?", (hidden, row["id"]))
|
||||
|
||||
|
||||
def _forget_imported_keys(conn: sqlite3.Connection) -> None:
|
||||
|
||||
@@ -19,7 +19,11 @@ A preference sets how much MeshTerm writes: ``log_level``, with the default ``WA
|
||||
Thus the file has the problems, not a full account of a session that works. When you
|
||||
look for the cause of a problem, set it lower, to ``INFO`` or ``DEBUG``.
|
||||
|
||||
A second preference, ``anonymize``, masks the personal details of each line before it
|
||||
No log ever holds a password or a secret. Both handlers hide each secret that
|
||||
:mod:`meshterm.core.secrets` finds in a record, and the file formatter does it again on
|
||||
the traceback. This does not depend on a preference.
|
||||
|
||||
A preference, ``anonymize``, masks the personal details of each line before it
|
||||
goes into the file: the account name in a home folder, the address of a radio, and the
|
||||
address of a host (refer to :mod:`meshterm.core.anonymize`). The mask applies as each line
|
||||
is written, so the file never holds what it masks. A line that was written before the
|
||||
@@ -142,6 +146,7 @@ def configure_logging(
|
||||
console=console, rich_tracebacks=True, show_path=False, markup=False
|
||||
)
|
||||
rich_handler.setLevel(level)
|
||||
rich_handler.addFilter(_HIDE_SECRETS)
|
||||
logger.addHandler(rich_handler)
|
||||
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -152,6 +157,7 @@ def configure_logging(
|
||||
encoding="utf-8",
|
||||
)
|
||||
file_handler.setLevel(file_level)
|
||||
file_handler.addFilter(_HIDE_SECRETS)
|
||||
file_handler.setFormatter(_FileFormatter(_LINE_FORMAT, datefmt=_TIME_FORMAT))
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
@@ -160,23 +166,65 @@ def configure_logging(
|
||||
return logger
|
||||
|
||||
|
||||
class _FileFormatter(logging.Formatter):
|
||||
"""The line format of the file, with the personal details masked when the user asks.
|
||||
class _HideSecrets(logging.Filter):
|
||||
"""Hide each password and secret in the message of a record, for every handler.
|
||||
|
||||
The formatter masks the whole formatted text: the message, and the traceback under
|
||||
it. A traceback holds the most home folders, because each frame names its file. The
|
||||
formatter reads the ``anonymize`` preference at each record, so a change on the
|
||||
Preferences page applies to the next line. The mask is a function of the text only,
|
||||
so it does not change the record. The console handler gets the same record and shows
|
||||
it as it is, because the console belongs to the user who reads it.
|
||||
A filter on a handler sees each record that the handler gets, also the records of a
|
||||
child logger and of the libraries in :data:`_LIBRARY_LOGGERS`. A filter on a logger
|
||||
sees only the records of that logger. The filter writes the hidden message back into
|
||||
the record, so each handler after it gets the hidden form. A second pass finds nothing
|
||||
more, so the order of the handlers does not matter.
|
||||
|
||||
The text of an exception can repeat a secret too. The console handler draws a
|
||||
traceback from the exception itself, and not from text that a filter can change. Thus,
|
||||
when the traceback of a record holds a secret, the filter adds the hidden traceback to
|
||||
the message as text and removes the exception from the record. Only such a record
|
||||
loses the drawn traceback of the console.
|
||||
"""
|
||||
|
||||
def filter(self, record: logging.LogRecord) -> bool:
|
||||
"""Replace the message of ``record`` with its hidden form, and keep the record."""
|
||||
from ..core.secrets import hide
|
||||
|
||||
try:
|
||||
message = record.getMessage()
|
||||
except Exception: # noqa: BLE001 - a bad format is the problem of the handler
|
||||
return True
|
||||
hidden = hide(message)
|
||||
if record.exc_info and record.exc_info[0] is not None:
|
||||
traceback = logging.Formatter().formatException(record.exc_info)
|
||||
hidden_traceback = hide(traceback)
|
||||
if hidden_traceback != traceback:
|
||||
hidden = f"{hidden}\n{hidden_traceback}"
|
||||
record.exc_info, record.exc_text = None, None
|
||||
if hidden != message or record.args:
|
||||
record.msg, record.args = hidden, None
|
||||
return True
|
||||
|
||||
|
||||
_HIDE_SECRETS = _HideSecrets()
|
||||
|
||||
|
||||
class _FileFormatter(logging.Formatter):
|
||||
"""The line format of the file, with no secret, and anonymized when the user asks.
|
||||
|
||||
The formatter works on the whole formatted text: the message, and the traceback under
|
||||
it. :class:`_HideSecrets` already hid the message. The text of an exception in a
|
||||
traceback can repeat a secret too, so the formatter hides secrets again here, always.
|
||||
Then it masks the personal details if the ``anonymize`` preference is on. A traceback
|
||||
holds the most home folders, because each frame names its file. The formatter reads
|
||||
the preference at each record, so a change on the Preferences page applies to the
|
||||
next line. The mask does not change the record. The console handler gets the same
|
||||
record and shows it unmasked, because the console belongs to the user who reads it.
|
||||
"""
|
||||
|
||||
def format(self, record: logging.LogRecord) -> str:
|
||||
"""Format ``record`` as one line (and its traceback), masked if ``anonymize`` is on."""
|
||||
"""Format ``record`` as one line (and its traceback), with no secret in it."""
|
||||
from ..core.anonymize import anonymize
|
||||
from ..core.preferences import current
|
||||
from ..core.secrets import hide
|
||||
|
||||
text = super().format(record)
|
||||
text = hide(super().format(record))
|
||||
return anonymize(text) if current().anonymize else text
|
||||
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ from ..core.models import (
|
||||
utcnow,
|
||||
)
|
||||
from ..core.regions import SCOPED_ROUTES, raw_scope_body
|
||||
from ..core.secrets import loggable
|
||||
from . import db
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -371,6 +372,10 @@ class Repository:
|
||||
) -> int:
|
||||
"""Store the start of a tool run.
|
||||
|
||||
The run log never keeps a secret. The arguments go through
|
||||
:func:`~meshterm.core.secrets.loggable`, so a password, a channel secret, or a PIN
|
||||
is stored as ``<secret>``.
|
||||
|
||||
Args:
|
||||
tool: The name of the tool.
|
||||
args: The arguments with which the tool was called (they must be
|
||||
@@ -382,10 +387,11 @@ class Repository:
|
||||
Returns:
|
||||
The primary key of the new run.
|
||||
"""
|
||||
stored = json.dumps(loggable(args), default=str)
|
||||
cur = self._conn.execute(
|
||||
"INSERT INTO runs (tool, profile, args_json, status, started_at) "
|
||||
"VALUES (?, ?, ?, 'running', ?)",
|
||||
(tool, profile, json.dumps(args, default=str), (at or utcnow()).isoformat()),
|
||||
(tool, profile, stored, (at or utcnow()).isoformat()),
|
||||
)
|
||||
self._conn.commit()
|
||||
return int(cur.lastrowid)
|
||||
@@ -400,6 +406,9 @@ class Repository:
|
||||
) -> None:
|
||||
"""Mark a run as finished.
|
||||
|
||||
The summary goes through :func:`~meshterm.core.secrets.loggable`, as the arguments
|
||||
do in :meth:`start_run`. An error message that repeats a secret is hidden too.
|
||||
|
||||
Args:
|
||||
run_id: The run to update.
|
||||
status: The final status (``ok`` or ``error``).
|
||||
@@ -410,7 +419,7 @@ class Repository:
|
||||
"UPDATE runs SET status = ?, summary_json = ?, finished_at = ? WHERE id = ?",
|
||||
(
|
||||
status,
|
||||
json.dumps(summary, default=str) if summary is not None else None,
|
||||
json.dumps(loggable(summary), default=str) if summary is not None else None,
|
||||
(at or utcnow()).isoformat(),
|
||||
run_id,
|
||||
),
|
||||
|
||||
+10
-35
@@ -16,6 +16,13 @@ from typing import TYPE_CHECKING, Any
|
||||
|
||||
from ..core import exitcodes
|
||||
|
||||
# The wrapper for a secret parameter, and its reader. They live with the other rules that
|
||||
# keep secrets out of the logs (refer to meshterm.core.secrets). The tools import them
|
||||
# from here, next to Tool and ToolResult.
|
||||
from ..core.secrets import Secret as Secret
|
||||
from ..core.secrets import loggable
|
||||
from ..core.secrets import reveal as reveal
|
||||
|
||||
if TYPE_CHECKING: # do not import typer or the context at module load, for a fast start
|
||||
import typer
|
||||
|
||||
@@ -23,40 +30,6 @@ if TYPE_CHECKING: # do not import typer or the context at module load, for a fa
|
||||
from ..ui.report import Report
|
||||
|
||||
|
||||
class Secret:
|
||||
"""A parameter of a run that the run log must not keep, for example a private key.
|
||||
|
||||
:meth:`Tool.execute` stores the parameters of each run in the ``runs`` table, and logs
|
||||
them at debug level. A secret in the parameters was thus stored as plain text: the key
|
||||
of ``config import-key`` stayed in the database. A secret goes into the parameters in
|
||||
this wrapper. The JSON of the run log and the log line show ``<secret>``, and the tool
|
||||
reads :attr:`value`.
|
||||
|
||||
The class does not derive from ``str``, because ``json.dumps`` writes the value of a
|
||||
``str`` subclass and never calls its ``default`` for it.
|
||||
|
||||
Attributes:
|
||||
value: The secret itself.
|
||||
"""
|
||||
|
||||
__slots__ = ("value",)
|
||||
|
||||
def __init__(self, value: str) -> None:
|
||||
"""Keep ``value`` out of each text form of this object."""
|
||||
self.value = value
|
||||
|
||||
def __repr__(self) -> str:
|
||||
"""``<secret>``, never the value."""
|
||||
return "<secret>"
|
||||
|
||||
__str__ = __repr__
|
||||
|
||||
|
||||
def reveal(value: Any) -> Any:
|
||||
"""The value inside a :class:`Secret`, or ``value`` itself if it is not a secret."""
|
||||
return value.value if isinstance(value, Secret) else value
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ToolResult:
|
||||
"""The result of a run of a tool.
|
||||
@@ -202,7 +175,9 @@ class Tool(ABC):
|
||||
from ..core.selection import DeviceSelectionError
|
||||
|
||||
run_id = ctx.repo.start_run(self.name, params, ctx.profile_name)
|
||||
ctx.log.debug("run %s start: tool=%s params=%s", run_id, self.name, params)
|
||||
# The copy with each secret hidden: a password, a channel secret, or a PIN in the
|
||||
# parameters must never reach the log (refer to meshterm.core.secrets).
|
||||
ctx.log.debug("run %s start: tool=%s params=%s", run_id, self.name, loggable(params))
|
||||
try:
|
||||
result = await self.run(ctx, {**params, "_run_id": run_id})
|
||||
except (
|
||||
|
||||
@@ -191,6 +191,23 @@ def _isolate_config_dir(tmp_path_factory: pytest.TempPathFactory) -> Iterator[No
|
||||
os.environ["MESHTERM_HOME"] = previous
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _empty_registries(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Each test starts with no registered secret and no registered host.
|
||||
|
||||
The two registries are process-wide on purpose: a secret that MeshTerm read once must
|
||||
stay hidden for the rest of the run. In a suite, that means that a password one test
|
||||
used ("hello", the default room password) hides the same word in each later test that
|
||||
looks for it. ``monkeypatch`` puts the shared objects back after the test.
|
||||
"""
|
||||
from meshterm.core import anonymize, secrets
|
||||
|
||||
monkeypatch.setattr(secrets, "_known", set())
|
||||
monkeypatch.setattr(secrets, "_known_pattern", None)
|
||||
monkeypatch.setattr(anonymize, "_hosts", {})
|
||||
monkeypatch.setattr(anonymize, "_host_pattern", None)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_platform() -> Iterator[None]:
|
||||
"""Each test starts and ends on :data:`~meshterm.platforms.REGULAR`.
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""No log ever holds a password or a secret: the log file, the run log, or the remote history.
|
||||
|
||||
JP, 2026-10-18: "NEVER log passwords or secrets!" The rule is a property of the places that
|
||||
keep records, and not a habit of each tool. Thus the most important test here does not
|
||||
look at the rules one by one. It runs each command that takes a secret, with a planted
|
||||
value, through the real CLI at the DEBUG log level, and then searches the log file and the
|
||||
``runs`` table for each planted value. A new command that takes a secret is covered when
|
||||
it is added to that list. The other tests hold each rule and each shape on its own, so
|
||||
that a failure says which rule broke.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from rich.console import Console
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from meshterm.core.device_config import DeviceConfigError, get_spec, parse_value
|
||||
from meshterm.core.models import Contact, Conversation
|
||||
from meshterm.core.remote_store import RemoteStore
|
||||
from meshterm.core.secrets import (
|
||||
SECRET_TEXT,
|
||||
Secret,
|
||||
hide,
|
||||
loggable,
|
||||
names_secret,
|
||||
remember,
|
||||
without_secret,
|
||||
)
|
||||
from meshterm.persistence import db
|
||||
from meshterm.persistence.logging import configure_logging, get_logger, log_path
|
||||
from meshterm.persistence.repository import Repository
|
||||
|
||||
#: Planted values. Each one is distinct, so a failure names the path that leaked it.
|
||||
_PASSWORD = "plantedpw-7c1e"
|
||||
_HEX = "5ec2e7a1" * 4
|
||||
|
||||
|
||||
# --- names ----------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name", ["password", "device_pin", "guest.password", "bridge.secret", "prv.key", "ble_pin"]
|
||||
)
|
||||
def test_a_name_with_a_secret_word_names_a_secret(name: str) -> None:
|
||||
"""A secret word anywhere in the name, between any separators, makes the name secret."""
|
||||
assert names_secret(name)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["pinned", "spin", "public_key", "key_prefix", "name", "to"])
|
||||
def test_a_name_that_only_contains_the_letters_is_not_secret(name: str) -> None:
|
||||
"""A word must be whole: ``pinned`` is not ``pin``, and a public key is not a secret."""
|
||||
assert not names_secret(name)
|
||||
|
||||
|
||||
# --- structures -----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_each_shape_of_a_secret_in_the_parameters_is_hidden() -> None:
|
||||
"""A key, a record, an operation, a dataclass field, a wrapper, and bytes: each is hidden."""
|
||||
params = {
|
||||
"room": "Lakeside",
|
||||
"password": _PASSWORD,
|
||||
"ops": [("set", "device_pin", 246813), ("import_key", Secret(_HEX))],
|
||||
"changes": [{"key": "device_pin", "previous": 135792, "value": 246813}],
|
||||
"set_channel": (5, "#lakeside", bytes.fromhex(_HEX)),
|
||||
"conversation": Conversation(
|
||||
label="#lakeside", is_channel=True, secret=bytes.fromhex(_HEX)
|
||||
),
|
||||
}
|
||||
shown = json.dumps(loggable(params), default=str)
|
||||
for value in (_PASSWORD, _HEX, "246813", "135792", "xc2"):
|
||||
assert value not in shown
|
||||
assert "Lakeside" in shown # what is not a secret stays
|
||||
assert "Conversation(label='#lakeside'" in shown
|
||||
assert "secret=<secret>" in shown
|
||||
|
||||
|
||||
def test_a_list_of_setting_names_is_not_an_operation() -> None:
|
||||
"""Only ``(verb, name, value)`` hides its value. A list of names has no value in it."""
|
||||
names = ["name", "device_pin", "radio_freq"]
|
||||
assert loggable({"settings": names}) == {"settings": names}
|
||||
assert loggable(("get", "device_pin")) == ("get", "device_pin")
|
||||
|
||||
|
||||
def test_an_absent_secret_stays_absent() -> None:
|
||||
"""A password that was not given is a fact that hides nothing. It stays ``None``."""
|
||||
assert loggable({"password": None, "secret": ""}) == {"password": None, "secret": ""}
|
||||
assert "secret=None" in str(loggable(Conversation(label="Bob", is_channel=False)))
|
||||
|
||||
|
||||
# --- text -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text,expected",
|
||||
[
|
||||
("password hunter2-new", f"password {SECRET_TEXT}"),
|
||||
("set guest.password hello there", f"set guest.password {SECRET_TEXT}"),
|
||||
("set radio 869.5,250,11,5", "set radio 869.5,250,11,5"),
|
||||
(
|
||||
f"meshcore://channel/add?name=Lakeside&secret={_HEX}",
|
||||
f"meshcore://channel/add?name=Lakeside&secret={SECRET_TEXT}",
|
||||
),
|
||||
(
|
||||
"Conversation(secret=b'\\x8b3\\x87', contact=None)",
|
||||
"Conversation(secret=<secret>, contact=None)",
|
||||
),
|
||||
("{'password': 'abcd1234'}", "{'password': <secret>}"),
|
||||
('{"pin": "998877"}', '{"pin": <secret>}'),
|
||||
("Conversation(secret=None)", "Conversation(secret=None)"),
|
||||
("device_pin: must be 0 (no PIN) or six digits", None),
|
||||
("BLE ProvidePin pairing for DD:F3:76:8B:0B:0B: status=19 ok=False", None),
|
||||
],
|
||||
)
|
||||
def test_a_text_hides_each_secret_that_its_form_shows(text: str, expected: str | None) -> None:
|
||||
"""A remote command, a URL, a ``repr``, and JSON all show which value is the secret."""
|
||||
assert hide(text) == (text if expected is None else expected)
|
||||
|
||||
|
||||
def test_a_known_secret_is_hidden_in_any_text() -> None:
|
||||
"""A registered secret is hidden where nothing names it: in an error, or inside a hex frame."""
|
||||
remember(_PASSWORD)
|
||||
remember(bytes.fromhex(_HEX))
|
||||
assert hide(f"login refused: {_PASSWORD} is wrong") == f"login refused: {SECRET_TEXT} is wrong"
|
||||
frame = "Unknown stats type: 66, data: 1842" + _HEX.upper() + "00ff"
|
||||
assert _HEX.upper() not in hide(frame)
|
||||
|
||||
|
||||
def test_a_short_or_common_value_is_not_registered() -> None:
|
||||
"""A value of three characters, or a secret word, would hide ordinary text everywhere."""
|
||||
remember("abc")
|
||||
remember("password")
|
||||
assert hide("abc password") == "abc password"
|
||||
|
||||
|
||||
def test_a_registered_pin_does_not_hide_part_of_a_longer_number() -> None:
|
||||
"""A PIN is found as a whole word only, so it cannot cut a timestamp or a count."""
|
||||
remember(246813)
|
||||
assert hide("pin 246813 and 12468139") == f"pin {SECRET_TEXT} and 12468139"
|
||||
|
||||
|
||||
# --- the remote command line ----------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_remote_history_keeps_the_command_without_its_secret(tmp_path: Path) -> None:
|
||||
"""↑ recalls ``password ``, ready to type, and the file never holds the password."""
|
||||
store = RemoteStore(tmp_path / "remote.json")
|
||||
node = Contact(name="Hilltop", public_key="3d" * 32)
|
||||
store.append_history(node, f"password {_PASSWORD}")
|
||||
store.append_history(node, f"set guest.password {_PASSWORD}")
|
||||
store.append_history(node, "set name Hilltop")
|
||||
assert store.history(node) == ["password ", "set guest.password ", "set name Hilltop"]
|
||||
assert _PASSWORD not in (tmp_path / "remote.json").read_text(encoding="utf-8")
|
||||
assert without_secret("ver") == "ver"
|
||||
|
||||
|
||||
# --- the device settings --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_refused_pin_is_not_repeated_in_the_error() -> None:
|
||||
"""The error of a secret setting states the rule. Each other setting still shows the value."""
|
||||
with pytest.raises(DeviceConfigError) as refused:
|
||||
parse_value(get_spec("device_pin"), "24681x")
|
||||
assert "24681x" not in str(refused.value)
|
||||
with pytest.raises(DeviceConfigError) as short:
|
||||
parse_value(get_spec("device_pin"), "24681")
|
||||
assert "24681" not in str(short.value)
|
||||
with pytest.raises(DeviceConfigError) as other:
|
||||
parse_value(get_spec("radio_sf"), "13")
|
||||
assert "13" in str(other.value)
|
||||
|
||||
|
||||
# --- the places that keep records -----------------------------------------------------
|
||||
|
||||
|
||||
def test_neither_log_handler_writes_a_secret(tmp_path: Path) -> None:
|
||||
"""The file, its traceback, and the console all get the hidden form of a record."""
|
||||
console = Console(file=io.StringIO(), width=200, record=True, color_system=None)
|
||||
configure_logging(console, tmp_path, file_level=logging.DEBUG, level=logging.DEBUG)
|
||||
log = get_logger("core.connection")
|
||||
log.debug("run 1 start: params=%s", {"password": _PASSWORD})
|
||||
try:
|
||||
raise ValueError(f"login with {_PASSWORD} refused")
|
||||
except ValueError:
|
||||
log.exception("login failed")
|
||||
for handler in get_logger().handlers:
|
||||
handler.flush()
|
||||
written = log_path(tmp_path).read_text(encoding="utf-8")
|
||||
assert _PASSWORD not in written
|
||||
assert SECRET_TEXT in written
|
||||
assert _PASSWORD not in console.export_text()
|
||||
|
||||
|
||||
def test_the_run_log_stores_no_secret(tmp_path: Path) -> None:
|
||||
"""The arguments and the summary of a run go into the ``runs`` table hidden."""
|
||||
repo = Repository(tmp_path / "test.db")
|
||||
run = repo.start_run("rooms", {"cli_action": "join", "password": _PASSWORD})
|
||||
repo.finish_run(run, "error", {"error": f"refused {_PASSWORD}"})
|
||||
repo.close()
|
||||
conn = sqlite3.connect(tmp_path / "test.db")
|
||||
args, summary = conn.execute("SELECT args_json, summary_json FROM runs").fetchone()
|
||||
conn.close()
|
||||
assert _PASSWORD not in args + summary
|
||||
assert SECRET_TEXT in args
|
||||
|
||||
|
||||
def test_the_migration_hides_the_secrets_that_older_versions_stored(tmp_path: Path) -> None:
|
||||
"""A database at version 19 loses each stored secret, one time, when it opens."""
|
||||
path = tmp_path / "old.db"
|
||||
db.connect(path).close()
|
||||
conn = sqlite3.connect(path)
|
||||
conversation = "Conversation(label='#x', is_channel=True, secret=b'\\x5e\\xc2', contact=None)"
|
||||
leaked = json.dumps({"conversation": conversation})
|
||||
conn.execute(
|
||||
"INSERT INTO runs (tool, args_json, summary_json, status, started_at) "
|
||||
"VALUES ('chat', ?, ?, 'ok', '2026-10-01T00:00:00')",
|
||||
(leaked, json.dumps({"password": _PASSWORD})),
|
||||
)
|
||||
conn.execute("UPDATE schema_meta SET value = '19' WHERE key = 'version'")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
db.connect(path).close()
|
||||
conn = sqlite3.connect(path)
|
||||
args, summary = conn.execute("SELECT args_json, summary_json FROM runs").fetchone()
|
||||
conn.close()
|
||||
assert "\\\\x5e" not in args and "secret=<secret>" in args
|
||||
assert _PASSWORD not in summary
|
||||
|
||||
|
||||
# --- the whole program ----------------------------------------------------------------
|
||||
|
||||
#: Each command that takes a secret, with a planted value in it, and the planted values.
|
||||
#: The commands run against the simulator. Some of them fail (a room that the simulator
|
||||
#: does not have), and that is fine: a failure is logged too, and with the parameters.
|
||||
_COMMANDS: list[tuple[list[str], tuple[str, ...]]] = [
|
||||
(["repeater-admin", "Yagi-Repeater", "ver", "--password", "plantedpw-a1"], ("plantedpw-a1",)),
|
||||
(["repeater-admin", "Yagi-Repeater", "password", "plantedpw-a2"], ("plantedpw-a2",)),
|
||||
(
|
||||
["repeater-admin", "Yagi-Repeater", "set", "guest.password", "plantedpw-a3"],
|
||||
("plantedpw-a3",),
|
||||
),
|
||||
(["rooms", "join", "Lakeside BBS", "--password", "plantedpw-r1"], ("plantedpw-r1",)),
|
||||
(
|
||||
["tx-optimize", "--path", "a1,b2", "--min", "20", "--max", "20", "--samples", "1"]
|
||||
+ ["--no-apply", "--password", "plantedpw-t1"],
|
||||
("plantedpw-t1",),
|
||||
),
|
||||
(["channels", "add", "5", "Planted", "--secret", "a1" * 16], ("a1" * 16,)),
|
||||
(["channels", "join", "6", "Planted2", "b2" * 16], ("b2" * 16,)),
|
||||
(
|
||||
["channels", "import", "7", "meshcore://channel/add?name=P3&secret=" + "c3" * 16],
|
||||
("c3" * 16,),
|
||||
),
|
||||
(["config", "channel", "5", "Planted4", "--secret", "d4" * 16], ("d4" * 16,)),
|
||||
(["config", "set", "device_pin", "246813"], ("246813",)),
|
||||
(["config", "set", "device_pin", "97531"], ("97531",)),
|
||||
]
|
||||
|
||||
|
||||
def test_no_command_writes_a_planted_secret_to_any_log(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""Each command with a secret, at the DEBUG level: no planted value in the log or the runs.
|
||||
|
||||
This is the test that holds the promise. The rules above can each be correct and a
|
||||
path between them can still leak, as the parameters of a chat run did. Thus this test
|
||||
asks the two places that keep records, and not the rules.
|
||||
"""
|
||||
home = tmp_path / "home"
|
||||
monkeypatch.setenv("MESHTERM_HOME", str(home))
|
||||
runner = CliRunner()
|
||||
assert runner.invoke(app_(), ["preferences", "set", "log_level", "DEBUG"]).exit_code == 0
|
||||
planted: list[str] = []
|
||||
for args, values in _COMMANDS:
|
||||
runner.invoke(app_(), ["--mock", *args])
|
||||
planted.extend(values)
|
||||
|
||||
for handler in get_logger().handlers:
|
||||
handler.flush()
|
||||
log = log_path(home).read_text(encoding="utf-8")
|
||||
assert log.count("start: tool=") >= len(_COMMANDS), "the commands did not run"
|
||||
conn = sqlite3.connect(home / "meshterm-mock.db")
|
||||
runs = "\n".join(
|
||||
f"{args} {summary}"
|
||||
for args, summary in conn.execute("SELECT args_json, summary_json FROM runs")
|
||||
)
|
||||
conn.close()
|
||||
for value in planted:
|
||||
assert value not in log, f"{value} is in the log"
|
||||
assert value not in runs, f"{value} is in the runs table"
|
||||
|
||||
|
||||
def app_():
|
||||
"""The CLI application, imported late so that the environment of the test applies."""
|
||||
from meshterm.cli import app
|
||||
|
||||
return app
|
||||
Reference in new issue
Block a user