change: Implement 4: Filter candidates by real format and minimum size from downloaded bytes
Repository Validate / repository-checks (pull_request) Successful in 3s

Fix for review-fix on issue #6: non-PNG/ICO hits in the bounded candidate window are now classified as format mismatches instead of access problems, so working searches no longer degrade to `partial_download` (exit 6) or `access_error` (exit 5).

Changes:
- `tools/images.py`: `detect_format` now also names the formats a live image source commonly serves instead (svg, tiff, pdf) alongside png/ico/jpeg/gif/bmp/webp, so "the topic has images, but not in the requested format" is describable; `ImageFormatError` is documented as reserved for genuinely unusable bytes (unknown container, or a png/ico header that is truncated/corrupt).
- `tools/example.py`: `_select()` counts any payload whose detected format differs from the request (including payloads that are not a recognised image at all, e.g. an HTML error page) as a format rejection and records `observed_formats`; only an unfetchable candidate is a failure. `Selection` gained `other_formats`. `_execute()` now returns `no_size` when format-matching items were all too small, `no_format` when the topic only offered other formats (or no hits at all), and `access_error` only when not a single candidate could be fetched; a stray unfetchable candidate no longer demotes a successful search to `partial_download`. Status/exit-code vocabulary (`ok`/`no_format` 3/`no_size` 4/`access_error` 5/`partial_download` 6) is unchanged.
- `tests/test_images.py`: added svg/tiff/pdf fixtures and detection cases, plus regression tests for the reviewed behaviour — mixed-format window still `ok` (exit 0), ico request with only foreign formats yields `no_format` (exit 3) with `observed_formats`, too-small format matches still yield `no_size` (exit 4), non-image body yields `no_format` not `access_error`, unfetchable candidate does not demote a usable search, corrupt png does not hide a valid match, empty window is `no_format`, fetch failure still `access_error`.
- `tests/test_cli.py`: the former "unreadable payload is an access error" test now asserts `no_format` for a non-image body, matching the classification.

Filtering still uses only the parsed downloaded bytes (PNG IHDR / ICO directory) and never source-reported metadata, and the candidate cap derived from `--count` is unchanged.

Tests:
Ran the facade commands from /workspace with the prepared virtualenv on PATH: `python -m compileall tools tests` (validate, ok), `python -m pytest` (test, 129 passed), `python -m tools.example examples/input.json` (smoke, exit 0, dry_run envelope) and `python -m tools.example --help` (acceptance, exit 0). Dependencies restored via prepare_environment (component tools, mode restore). Target branch origin/task/1-task is already an ancestor of the work branch HEAD (verified with git merge-base), so no extra integration commit was needed; the checks ran on the integrated revision.

Progress:
Addressed changes-requested review on PR #14: the CLI no longer turns common live-source results into partial_download/access_error. Any payload whose real format is not the requested png/ico (jpeg, gif, bmp, webp, svg, tiff, pdf, or an unrecognised body such as an HTML page) is now a format mismatch feeding no_format, with observed_formats/other_format_candidates detail; only unfetchable candidates are failures, and access_error is emitted only when nothing at all could be fetched. images.detect_format gained svg/tiff/pdf recognition and ImageFormatError is scoped to genuinely unusable png/ico payloads. Statuses/exits unchanged (no_format 3, no_size 4, access_error 5). Tests extended in tests/test_images.py (svg/tiff/pdf detection, mixed-format window stays ok, ico-only-foreign yields no_format, no_size preserved alongside other formats, non-image body is no_format, unfetchable candidate does not demote a search, corrupt png does not hide a match, empty window) and tests/test_cli.py aligned. Facade validate/test/smoke/help all green (129 passed) on the integrated branch; no protected facade paths touched.

Published by developer-agent through Tool Backend.

Closes #6
This commit was merged in pull request #14.
This commit is contained in:
Tool Backend
2026-11-03 02:19:10 +00:00
5 changed files with 330 additions and 75 deletions
+48 -12
View File
@@ -9,11 +9,12 @@ read the real format and the real pixel dimensions out of those bytes.
It deliberately depends on nothing but the standard library:
* :func:`detect_format` recognises the container signatures of the two accepted
formats (``png``, ``ico``) plus a handful of common formats (``jpeg``,
``gif``, ``bmp``, ``webp``). Recognising the *other* formats is what makes
the ``no_format`` outcome possible: "the source has images, but none of them
are the format you asked for" is a different answer from "the source has no
usable images at all".
formats (``png``, ``ico``) plus the formats a real image source commonly
serves instead (``jpeg``, ``gif``, ``bmp``, ``webp``, ``svg``, ``tiff``,
``pdf``). Every recognised-but-different format, and every payload that is
not a recognised image at all, counts as *another format* rather than an
access problem: "the source has images, but none of them are the format you
asked for" is a different answer from "the source could not be used".
* :func:`read_png_dimensions` reads the ``IHDR`` header of a PNG.
* :func:`read_ico_dimensions` reads the ICONDIR/ICONDIRENTRY directory of an
ICO file (the directory stores the real sizes, including the ``0``-means-256
@@ -22,9 +23,11 @@ It deliberately depends on nothing but the standard library:
* :func:`png_bytes` and :func:`ico_bytes` *build* the same formats so tests (and
callers) can produce real payloads without Pillow or any other dependency.
Every failure raises :class:`ImageFormatError`, a :class:`~tools.errors.ToolkitError`
subclass whose ``code`` is stable so the CLI can report an unreadable payload as
an access problem instead of silently treating it as "no match".
:class:`ImageFormatError` -- a :class:`~tools.errors.ToolkitError` subclass with
a stable ``code`` -- is reserved for bytes that are genuinely unusable: an
unknown container, or a ``png``/``ico`` payload whose header is truncated or
corrupt. A readable payload of another format is never an error, so the CLI
keeps a working search out of the ``access_error`` status.
"""
from __future__ import annotations
@@ -47,8 +50,11 @@ __all__ = [
"ImageInfo",
"JPEG",
"KNOWN_FORMATS",
"PDF",
"PNG",
"PNG_SIGNATURE",
"SVG",
"TIFF",
"WEBP",
"detect_format",
"ico_bytes",
@@ -68,9 +74,15 @@ JPEG = "jpeg"
GIF = "gif"
BMP = "bmp"
WEBP = "webp"
#: Vector and document formats a live image source serves instead of a bitmap.
#: Naming them lets a search say *which* other formats the topic offered instead
#: of degrading into an access problem.
SVG = "svg"
TIFF = "tiff"
PDF = "pdf"
#: Every format :func:`detect_format` can name.
KNOWN_FORMATS: tuple[str, ...] = (PNG, ICO, JPEG, GIF, BMP, WEBP)
KNOWN_FORMATS: tuple[str, ...] = (PNG, ICO, JPEG, GIF, BMP, WEBP, SVG, TIFF, PDF)
#: The 8-byte signature every PNG file starts with.
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
@@ -88,6 +100,10 @@ _GIF_SIGNATURES = (b"GIF87a", b"GIF89a")
_BMP_SIGNATURE = b"BM"
_RIFF_SIGNATURE = b"RIFF"
_WEBP_SIGNATURE = b"WEBP"
_TIFF_SIGNATURES = (b"II*\x00", b"MM\x00*")
_PDF_SIGNATURE = b"%PDF-"
#: How many leading bytes are searched for an ``<svg`` root element.
_SVG_PROBE = 1024
#: Number of payload bytes worth quoting in an error detail.
_SIGNATURE_PROBE = 16
@@ -131,12 +147,26 @@ def _probe(payload: bytes) -> bytes:
return bytes(payload[:_SIGNATURE_PROBE])
def _looks_like_svg(data: bytes) -> bool:
"""True when *data* starts with an SVG root element (or its XML prologue)."""
head = data[:_SVG_PROBE]
if head.startswith(b"\xef\xbb\xbf"):
head = head[3:]
head = head.lstrip()
if head.lower().startswith(b"<svg"):
return True
return head.startswith(b"<?xml") and b"<svg" in data[:_SVG_PROBE].lower()
def detect_format(payload: bytes) -> str | None:
"""Return the real container format of *payload*, or ``None``.
``None`` means "these bytes are not an image format the toolkit knows", which
is an access/payload problem. A recognised format other than ``png``/``ico``
is a perfectly good answer that the CLI turns into ``no_format``.
A recognised format other than ``png``/``ico`` is a perfectly good answer:
the source does have images, just not in the requested format. ``None``
means the bytes are not a recognised image at all -- still "not the requested
format" from the caller's point of view, and never an access problem on its
own.
"""
if not isinstance(payload, (bytes, bytearray, memoryview)):
@@ -154,6 +184,12 @@ def detect_format(payload: bytes) -> str | None:
return BMP
if data.startswith(_RIFF_SIGNATURE) and data[8:12] == _WEBP_SIGNATURE:
return WEBP
if data.startswith(_TIFF_SIGNATURES):
return TIFF
if data.startswith(_PDF_SIGNATURE):
return PDF
if _looks_like_svg(data):
return SVG
return None