Files
product-54354138-1/tests/test_cli.py
T
Tool Backend d5bb24fc01
Repository Validate / repository-checks (pull_request) Successful in 3s
change: Implement 4: Filter candidates by real format and minimum size from downloaded bytes
Fix for review-fix on issue #6: non-PNG/ICO hits in the bounded candidate window are now classified as format mismatches instead of access problems, so working searches no longer degrade to `partial_download` (exit 6) or `access_error` (exit 5).

Changes:
- `tools/images.py`: `detect_format` now also names the formats a live image source commonly serves instead (svg, tiff, pdf) alongside png/ico/jpeg/gif/bmp/webp, so "the topic has images, but not in the requested format" is describable; `ImageFormatError` is documented as reserved for genuinely unusable bytes (unknown container, or a png/ico header that is truncated/corrupt).
- `tools/example.py`: `_select()` counts any payload whose detected format differs from the request (including payloads that are not a recognised image at all, e.g. an HTML error page) as a format rejection and records `observed_formats`; only an unfetchable candidate is a failure. `Selection` gained `other_formats`. `_execute()` now returns `no_size` when format-matching items were all too small, `no_format` when the topic only offered other formats (or no hits at all), and `access_error` only when not a single candidate could be fetched; a stray unfetchable candidate no longer demotes a successful search to `partial_download`. Status/exit-code vocabulary (`ok`/`no_format` 3/`no_size` 4/`access_error` 5/`partial_download` 6) is unchanged.
- `tests/test_images.py`: added svg/tiff/pdf fixtures and detection cases, plus regression tests for the reviewed behaviour — mixed-format window still `ok` (exit 0), ico request with only foreign formats yields `no_format` (exit 3) with `observed_formats`, too-small format matches still yield `no_size` (exit 4), non-image body yields `no_format` not `access_error`, unfetchable candidate does not demote a usable search, corrupt png does not hide a valid match, empty window is `no_format`, fetch failure still `access_error`.
- `tests/test_cli.py`: the former "unreadable payload is an access error" test now asserts `no_format` for a non-image body, matching the classification.

Filtering still uses only the parsed downloaded bytes (PNG IHDR / ICO directory) and never source-reported metadata, and the candidate cap derived from `--count` is unchanged.

Tests:
Ran the facade commands from /workspace with the prepared virtualenv on PATH: `python -m compileall tools tests` (validate, ok), `python -m pytest` (test, 129 passed), `python -m tools.example examples/input.json` (smoke, exit 0, dry_run envelope) and `python -m tools.example --help` (acceptance, exit 0). Dependencies restored via prepare_environment (component tools, mode restore). Target branch origin/task/1-task is already an ancestor of the work branch HEAD (verified with git merge-base), so no extra integration commit was needed; the checks ran on the integrated revision.

Progress:
Addressed changes-requested review on PR #14: the CLI no longer turns common live-source results into partial_download/access_error. Any payload whose real format is not the requested png/ico (jpeg, gif, bmp, webp, svg, tiff, pdf, or an unrecognised body such as an HTML page) is now a format mismatch feeding no_format, with observed_formats/other_format_candidates detail; only unfetchable candidates are failures, and access_error is emitted only when nothing at all could be fetched. images.detect_format gained svg/tiff/pdf recognition and ImageFormatError is scoped to genuinely unusable png/ico payloads. Statuses/exits unchanged (no_format 3, no_size 4, access_error 5). Tests extended in tests/test_images.py (svg/tiff/pdf detection, mixed-format window stays ok, ico-only-foreign yields no_format, no_size preserved alongside other formats, non-image body is no_format, unfetchable candidate does not demote a search, corrupt png does not hide a match, empty window) and tests/test_cli.py aligned. Facade validate/test/smoke/help all green (129 passed) on the integrated branch; no protected facade paths touched.

Published by developer-agent through Tool Backend.

Closes #6
2026-11-03 02:19:10 +00:00

588 lines
20 KiB
Python

"""Contract tests for the native-process CLI in :mod:`tools.example`."""
from __future__ import annotations
import json
import struct
import subprocess
import sys
import zlib
from pathlib import Path
from typing import Any, Sequence
import pytest
from tools import envelope, example
from tools.errors import SourceAccessError
REPO_ROOT = Path(__file__).resolve().parents[1]
EXAMPLE_REQUEST = REPO_ROOT / "examples" / "input.json"
def run_cli(*args: str) -> subprocess.CompletedProcess[str]:
return subprocess.run(
[sys.executable, "-m", "tools.example", *args],
cwd=REPO_ROOT,
capture_output=True,
text=True,
check=False,
)
def single_document(result: subprocess.CompletedProcess[str]) -> dict[str, Any]:
"""Assert stdout holds exactly one machine-readable JSON document."""
lines = [line for line in result.stdout.splitlines() if line.strip()]
assert len(lines) == 1, result.stdout
assert result.stderr == ""
document = json.loads(lines[0])
assert isinstance(document, dict)
assert set(document) == {"status", "source", "request", "items", "error"}
return document
def test_status_vocabulary_and_exit_codes_are_fixed() -> None:
assert envelope.STATUS_EXIT_CODES == {
"ok": 0,
"dry_run": 0,
"invalid_request": 2,
"no_format": 3,
"no_size": 4,
"access_error": 5,
"partial_download": 6,
}
assert envelope.STATUS_VOCABULARY == (
"ok",
"dry_run",
"invalid_request",
"no_format",
"no_size",
"access_error",
"partial_download",
)
def test_help_documents_every_parameter() -> None:
result = run_cli("--help")
assert result.returncode == 0
for fragment in (
"REQUEST.json",
"--topic",
"--width",
"--height",
"--format",
"--source",
"--count",
"--download",
"--output-dir",
"--resize",
"--dry-run",
"png|ico",
"JSON",
):
assert fragment in result.stdout, fragment
def test_example_request_document_is_canonical_dry_run() -> None:
document = json.loads(EXAMPLE_REQUEST.read_text(encoding="utf-8"))
assert document["dry_run"] is True
assert document["format"] in ("png", "ico")
result = run_cli("examples/input.json")
assert result.returncode == envelope.EXIT_OK
payload = single_document(result)
assert payload["status"] == envelope.STATUS_DRY_RUN
assert payload["source"] == document["source"]
assert payload["items"] == []
assert payload["error"] is None
assert payload["request"]["topic"] == document["topic"]
assert payload["request"]["format"] == document["format"]
def test_flags_dry_run_emits_one_document() -> None:
result = run_cli("--topic", "fox", "--format", "png", "--width", "640", "--dry-run")
assert result.returncode == envelope.EXIT_OK
payload = single_document(result)
assert payload["status"] == envelope.STATUS_DRY_RUN
assert payload["request"]["width"] == 640
assert payload["request"]["dry_run"] is True
def test_unsupported_format_is_invalid_request() -> None:
result = run_cli("--topic", "fox", "--format", "gif")
assert result.returncode == envelope.EXIT_INVALID_REQUEST
payload = single_document(result)
assert payload["status"] == envelope.STATUS_INVALID_REQUEST
assert payload["error"]["code"] == "request_format_unsupported"
assert payload["error"]["details"]["accepted_formats"] == ["png", "ico"]
def test_unknown_source_lists_available_ids() -> None:
result = run_cli("--topic", "fox", "--format", "png", "--source", "nope")
assert result.returncode == envelope.EXIT_INVALID_REQUEST
payload = single_document(result)
assert payload["status"] == envelope.STATUS_INVALID_REQUEST
assert payload["error"]["code"] == "source_unknown"
available = payload["error"]["details"]["available_sources"]
assert "nope" not in available
assert "wikimedia_commons" in available
assert len(available) >= 2
def test_no_request_is_invalid_request() -> None:
result = run_cli()
assert result.returncode == envelope.EXIT_INVALID_REQUEST
payload = single_document(result)
assert payload["status"] == envelope.STATUS_INVALID_REQUEST
def test_document_combined_with_flags_is_invalid_request() -> None:
result = run_cli("examples/input.json", "--topic", "fox")
assert result.returncode == envelope.EXIT_INVALID_REQUEST
payload = single_document(result)
assert payload["error"]["details"]["conflicting_flags"] == ["topic"]
@pytest.mark.parametrize(
"arguments",
[
("--format", "png"),
("--topic", "fox"),
("--topic", "fox", "--format", "png", "--count", "0"),
("--topic", " ", "--format", "png"),
],
)
def test_incomplete_requests_are_invalid(arguments: Sequence[str]) -> None:
result = run_cli(*arguments)
assert result.returncode == envelope.EXIT_INVALID_REQUEST
assert single_document(result)["status"] == envelope.STATUS_INVALID_REQUEST
def test_missing_document_file_is_invalid_request(tmp_path: Path) -> None:
result = run_cli(str(tmp_path / "absent.json"))
assert result.returncode == envelope.EXIT_INVALID_REQUEST
assert single_document(result)["status"] == envelope.STATUS_INVALID_REQUEST
def test_malformed_document_is_invalid_request(tmp_path: Path) -> None:
broken = tmp_path / "broken.json"
broken.write_text("{not json", encoding="utf-8")
result = run_cli(str(broken))
assert result.returncode == envelope.EXIT_INVALID_REQUEST
assert single_document(result)["status"] == envelope.STATUS_INVALID_REQUEST
class FakeCandidate:
"""Minimal stand-in for :class:`tools.sources.SourceCandidate`."""
def __init__(
self,
title: str,
file_url: str,
mime_type: str,
width: int | None,
height: int | None,
*,
payload: bytes | None = None,
fail: bool = False,
) -> None:
self.source_id = "fake"
self.title = title
self.file_url = file_url
self.page_url = file_url + ".html"
self.mime_type = mime_type
self.width = width
self.height = height
self.license = "CC0"
self.creator = "tester"
self._payload = payload
self._fail = fail
def to_dict(self) -> dict[str, Any]:
return {
"source_id": self.source_id,
"title": self.title,
"file_url": self.file_url,
"page_url": self.page_url,
"mime_type": self.mime_type,
"width": self.width,
"height": self.height,
}
def fetch_bytes(self) -> bytes:
if self._fail:
raise SourceAccessError("download refused", url=self.file_url)
if self._payload is not None:
return self._payload
# Default to bytes that really are a PNG of the advertised size, because
# the CLI filters on downloaded bytes and never on source metadata.
return png_bytes(self.width or 1, self.height or 1)
class FakeRegistry:
def __init__(self, candidates: Sequence[FakeCandidate] = (), error: Exception | None = None) -> None:
self._candidates = list(candidates)
self._error = error
self.calls: list[tuple[str, str, int]] = []
def available_sources(self) -> Sequence[str]:
return ["fake"]
def search(self, source_id: str, topic: str, limit: int = 10) -> list[FakeCandidate]:
self.calls.append((source_id, topic, limit))
if self._error is not None:
raise self._error
return self._candidates[:limit]
def run_main(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], registry: FakeRegistry, *args: str) -> tuple[int, dict[str, Any]]:
monkeypatch.setattr(example, "default_registry", lambda: registry)
code = example.main(list(args))
captured = capsys.readouterr()
assert captured.err == ""
lines = [line for line in captured.out.splitlines() if line.strip()]
assert len(lines) == 1, captured.out
return code, json.loads(lines[0])
BASE_ARGS = ("--topic", "fox", "--format", "png", "--width", "100", "--height", "100", "--source", "fake")
def test_ok_lists_items_without_download(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
registry = FakeRegistry([FakeCandidate("fox one", "https://example.test/a.png", "image/png", 100, 100)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_OK
assert payload["status"] == envelope.STATUS_OK
assert payload["source"] == "fake"
item = payload["items"][0]
assert item["url"] == "https://example.test/a.png"
assert item["file_url"] == "https://example.test/a.png"
assert item["source"] == "fake"
assert item["format"] == "png"
assert item["width"] == 100
assert item["height"] == 100
assert registry.calls == [("fake", "fox", example.candidate_cap(10))]
def test_no_format_when_every_payload_really_is_another_format(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.jpg", "image/jpeg", 100, 100, payload=gif_bytes())]
)
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_NO_FORMAT
assert payload["status"] == envelope.STATUS_NO_FORMAT
assert payload["error"]["details"]["unreadable_candidates"] == 0
assert payload["error"]["details"]["requested_format"] == "png"
def test_no_size_when_only_smaller_images_exist(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
registry = FakeRegistry([FakeCandidate("fox", "https://example.test/a.png", "image/png", 50, 50)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_NO_SIZE
assert payload["status"] == envelope.STATUS_NO_SIZE
def test_requested_size_is_a_minimum_not_an_exact_match(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
registry = FakeRegistry([FakeCandidate("fox", "https://example.test/a.png", "image/png", 400, 300)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_OK
assert payload["status"] == envelope.STATUS_OK
assert payload["request"]["resize"] is False
assert [item["width"] for item in payload["items"]] == [400]
assert [item["height"] for item in payload["items"]] == [300]
def test_exact_size_is_accepted(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
registry = FakeRegistry([FakeCandidate("fox", "https://example.test/a.png", "image/png", 100, 100)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_OK
assert len(payload["items"]) == 1
def test_resize_without_download_keeps_actual_size(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
registry = FakeRegistry([FakeCandidate("fox", "https://example.test/a.png", "image/png", 400, 300)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS, "--resize")
assert code == envelope.EXIT_OK
assert payload["request"]["resize"] is True
assert payload["items"][0]["width"] == 400
assert "download" not in payload["items"][0]
def test_download_success_writes_files(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None:
expected = png_bytes(100, 100)
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.png", "image/png", 100, 100, payload=expected)]
)
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS, "--download", "--output-dir", str(tmp_path))
assert code == envelope.EXIT_OK
download = payload["items"][0]["download"]
assert download["ok"] is True
target = Path(download["path"])
assert target.parent == tmp_path
# The verified bytes are stored verbatim, without a second download.
assert target.read_bytes() == expected
def test_partial_download_reports_status(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None:
registry = FakeRegistry(
[
FakeCandidate("fox", "https://example.test/a.png", "image/png", 100, 100),
FakeCandidate("fox2", "https://example.test/b.png", "image/png", 100, 100, fail=True),
]
)
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS, "--count", "2", "--download", "--output-dir", str(tmp_path))
assert code == envelope.EXIT_PARTIAL_DOWNLOAD
assert payload["status"] == envelope.STATUS_PARTIAL_DOWNLOAD
# The readable candidate is still reported and downloaded.
assert payload["items"][0]["download"]["ok"] is True
assert payload["error"]["details"]["failed"] == 1
def test_all_downloads_failing_is_access_error(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None:
registry = FakeRegistry([FakeCandidate("fox", "https://example.test/a.png", "image/png", 100, 100, fail=True)])
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS, "--download", "--output-dir", str(tmp_path))
assert code == envelope.EXIT_ACCESS_ERROR
assert payload["status"] == envelope.STATUS_ACCESS_ERROR
assert payload["error"]["code"] == "candidate_unreadable"
def test_search_access_error_maps_to_access_error(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
registry = FakeRegistry(error=SourceAccessError("host unreachable", url="https://example.test"))
code, payload = run_main(monkeypatch, capsys, registry, *BASE_ARGS)
assert code == envelope.EXIT_ACCESS_ERROR
assert payload["status"] == envelope.STATUS_ACCESS_ERROR
assert payload["error"]["code"] == "source_access"
# ---------------------------------------------------------------------------
# Size filtering and real downscaling of downloaded images
# ---------------------------------------------------------------------------
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
def png_bytes(width: int, height: int, rgba: tuple[int, int, int, int] = (255, 0, 0, 255)) -> bytes:
"""Build a minimal, valid RGBA PNG without any third-party dependency."""
raw = bytearray()
for _ in range(height):
raw.append(0)
raw += bytes(rgba) * width
def chunk(kind: bytes, data: bytes) -> bytes:
crc = zlib.crc32(kind + data) & 0xFFFFFFFF
return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", crc)
header = struct.pack(">IIBBBBB", width, height, 8, 6, 0, 0, 0)
return (
PNG_SIGNATURE
+ chunk(b"IHDR", header)
+ chunk(b"IDAT", zlib.compress(bytes(raw)))
+ chunk(b"IEND", b"")
)
def gif_bytes(width: int = 100, height: int = 100) -> bytes:
"""Build a payload that really is a GIF, i.e. a format we must never accept."""
return b"GIF89a" + struct.pack("<HH", width, height) + b"\x00" * 8
def png_size(payload: bytes) -> tuple[int, int]:
assert payload.startswith(PNG_SIGNATURE)
return struct.unpack(">II", payload[16:24])
def ico_bytes(inner_png: bytes, width: int, height: int) -> bytes:
header = struct.pack("<HHH", 0, 1, 1)
entry = struct.pack(
"<BBBBHHII",
width % 256,
height % 256,
0,
0,
1,
32,
len(inner_png),
6 + 16,
)
return header + entry + inner_png
def ico_payload(payload: bytes) -> tuple[int, int, bytes]:
reserved, kind, count = struct.unpack("<HHH", payload[:6])
assert (reserved, kind, count) == (0, 1, 1)
width, height, _colours, _reserved, _planes, _bits, size, offset = struct.unpack(
"<BBBBHHII", payload[6:22]
)
return width or 256, height or 256, payload[offset : offset + size]
DOWNLOAD_ARGS = ("--topic", "fox", "--format", "png", "--source", "fake")
def test_download_resize_downscales_stored_image(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path
) -> None:
payload = png_bytes(200, 100)
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.png", "image/png", 200, 100, payload=payload)]
)
code, document = run_main(
monkeypatch, capsys, registry, *BASE_ARGS, "--resize", "--download", "--output-dir", str(tmp_path)
)
assert code == envelope.EXIT_OK
download = document["items"][0]["download"]
assert download["ok"] is True
assert download["resize"]["applied"] is True
assert download["resize"]["from"] == {"width": 200, "height": 100}
assert download["resize"]["to"] == {"width": 100, "height": 50}
stored = Path(download["path"]).read_bytes()
assert png_size(stored) == (100, 50)
assert download["width"] == 100
assert download["height"] == 50
def test_download_without_resize_keeps_original_payload(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path
) -> None:
payload = png_bytes(200, 100)
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.png", "image/png", 200, 100, payload=payload)]
)
code, document = run_main(
monkeypatch, capsys, registry, *BASE_ARGS, "--download", "--output-dir", str(tmp_path)
)
assert code == envelope.EXIT_OK
download = document["items"][0]["download"]
assert "resize" not in download
assert Path(download["path"]).read_bytes() == payload
def test_resize_never_scales_up(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path
) -> None:
payload = png_bytes(50, 50)
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.png", "image/png", 50, 50, payload=payload)]
)
code, document = run_main(
monkeypatch, capsys, registry, *DOWNLOAD_ARGS, "--resize", "--download", "--output-dir", str(tmp_path)
)
assert code == envelope.EXIT_OK
download = document["items"][0]["download"]
assert download["resize"]["applied"] is False
assert download["resize"]["reason"] == "already_fits"
assert Path(download["path"]).read_bytes() == payload
def test_resize_downscales_png_stored_inside_ico(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path
) -> None:
payload = ico_bytes(png_bytes(100, 100), 100, 100)
registry = FakeRegistry(
[FakeCandidate("fox", "https://example.test/a.ico", "image/x-icon", 100, 100, payload=payload)]
)
code, document = run_main(
monkeypatch,
capsys,
registry,
"--topic",
"fox",
"--format",
"ico",
"--source",
"fake",
"--width",
"50",
"--height",
"50",
"--resize",
"--download",
"--output-dir",
str(tmp_path),
)
assert code == envelope.EXIT_OK
download = document["items"][0]["download"]
assert download["resize"]["to"] == {"width": 50, "height": 50}
width, height, inner = ico_payload(Path(download["path"]).read_bytes())
assert (width, height) == (50, 50)
assert png_size(inner) == (50, 50)
def test_non_image_payload_is_no_format_not_access_error(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Path
) -> None:
"""A body that is not a png really is "not the requested format".
Live sources answer with HTML error pages, licence documents and vector
files; those must produce the topic verdict ``no_format`` (exit 3) instead
of an access error or a partial download.
"""
registry = FakeRegistry(
[
FakeCandidate(
"fox", "https://example.test/a.png", "image/png", 100, 100, payload=b"not-an-image"
)
]
)
code, document = run_main(
monkeypatch, capsys, registry, *BASE_ARGS, "--resize", "--download", "--output-dir", str(tmp_path)
)
assert code == envelope.EXIT_NO_FORMAT
assert document["status"] == envelope.STATUS_NO_FORMAT
assert document["error"]["code"] == "no_format_match"
assert document["error"]["details"]["other_format_candidates"] == 1
assert document["items"] == []
assert list(tmp_path.iterdir()) == []