AFI conformance test suite
Substrate-level gate: same @client fixture registered in both backends
emits equivalent schemas, therefore the codegen produces equivalent
TypeScript regardless of which backend the frontend is generated against.
Catches adapter symmetry problems (Pydantic→OpenAPI converter divergence,
metadata leakage, ordering non-determinism) without docker, browser, or
Playwright.
What ships:
backends/mizan-fastapi/src/mizan_fastapi/schema.py — build_schema():
- Builds OpenAPI 3.0 from registered Mizan functions, mirroring the
shape mizan-django's export emits.
- Drives FastAPI's native OpenAPI generation by registering a stub POST
endpoint per function with its Input/Output Pydantic models, then
appends x-mizan-functions and x-mizan-contexts extensions.
- Param-elevation logic mirrors mizan-django/src/mizan/export/__init__.py
exactly (sharedBy tracking, required iff every function in context has
the param).
- snake_to_camel and metadata field shapes match Django for byte-equality
on the AFI surface.
tests/afi/ — the conformance harness:
- fixture.py: 5 @client functions covering the protocol axes (plain,
context, mutation+affects). No channels/forms — those aren't AFI-common.
- django_app/: minimal Django project (settings, urls, AppConfig.ready
registers the fixture). manage.py adds tests/afi/ to sys.path so both
backends import the same fixture module.
- fastapi_app.py: thin make_app() that registers fixture and mounts router.
- schema_normalizer.py: drops backend-specific framing — Ninja-vs-FastAPI
envelope differences (info/servers/tags), Django-only function fields
(form metadata), x-mizan-channels. Plus afi_subset() and
function_io_schemas() helpers for narrower comparisons.
- test_codegen_parity.py: three gates
1. x-mizan-functions match across backends
2. x-mizan-contexts match across backends
3. Per-function Input/Output OpenAPI schemas match (what codegen feeds
to openapi-typescript for type generation)
The full normalized OpenAPI envelopes do diverge — FastAPI adds
HTTPValidationError, the two converters wrap things slightly differently
in non-AFI-essential ways. That's not in the test scope. The codegen
only consumes x-mizan-functions, x-mizan-contexts, and the per-function
type schemas; those are what the test gates.
Makefile: test-afi target added; rolls into the test aggregate.
Verified: 3/3 conformance tests pass. Other surfaces unaffected —
mizan-core 15/15, mizan-django 348 pass, mizan-fastapi 11/11.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
88
tests/afi/test_codegen_parity.py
Normal file
88
tests/afi/test_codegen_parity.py
Normal file
@@ -0,0 +1,88 @@
|
||||
"""
|
||||
AFI conformance — same @client fixture, same schema, both adapters.
|
||||
|
||||
Gates that mizan-django and mizan-fastapi emit equivalent schemas for the
|
||||
same registered functions. If this passes, the codegen produces equivalent
|
||||
TypeScript output regardless of which backend the frontend is generated
|
||||
against (codegen is deterministic over schema input).
|
||||
|
||||
This is a substrate-level gate, not e2e. It catches adapter symmetry
|
||||
problems — Pydantic→OpenAPI converter divergence, metadata leakage,
|
||||
ordering non-determinism — without running a real frontend or backend.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
HERE = Path(__file__).parent
|
||||
DJANGO_MANAGE = HERE / "django_app" / "manage.py"
|
||||
|
||||
|
||||
# ─── Schema fetchers ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _fetch_django_schema() -> dict:
|
||||
"""Spawn Django's management command and parse its stdout JSON."""
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(DJANGO_MANAGE), "export_mizan_schema", "--indent", "0"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"},
|
||||
)
|
||||
if result.returncode != 0:
|
||||
pytest.fail(f"export_mizan_schema failed:\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}")
|
||||
return json.loads(result.stdout)
|
||||
|
||||
|
||||
def _fetch_fastapi_schema() -> dict:
|
||||
"""Build the FastAPI app inline (fresh registry) and call build_schema()."""
|
||||
sys.path.insert(0, str(HERE))
|
||||
try:
|
||||
from mizan_core.registry import clear_registry
|
||||
from mizan_fastapi import build_schema
|
||||
from fastapi_app import make_app
|
||||
|
||||
clear_registry()
|
||||
make_app()
|
||||
return build_schema()
|
||||
finally:
|
||||
sys.path.remove(str(HERE))
|
||||
|
||||
|
||||
# ─── Tests ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def schemas() -> tuple[dict, dict]:
|
||||
return _fetch_django_schema(), _fetch_fastapi_schema()
|
||||
|
||||
|
||||
class AFISubsetTests:
|
||||
"""The AFI surface — x-mizan-functions and x-mizan-contexts — must match."""
|
||||
|
||||
def test_x_mizan_functions_match(self, schemas):
|
||||
from schema_normalizer import afi_subset
|
||||
django, fastapi = schemas
|
||||
assert afi_subset(django)["x-mizan-functions"] == afi_subset(fastapi)["x-mizan-functions"]
|
||||
|
||||
def test_x_mizan_contexts_match(self, schemas):
|
||||
from schema_normalizer import afi_subset
|
||||
django, fastapi = schemas
|
||||
assert afi_subset(django)["x-mizan-contexts"] == afi_subset(fastapi)["x-mizan-contexts"]
|
||||
|
||||
|
||||
class TypeSchemasTests:
|
||||
"""Per-function Input/Output Pydantic→OpenAPI schemas — codegen feeds these to openapi-typescript."""
|
||||
|
||||
def test_function_io_schemas_match(self, schemas):
|
||||
from schema_normalizer import function_io_schemas
|
||||
django, fastapi = schemas
|
||||
assert function_io_schemas(django) == function_io_schemas(fastapi)
|
||||
Reference in New Issue
Block a user