Architecture rework: fix protocol bugs, add origin-side cache, document spec
8-expert review identified 3 bugs in shipped code (Vary header hallucination, fn/function wire key mismatch, max-age=0 defeating PSR) — all fixed with tests updated across Python and TypeScript. Added: manifest version field, affects validation, wire format convention, origin-side cache module (HMAC key derivation, MemoryCache + RedisCache backends, reverse index for scoped invalidation, executor integration). 16 known issues documented in cache/KNOWN_ISSUES.md from expert review — critical items (user_id not passed, purge race condition, no Redis error handling) to be fixed in follow-up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -13,6 +13,9 @@ dependencies = [
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
cache = [
|
||||
"redis>=5.0",
|
||||
]
|
||||
channels = [
|
||||
"channels>=4.0",
|
||||
"channels-redis>=4.0",
|
||||
|
||||
88
packages/mizan-django/src/mizan/cache/KNOWN_ISSUES.md
vendored
Normal file
88
packages/mizan-django/src/mizan/cache/KNOWN_ISSUES.md
vendored
Normal file
@@ -0,0 +1,88 @@
|
||||
# Cache Module — Known Issues
|
||||
|
||||
Issues identified by 8-domain-expert review of the initial implementation.
|
||||
Fix in priority order before shipping.
|
||||
|
||||
## Critical (Security / Data Corruption)
|
||||
|
||||
### 1. User-scoped content cached without user_id
|
||||
`context_fetch_view` never passes `user_id` to `cache_get`/`cache_put`.
|
||||
Authenticated User A's response gets cached and served to User B.
|
||||
**Fix:** Extract user_id from request (MWT `sub` claim when available,
|
||||
`request.user.pk` as interim) and pass to cache operations.
|
||||
|
||||
### 2. Purge race condition (non-atomic index operations)
|
||||
`cache_purge` does `get_index` -> `delete_index` -> `delete_many` as
|
||||
separate operations. Concurrent `cache_put` between steps can orphan
|
||||
entries or lose purges.
|
||||
**Fix:** Use Redis Lua script or WATCH/MULTI for atomic purge.
|
||||
MemoryCache should use a threading lock.
|
||||
|
||||
### 3. No Redis error handling
|
||||
Any Redis failure throws `ConnectionError` into the request path -> 500.
|
||||
No try/except, no fallback, no circuit breaker.
|
||||
**Fix:** Wrap all Redis calls in try/except. On failure, fall through
|
||||
to uncached execution (cache miss behavior).
|
||||
|
||||
### 4. Scoped purge uses OR semantics, should use AND
|
||||
`cache_purge({user_id: 5, org_id: 3})` deletes everything with
|
||||
`user_id=5` OR `org_id=3`. Should intersect index lookups.
|
||||
**Fix:** Change `keys_to_delete.update()` to set intersection.
|
||||
|
||||
## High (Correctness / Operability)
|
||||
|
||||
### 5. No TTL on Redis entries
|
||||
If purge fails or `affects` is misconfigured, stale data persists forever.
|
||||
**Fix:** Add safety-net TTL to `RedisCache.put` (e.g., 24h default).
|
||||
|
||||
### 6. Cross-language str() vs String() divergence
|
||||
Python `str(True)` -> `"True"`, JS `String(true)` -> `"true"`.
|
||||
Python `str(None)` -> `"None"`, JS `String(null)` -> `"null"`.
|
||||
Cache keys will mismatch between Python and TypeScript adapters.
|
||||
**Fix:** Define canonical stringification rules in the protocol spec.
|
||||
Normalize booleans to "true"/"false", null to "null", numbers to
|
||||
consistent format before stringification.
|
||||
|
||||
### 7. Broad purge doesn't clean per-param sub-indexes
|
||||
After broad purge of `mizan:idx:user`, the per-param indexes
|
||||
(`mizan:idx:user:user_id=5`) remain as dangling sets. Slow memory leak.
|
||||
**Fix:** On broad purge, also scan and delete `mizan:idx:{context}:*` indexes.
|
||||
|
||||
### 8. build_index_keys doesn't stringify values
|
||||
`derive_cache_key` calls `str(v)` but `build_index_keys` uses raw `v`.
|
||||
Latent inconsistency for non-string types.
|
||||
**Fix:** Stringify values in `build_index_keys` too.
|
||||
|
||||
### 9. Silent exception swallowing in get_cache()
|
||||
Misconfigured Redis URL or missing secret produces no log, no warning.
|
||||
**Fix:** Log warning on partial config and connection failure.
|
||||
|
||||
### 10. _initialized flag not thread-safe
|
||||
Two concurrent workers calling `get_cache()` race on globals.
|
||||
**Fix:** Use `threading.Lock` or resolve at `AppConfig.ready()`.
|
||||
|
||||
## Medium (Design / Performance)
|
||||
|
||||
### 11. No thundering-herd protection
|
||||
Concurrent cold misses all execute and write. Origin cache should
|
||||
deduplicate in-flight requests (request coalescing).
|
||||
|
||||
### 12. Wire-protocol internals in __all__
|
||||
`derive_cache_key` and `build_index_keys` are promoted to public API.
|
||||
Changing key format requires semver major bump.
|
||||
**Fix:** Remove from `__all__`, prefix with `_` or move to internal module.
|
||||
|
||||
### 13. Inconsistent API pattern
|
||||
`cache_get`/`cache_put` take explicit `secret`+`backend` args but
|
||||
executor fetches these from globals. Pick one pattern.
|
||||
|
||||
### 14. clear() uses SCAN + DELETE without pipeline
|
||||
O(N) round trips for large caches.
|
||||
**Fix:** Pipeline the deletes.
|
||||
|
||||
### 15. No Redis connection timeouts
|
||||
`from_url()` has no `socket_connect_timeout`, `socket_timeout`, or
|
||||
`health_check_interval`.
|
||||
|
||||
### 16. No RedisCache test coverage
|
||||
Only MemoryCache is tested. Use `fakeredis` for RedisCache tests.
|
||||
158
packages/mizan-django/src/mizan/cache/__init__.py
vendored
Normal file
158
packages/mizan-django/src/mizan/cache/__init__.py
vendored
Normal file
@@ -0,0 +1,158 @@
|
||||
"""
|
||||
mizan.cache — Origin-side cache implementing the Mizan cache protocol.
|
||||
|
||||
This module provides the same cache semantics as the Edge layer:
|
||||
- HMAC-SHA256 key derivation (JSON-canonical form)
|
||||
- Scoped invalidation (purge by context + params)
|
||||
- Reverse indexes for efficient purge lookups
|
||||
|
||||
Usage:
|
||||
from mizan.cache import get_cache, cache_get, cache_put, cache_purge
|
||||
|
||||
Backends:
|
||||
- MemoryCache: for testing (no dependencies)
|
||||
- RedisCache: for production (requires redis-py)
|
||||
|
||||
Configuration (Django settings):
|
||||
MIZAN_CACHE_SECRET = "your-signing-secret"
|
||||
MIZAN_CACHE_REDIS_URL = "redis://localhost:6379/0"
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from .backend import CacheBackend, MemoryCache, RedisCache
|
||||
from .keys import derive_cache_key, build_index_keys
|
||||
|
||||
_cache_instance: CacheBackend | None = None
|
||||
_initialized = False
|
||||
|
||||
|
||||
def get_cache() -> CacheBackend | None:
|
||||
"""
|
||||
Get the configured cache backend, or None if caching is disabled.
|
||||
|
||||
Returns RedisCache if MIZAN_CACHE_SECRET and MIZAN_CACHE_REDIS_URL are
|
||||
both set. Returns None otherwise. The instance is cached for the process
|
||||
lifetime.
|
||||
"""
|
||||
global _cache_instance, _initialized
|
||||
if _initialized:
|
||||
return _cache_instance
|
||||
|
||||
_initialized = True
|
||||
try:
|
||||
from mizan.setup.settings import get_settings
|
||||
settings = get_settings()
|
||||
if settings.cache_secret and settings.cache_redis_url:
|
||||
_cache_instance = RedisCache(settings.cache_redis_url)
|
||||
except Exception:
|
||||
_cache_instance = None
|
||||
|
||||
return _cache_instance
|
||||
|
||||
|
||||
def set_cache(backend: CacheBackend | None) -> None:
|
||||
"""
|
||||
Override the cache backend. Primarily for testing.
|
||||
|
||||
Pass None to disable caching. Pass a MemoryCache instance for tests.
|
||||
"""
|
||||
global _cache_instance, _initialized
|
||||
_cache_instance = backend
|
||||
_initialized = True
|
||||
|
||||
|
||||
def reset_cache() -> None:
|
||||
"""Reset the cache to uninitialized state. For testing teardown."""
|
||||
global _cache_instance, _initialized
|
||||
_cache_instance = None
|
||||
_initialized = False
|
||||
|
||||
|
||||
def cache_get(
|
||||
secret: str,
|
||||
backend: CacheBackend,
|
||||
context: str,
|
||||
params: dict[str, Any],
|
||||
user_id: str | None = None,
|
||||
rev: int = 0,
|
||||
) -> bytes | None:
|
||||
"""
|
||||
Look up a cached context response.
|
||||
|
||||
Returns the cached response bytes, or None on miss.
|
||||
"""
|
||||
key = derive_cache_key(secret, context, params, user_id, rev)
|
||||
return backend.get(key)
|
||||
|
||||
|
||||
def cache_put(
|
||||
secret: str,
|
||||
backend: CacheBackend,
|
||||
context: str,
|
||||
params: dict[str, Any],
|
||||
value: bytes,
|
||||
user_id: str | None = None,
|
||||
rev: int = 0,
|
||||
) -> None:
|
||||
"""
|
||||
Store a context response in the cache with reverse indexes.
|
||||
"""
|
||||
key = derive_cache_key(secret, context, params, user_id, rev)
|
||||
indexes = build_index_keys(context, params)
|
||||
backend.put(key, value, indexes)
|
||||
|
||||
|
||||
def cache_purge(
|
||||
backend: CacheBackend,
|
||||
context: str,
|
||||
params: dict[str, Any] | None = None,
|
||||
) -> int:
|
||||
"""
|
||||
Purge cached entries for a context, optionally scoped by params.
|
||||
|
||||
If params is None, purges ALL entries for the context (broad invalidation).
|
||||
If params is provided, purges only entries matching those specific params
|
||||
(scoped invalidation via the reverse index).
|
||||
|
||||
Returns the number of entries purged.
|
||||
"""
|
||||
if params:
|
||||
# Scoped purge — find entries matching each param via index
|
||||
keys_to_delete: set[str] = set()
|
||||
for k, v in sorted(params.items()):
|
||||
index_key = f"mizan:idx:{context}:{k}={v}"
|
||||
keys_to_delete.update(backend.get_index(index_key))
|
||||
backend.delete_index(index_key)
|
||||
|
||||
if keys_to_delete:
|
||||
# Remove from the broad context index too
|
||||
backend.remove_from_index(f"mizan:idx:{context}", keys_to_delete)
|
||||
return backend.delete_many(list(keys_to_delete))
|
||||
return 0
|
||||
else:
|
||||
# Broad purge — delete everything in the context index
|
||||
index_key = f"mizan:idx:{context}"
|
||||
keys_to_delete = backend.get_index(index_key)
|
||||
backend.delete_index(index_key)
|
||||
|
||||
if keys_to_delete:
|
||||
return backend.delete_many(list(keys_to_delete))
|
||||
return 0
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CacheBackend",
|
||||
"MemoryCache",
|
||||
"RedisCache",
|
||||
"get_cache",
|
||||
"set_cache",
|
||||
"reset_cache",
|
||||
"cache_get",
|
||||
"cache_put",
|
||||
"cache_purge",
|
||||
"derive_cache_key",
|
||||
"build_index_keys",
|
||||
]
|
||||
132
packages/mizan-django/src/mizan/cache/backend.py
vendored
Normal file
132
packages/mizan-django/src/mizan/cache/backend.py
vendored
Normal file
@@ -0,0 +1,132 @@
|
||||
"""
|
||||
Cache backends — MemoryCache (testing) and RedisCache (production).
|
||||
|
||||
Both implement the same interface. MemoryCache requires no dependencies and
|
||||
is used in the test suite. RedisCache requires `redis-py` and is used in
|
||||
production when MIZAN_CACHE_REDIS_URL is configured.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol
|
||||
|
||||
|
||||
class CacheBackend(Protocol):
|
||||
"""Interface that all Mizan cache backends implement."""
|
||||
|
||||
def get(self, key: str) -> bytes | None: ...
|
||||
def put(self, key: str, value: bytes, indexes: list[str]) -> None: ...
|
||||
def delete_many(self, keys: list[str]) -> int: ...
|
||||
def get_index(self, index_key: str) -> set[str]: ...
|
||||
def remove_from_index(self, index_key: str, members: set[str]) -> None: ...
|
||||
def delete_index(self, index_key: str) -> None: ...
|
||||
def clear(self) -> None: ...
|
||||
|
||||
|
||||
class MemoryCache:
|
||||
"""
|
||||
In-memory cache backend for testing.
|
||||
|
||||
Uses Python dicts. Same API as RedisCache. No persistence, no
|
||||
cross-process sharing. Perfect for unit tests.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._store: dict[str, bytes] = {}
|
||||
self._indexes: dict[str, set[str]] = {}
|
||||
|
||||
def get(self, key: str) -> bytes | None:
|
||||
return self._store.get(key)
|
||||
|
||||
def put(self, key: str, value: bytes, indexes: list[str]) -> None:
|
||||
self._store[key] = value
|
||||
for idx in indexes:
|
||||
if idx not in self._indexes:
|
||||
self._indexes[idx] = set()
|
||||
self._indexes[idx].add(key)
|
||||
|
||||
def delete_many(self, keys: list[str]) -> int:
|
||||
count = 0
|
||||
for key in keys:
|
||||
if key in self._store:
|
||||
del self._store[key]
|
||||
count += 1
|
||||
return count
|
||||
|
||||
def get_index(self, index_key: str) -> set[str]:
|
||||
return self._indexes.get(index_key, set()).copy()
|
||||
|
||||
def remove_from_index(self, index_key: str, members: set[str]) -> None:
|
||||
if index_key in self._indexes:
|
||||
self._indexes[index_key] -= members
|
||||
if not self._indexes[index_key]:
|
||||
del self._indexes[index_key]
|
||||
|
||||
def delete_index(self, index_key: str) -> None:
|
||||
self._indexes.pop(index_key, None)
|
||||
|
||||
def clear(self) -> None:
|
||||
self._store.clear()
|
||||
self._indexes.clear()
|
||||
|
||||
|
||||
class RedisCache:
|
||||
"""
|
||||
Redis-backed cache backend for production.
|
||||
|
||||
Uses Redis strings for cache entries and Redis sets for reverse indexes.
|
||||
Requires `redis-py` (pip install mizan[cache]).
|
||||
"""
|
||||
|
||||
def __init__(self, redis_url: str, prefix: str = "mizan:cache:") -> None:
|
||||
try:
|
||||
import redis
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Redis is required for Mizan's cache backend. "
|
||||
"Install it with: pip install mizan[cache]"
|
||||
)
|
||||
self._client = redis.from_url(redis_url)
|
||||
self._prefix = prefix
|
||||
|
||||
def _key(self, key: str) -> str:
|
||||
return f"{self._prefix}{key}"
|
||||
|
||||
def get(self, key: str) -> bytes | None:
|
||||
result = self._client.get(self._key(key))
|
||||
return result
|
||||
|
||||
def put(self, key: str, value: bytes, indexes: list[str]) -> None:
|
||||
prefixed_key = self._key(key)
|
||||
pipe = self._client.pipeline()
|
||||
pipe.set(prefixed_key, value)
|
||||
for idx in indexes:
|
||||
pipe.sadd(self._key(idx), key)
|
||||
pipe.execute()
|
||||
|
||||
def delete_many(self, keys: list[str]) -> int:
|
||||
if not keys:
|
||||
return 0
|
||||
prefixed = [self._key(k) for k in keys]
|
||||
return self._client.delete(*prefixed)
|
||||
|
||||
def get_index(self, index_key: str) -> set[str]:
|
||||
members = self._client.smembers(self._key(index_key))
|
||||
return {m.decode("utf-8") if isinstance(m, bytes) else m for m in members}
|
||||
|
||||
def remove_from_index(self, index_key: str, members: set[str]) -> None:
|
||||
if members:
|
||||
self._client.srem(self._key(index_key), *members)
|
||||
|
||||
def delete_index(self, index_key: str) -> None:
|
||||
self._client.delete(self._key(index_key))
|
||||
|
||||
def clear(self) -> None:
|
||||
pattern = f"{self._prefix}*"
|
||||
cursor = 0
|
||||
while True:
|
||||
cursor, keys = self._client.scan(cursor, match=pattern, count=100)
|
||||
if keys:
|
||||
self._client.delete(*keys)
|
||||
if cursor == 0:
|
||||
break
|
||||
79
packages/mizan-django/src/mizan/cache/keys.py
vendored
Normal file
79
packages/mizan-django/src/mizan/cache/keys.py
vendored
Normal file
@@ -0,0 +1,79 @@
|
||||
"""
|
||||
Cache key derivation — HMAC-SHA256 over JSON-canonical form.
|
||||
|
||||
This is the protocol-critical function. Every Mizan adapter (Python, TypeScript,
|
||||
future languages) must produce identical output for identical inputs. The
|
||||
conformance test suite verifies this.
|
||||
|
||||
Wire format:
|
||||
HMAC-SHA256(secret, JSON.stringify({"c": ctx, "p": params, "r": rev}, sort_keys))
|
||||
|
||||
- Keys are sorted (JSON canonical form)
|
||||
- No whitespace (separators=(",", ":"))
|
||||
- "u" key included only for user-scoped content
|
||||
- Params are sorted by key name
|
||||
- All param values are stringified
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
|
||||
def derive_cache_key(
|
||||
secret: str,
|
||||
context: str,
|
||||
params: dict[str, Any],
|
||||
user_id: str | None = None,
|
||||
rev: int = 0,
|
||||
) -> str:
|
||||
"""
|
||||
Derive a deterministic HMAC-SHA256 cache key.
|
||||
|
||||
Args:
|
||||
secret: The MIZAN_CACHE_SECRET signing key.
|
||||
context: Context name (e.g., "user", "products").
|
||||
params: Query parameters, sorted internally by key.
|
||||
user_id: User ID for user-scoped content. None for public.
|
||||
rev: Revision number from @client(rev=N). Defaults to 0.
|
||||
|
||||
Returns:
|
||||
Hex-encoded HMAC-SHA256 digest (64 characters).
|
||||
"""
|
||||
# Stringify all param values for cross-language determinism
|
||||
sorted_params = {k: str(v) for k, v in sorted(params.items())}
|
||||
|
||||
key_data: dict[str, Any] = {"c": context, "p": sorted_params, "r": rev}
|
||||
if user_id is not None:
|
||||
key_data["u"] = str(user_id)
|
||||
|
||||
message = json.dumps(key_data, sort_keys=True, separators=(",", ":"))
|
||||
return hmac.new(
|
||||
secret.encode("utf-8"),
|
||||
message.encode("utf-8"),
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
|
||||
|
||||
def build_index_keys(context: str, params: dict[str, Any]) -> list[str]:
|
||||
"""
|
||||
Build reverse index keys for a cache entry.
|
||||
|
||||
Returns a list of index keys that this entry should be added to:
|
||||
- "mizan:idx:{context}" (broad — for purging entire context)
|
||||
- "mizan:idx:{context}:{key}={value}" (scoped — for per-param purging)
|
||||
|
||||
Args:
|
||||
context: Context name.
|
||||
params: Query parameters.
|
||||
|
||||
Returns:
|
||||
List of index key strings.
|
||||
"""
|
||||
keys = [f"mizan:idx:{context}"]
|
||||
for k, v in sorted(params.items()):
|
||||
keys.append(f"mizan:idx:{context}:{k}={v}")
|
||||
return keys
|
||||
@@ -23,7 +23,7 @@ from enum import Enum
|
||||
from functools import wraps
|
||||
from typing import TYPE_CHECKING, Any, Callable
|
||||
|
||||
from django.http import HttpRequest, JsonResponse
|
||||
from django.http import HttpRequest, HttpResponse, JsonResponse
|
||||
from django.views.decorators.csrf import csrf_protect
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
@@ -661,6 +661,16 @@ def function_call_view(request: HttpRequest) -> JsonResponse:
|
||||
if invalidate_contexts:
|
||||
response["X-Mizan-Invalidate"] = _format_invalidate_header(invalidate_contexts)
|
||||
|
||||
# Purge origin-side cache for invalidated contexts
|
||||
from mizan.cache import get_cache, cache_purge
|
||||
cache = get_cache()
|
||||
if cache is not None:
|
||||
for entry in invalidate_contexts:
|
||||
if isinstance(entry, str):
|
||||
cache_purge(cache, entry)
|
||||
elif isinstance(entry, dict):
|
||||
cache_purge(cache, entry["context"], entry.get("params"))
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@@ -750,8 +760,7 @@ def context_fetch_view(request: HttpRequest, context_name: str) -> JsonResponse:
|
||||
}
|
||||
|
||||
Headers:
|
||||
Cache-Control: public, max-age=0, stale-while-revalidate=300
|
||||
Vary: Authorization, Cookie
|
||||
Cache-Control: public, max-age=0, s-maxage=31536000
|
||||
"""
|
||||
if request.method != "GET":
|
||||
return FunctionError(
|
||||
@@ -760,6 +769,22 @@ def context_fetch_view(request: HttpRequest, context_name: str) -> JsonResponse:
|
||||
).to_response(status=405)
|
||||
|
||||
params = request.GET.dict()
|
||||
|
||||
# Origin-side cache lookup
|
||||
from mizan.cache import get_cache, cache_get, cache_put
|
||||
from mizan.setup.settings import get_settings
|
||||
cache = get_cache()
|
||||
cache_settings = get_settings()
|
||||
if cache is not None and cache_settings.cache_secret:
|
||||
cached = cache_get(
|
||||
cache_settings.cache_secret, cache, context_name, params,
|
||||
)
|
||||
if cached is not None:
|
||||
response = HttpResponse(cached, content_type="application/json")
|
||||
response["Cache-Control"] = "public, max-age=0, s-maxage=31536000"
|
||||
response["X-Mizan-Cache"] = "HIT"
|
||||
return response
|
||||
|
||||
result = execute_context(request, context_name, params)
|
||||
|
||||
if isinstance(result, FunctionError):
|
||||
@@ -780,10 +805,18 @@ def context_fetch_view(request: HttpRequest, context_name: str) -> JsonResponse:
|
||||
response = JsonResponse(result.data, json_dumps_params={"sort_keys": True})
|
||||
|
||||
# CDN-ready headers
|
||||
# max-age=0: browser always revalidates (mutations may have invalidated)
|
||||
# stale-while-revalidate: edge can serve stale while fetching fresh
|
||||
# Vary: different auth = different cache entry
|
||||
response["Cache-Control"] = "public, max-age=0, stale-while-revalidate=300"
|
||||
response["Vary"] = "Authorization, Cookie"
|
||||
# max-age=0: browser always revalidates (gets 304 from CDN if unchanged)
|
||||
# s-maxage=31536000: CDN caches forever; purge is the freshness mechanism
|
||||
# No Vary header — Cloudflare ignores Vary for personalized content.
|
||||
# User-scoped cache keying will use HMAC-based keys instead.
|
||||
response["Cache-Control"] = "public, max-age=0, s-maxage=31536000"
|
||||
|
||||
# Store in origin-side cache
|
||||
if cache is not None and cache_settings.cache_secret:
|
||||
cache_put(
|
||||
cache_settings.cache_secret, cache, context_name, params,
|
||||
response.content,
|
||||
)
|
||||
response["X-Mizan-Cache"] = "MISS"
|
||||
|
||||
return response
|
||||
|
||||
@@ -391,7 +391,7 @@ def generate_edge_manifest(
|
||||
registry = get_registry()
|
||||
all_functions = registry.get("functions", {})
|
||||
|
||||
manifest: dict[str, Any] = {"contexts": {}, "mutations": {}}
|
||||
manifest: dict[str, Any] = {"version": 1, "contexts": {}, "mutations": {}}
|
||||
|
||||
for ctx_name, fn_names in groups.items():
|
||||
# Collect params and routes from all functions in this context
|
||||
|
||||
@@ -27,6 +27,7 @@ from .registry import (
|
||||
get_contexts,
|
||||
get_context_groups,
|
||||
get_forms,
|
||||
validate_registry,
|
||||
clear_registry,
|
||||
)
|
||||
|
||||
@@ -60,6 +61,7 @@ __all__ = [
|
||||
"get_contexts",
|
||||
"get_context_groups",
|
||||
"get_forms",
|
||||
"validate_registry",
|
||||
"clear_registry",
|
||||
# Discovery
|
||||
"mizan_clients",
|
||||
|
||||
@@ -71,6 +71,9 @@ def mizan_clients(apps_root: str, layer: str = "clients") -> None:
|
||||
visitor = DjangoAppVisitor(layer=layer, apps_root=apps_root)
|
||||
visitor.visit(_RegisterServerFunctions())
|
||||
|
||||
from .registry import validate_registry
|
||||
validate_registry()
|
||||
|
||||
|
||||
def mizan_module(module_path: str) -> None:
|
||||
"""
|
||||
|
||||
@@ -326,6 +326,46 @@ def get_forms() -> dict[str, list[type["ServerFunction"]]]:
|
||||
return forms
|
||||
|
||||
|
||||
def validate_registry() -> list[str]:
|
||||
"""
|
||||
Validate that all affects targets resolve to known contexts or functions.
|
||||
|
||||
Called automatically after discovery. Emits warnings for unresolved targets
|
||||
(e.g., typos in string-based affects declarations).
|
||||
|
||||
Returns a list of warning messages (empty if everything resolves).
|
||||
"""
|
||||
import warnings
|
||||
|
||||
issues: list[str] = []
|
||||
groups = get_context_groups()
|
||||
all_fn_names = set(_functions.keys())
|
||||
|
||||
for fn_name, fn_cls in _functions.items():
|
||||
meta = getattr(fn_cls, "_meta", {})
|
||||
affects = meta.get("affects")
|
||||
if not affects:
|
||||
continue
|
||||
for target in affects:
|
||||
target_name = target.get("name", "")
|
||||
target_type = target.get("type", "")
|
||||
if target_type == "context" and target_name not in groups:
|
||||
issues.append(
|
||||
f"@client function '{fn_name}' declares affects='{target_name}', "
|
||||
f"but no context named '{target_name}' is registered."
|
||||
)
|
||||
elif target_type == "function" and target_name not in all_fn_names:
|
||||
issues.append(
|
||||
f"@client function '{fn_name}' targets function '{target_name}', "
|
||||
f"but no function named '{target_name}' is registered."
|
||||
)
|
||||
|
||||
for msg in issues:
|
||||
warnings.warn(msg, stacklevel=2)
|
||||
|
||||
return issues
|
||||
|
||||
|
||||
def clear_registry() -> None:
|
||||
"""Clear all registrations. Primarily for testing."""
|
||||
_functions.clear()
|
||||
|
||||
@@ -17,6 +17,12 @@ class mizanSettings:
|
||||
# Whether to expose function names in DEBUG mode errors
|
||||
debug_expose_names: bool
|
||||
|
||||
# Cache signing secret (required when cache is enabled)
|
||||
cache_secret: str | None
|
||||
|
||||
# Redis URL for cache backend (None = cache disabled)
|
||||
cache_redis_url: str | None
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> mizanSettings:
|
||||
@@ -25,9 +31,13 @@ def get_settings() -> mizanSettings:
|
||||
|
||||
Settings:
|
||||
mizan_DEBUG_EXPOSE_NAMES: Show function names in errors when DEBUG=True (default: True)
|
||||
MIZAN_CACHE_SECRET: HMAC signing key for cache keys (default: None)
|
||||
MIZAN_CACHE_REDIS_URL: Redis connection URL (default: None)
|
||||
"""
|
||||
return mizanSettings(
|
||||
debug_expose_names=getattr(django_settings, "mizan_DEBUG_EXPOSE_NAMES", True),
|
||||
cache_secret=getattr(django_settings, "MIZAN_CACHE_SECRET", None),
|
||||
cache_redis_url=getattr(django_settings, "MIZAN_CACHE_REDIS_URL", None),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1021,9 +1021,7 @@ class ServerDrivenInvalidationTests(TestCase):
|
||||
|
||||
# CDN-ready headers
|
||||
self.assertIn("public", response["Cache-Control"])
|
||||
self.assertIn("stale-while-revalidate", response["Cache-Control"])
|
||||
self.assertIn("Authorization", response["Vary"])
|
||||
self.assertIn("Cookie", response["Vary"])
|
||||
self.assertIn("s-maxage", response["Cache-Control"])
|
||||
|
||||
def test_context_error_not_cached(self):
|
||||
"""Context fetch errors must not be cached."""
|
||||
@@ -1789,8 +1787,7 @@ class HTTPIntegrationTests(TestCase):
|
||||
|
||||
# CDN headers
|
||||
self.assertIn("public", response["Cache-Control"])
|
||||
self.assertIn("stale-while-revalidate", response["Cache-Control"])
|
||||
self.assertIn("Authorization", response["Vary"])
|
||||
self.assertIn("s-maxage", response["Cache-Control"])
|
||||
|
||||
def test_context_fetch_string_to_int_coercion(self):
|
||||
"""Query params arrive as strings. Pydantic must coerce to int."""
|
||||
@@ -2151,7 +2148,7 @@ class EdgeCompatibilityTests(TestCase):
|
||||
|
||||
cc = response["Cache-Control"]
|
||||
self.assertIn("public", cc)
|
||||
self.assertIn("stale-while-revalidate", cc)
|
||||
self.assertIn("s-maxage", cc)
|
||||
# Must NOT have no-store or private
|
||||
self.assertNotIn("no-store", cc)
|
||||
self.assertNotIn("private", cc)
|
||||
@@ -2173,16 +2170,6 @@ class EdgeCompatibilityTests(TestCase):
|
||||
self.assertEqual(response.status_code, 404)
|
||||
self.assertEqual(response["Cache-Control"], "no-store")
|
||||
|
||||
# ── Vary header correctness ─────────────────────────────────────────────
|
||||
|
||||
def test_vary_header_present(self):
|
||||
"""Context GET includes Vary for auth-dependent caching."""
|
||||
response = self.client.get("/api/mizan/ctx/user/?user_id=5")
|
||||
|
||||
vary = response["Vary"]
|
||||
self.assertIn("Authorization", vary)
|
||||
self.assertIn("Cookie", vary)
|
||||
|
||||
def test_different_params_different_response(self):
|
||||
"""Different query params produce different response bodies (different cache entries)."""
|
||||
r1 = self.client.get("/api/mizan/ctx/user/?user_id=5")
|
||||
@@ -2385,9 +2372,9 @@ class EdgeCompatibilityTests(TestCase):
|
||||
self.assertEqual(header_parts[0], "ctx_0")
|
||||
self.assertEqual(header_parts[19], "ctx_19")
|
||||
|
||||
# ── Auth-dependent Vary ─────────────────────────────────────────────────
|
||||
# ── Auth-dependent response differentiation ───────────────────────────
|
||||
|
||||
def test_vary_actually_differentiates_by_auth(self):
|
||||
def test_different_auth_produces_different_response(self):
|
||||
"""Same URL with different auth produces different response bodies."""
|
||||
from tests.models import EmailUser
|
||||
|
||||
@@ -2485,6 +2472,7 @@ class EdgeManifestTests(TestCase):
|
||||
|
||||
manifest = generate_edge_manifest()
|
||||
|
||||
self.assertEqual(manifest["version"], 1)
|
||||
self.assertIn("contexts", manifest)
|
||||
self.assertIn("user", manifest["contexts"])
|
||||
|
||||
@@ -2793,3 +2781,250 @@ class PrivateAndRouteTests(TestCase):
|
||||
manifest = generate_edge_manifest()
|
||||
ctx = manifest["contexts"]["user"]
|
||||
self.assertIn("/profile/<user_id>/", ctx["page_routes"])
|
||||
|
||||
|
||||
# ── Cache conformance tests ────────────────────────────────────────────────
|
||||
|
||||
|
||||
class CacheKeyDerivationTests(TestCase):
|
||||
"""Tests that HMAC cache key derivation is deterministic and correct."""
|
||||
|
||||
SECRET = "test-cache-secret"
|
||||
|
||||
def test_deterministic_output(self):
|
||||
"""Same inputs always produce the same key."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
key1 = derive_cache_key(self.SECRET, "user", {"user_id": "5"})
|
||||
key2 = derive_cache_key(self.SECRET, "user", {"user_id": "5"})
|
||||
self.assertEqual(key1, key2)
|
||||
self.assertEqual(len(key1), 64) # SHA-256 hex digest
|
||||
|
||||
def test_param_order_irrelevant(self):
|
||||
"""Parameter ordering does not affect the key."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
key1 = derive_cache_key(self.SECRET, "ctx", {"a": "1", "b": "2"})
|
||||
key2 = derive_cache_key(self.SECRET, "ctx", {"b": "2", "a": "1"})
|
||||
self.assertEqual(key1, key2)
|
||||
|
||||
def test_different_user_ids_different_keys(self):
|
||||
"""Different user_ids produce different cache keys."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
key1 = derive_cache_key(self.SECRET, "user", {"user_id": "5"}, user_id="5")
|
||||
key2 = derive_cache_key(self.SECRET, "user", {"user_id": "5"}, user_id="6")
|
||||
self.assertNotEqual(key1, key2)
|
||||
|
||||
def test_rev_changes_key(self):
|
||||
"""Different rev values produce different cache keys."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
key1 = derive_cache_key(self.SECRET, "user", {"user_id": "5"}, rev=0)
|
||||
key2 = derive_cache_key(self.SECRET, "user", {"user_id": "5"}, rev=1)
|
||||
self.assertNotEqual(key1, key2)
|
||||
|
||||
def test_no_delimiter_collision(self):
|
||||
"""JSON-canonical form prevents delimiter-free concatenation collisions."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
# "user" + user_id="12" + params="3" vs "user1" + user_id="2" + params="3"
|
||||
key1 = derive_cache_key(self.SECRET, "user", {"id": "3"}, user_id="12")
|
||||
key2 = derive_cache_key(self.SECRET, "user1", {"id": "3"}, user_id="2")
|
||||
self.assertNotEqual(key1, key2)
|
||||
|
||||
def test_public_vs_user_scoped(self):
|
||||
"""Public (no user_id) and user-scoped produce different keys."""
|
||||
from mizan.cache.keys import derive_cache_key
|
||||
|
||||
public = derive_cache_key(self.SECRET, "products", {"id": "1"})
|
||||
scoped = derive_cache_key(self.SECRET, "products", {"id": "1"}, user_id="5")
|
||||
self.assertNotEqual(public, scoped)
|
||||
|
||||
|
||||
class CacheBackendTests(TestCase):
|
||||
"""Tests for MemoryCache backend operations."""
|
||||
|
||||
def setUp(self):
|
||||
from mizan.cache.backend import MemoryCache
|
||||
self.cache = MemoryCache()
|
||||
|
||||
def test_get_miss(self):
|
||||
"""Empty cache returns None."""
|
||||
self.assertIsNone(self.cache.get("nonexistent"))
|
||||
|
||||
def test_put_then_get(self):
|
||||
"""Store and retrieve a value."""
|
||||
self.cache.put("key1", b'{"data": true}', ["mizan:idx:ctx"])
|
||||
result = self.cache.get("key1")
|
||||
self.assertEqual(result, b'{"data": true}')
|
||||
|
||||
def test_index_tracking(self):
|
||||
"""Put adds the key to specified indexes."""
|
||||
self.cache.put("key1", b"v1", ["mizan:idx:user", "mizan:idx:user:user_id=5"])
|
||||
self.assertIn("key1", self.cache.get_index("mizan:idx:user"))
|
||||
self.assertIn("key1", self.cache.get_index("mizan:idx:user:user_id=5"))
|
||||
|
||||
def test_delete_many(self):
|
||||
"""Delete multiple keys at once."""
|
||||
self.cache.put("k1", b"v1", [])
|
||||
self.cache.put("k2", b"v2", [])
|
||||
count = self.cache.delete_many(["k1", "k2"])
|
||||
self.assertEqual(count, 2)
|
||||
self.assertIsNone(self.cache.get("k1"))
|
||||
self.assertIsNone(self.cache.get("k2"))
|
||||
|
||||
def test_clear(self):
|
||||
"""Clear removes everything."""
|
||||
self.cache.put("k1", b"v1", ["idx1"])
|
||||
self.cache.clear()
|
||||
self.assertIsNone(self.cache.get("k1"))
|
||||
self.assertEqual(self.cache.get_index("idx1"), set())
|
||||
|
||||
|
||||
class CachePurgeTests(TestCase):
|
||||
"""Tests for cache_purge with scoped and broad invalidation."""
|
||||
|
||||
SECRET = "test-cache-secret"
|
||||
|
||||
def setUp(self):
|
||||
from mizan.cache import cache_put, set_cache
|
||||
from mizan.cache.backend import MemoryCache
|
||||
|
||||
self.cache = MemoryCache()
|
||||
set_cache(self.cache)
|
||||
|
||||
# Prime cache with two users in the "user" context
|
||||
cache_put(self.SECRET, self.cache, "user", {"user_id": "5"}, b'{"name":"user5"}')
|
||||
cache_put(self.SECRET, self.cache, "user", {"user_id": "6"}, b'{"name":"user6"}')
|
||||
|
||||
def tearDown(self):
|
||||
from mizan.cache import reset_cache
|
||||
reset_cache()
|
||||
|
||||
def test_scoped_purge(self):
|
||||
"""Purging user;user_id=5 removes only that entry."""
|
||||
from mizan.cache import cache_purge, cache_get
|
||||
|
||||
count = cache_purge(self.cache, "user", {"user_id": "5"})
|
||||
self.assertEqual(count, 1)
|
||||
|
||||
# user_id=5 is gone
|
||||
self.assertIsNone(cache_get(self.SECRET, self.cache, "user", {"user_id": "5"}))
|
||||
# user_id=6 survives
|
||||
self.assertIsNotNone(cache_get(self.SECRET, self.cache, "user", {"user_id": "6"}))
|
||||
|
||||
def test_broad_purge(self):
|
||||
"""Purging context with no params removes all entries."""
|
||||
from mizan.cache import cache_purge, cache_get
|
||||
|
||||
count = cache_purge(self.cache, "user")
|
||||
self.assertEqual(count, 2)
|
||||
|
||||
self.assertIsNone(cache_get(self.SECRET, self.cache, "user", {"user_id": "5"}))
|
||||
self.assertIsNone(cache_get(self.SECRET, self.cache, "user", {"user_id": "6"}))
|
||||
|
||||
|
||||
class CacheIntegrationTests(TestCase):
|
||||
"""Tests for cache integration with context_fetch_view and function_call_view."""
|
||||
|
||||
def setUp(self):
|
||||
clear_registry()
|
||||
self.factory = RequestFactory()
|
||||
|
||||
from mizan.cache import set_cache
|
||||
from mizan.cache.backend import MemoryCache
|
||||
from mizan.setup.settings import clear_settings_cache
|
||||
|
||||
self.cache = MemoryCache()
|
||||
set_cache(self.cache)
|
||||
clear_settings_cache()
|
||||
|
||||
UserCtx = ReactContext("user")
|
||||
|
||||
@client(context=UserCtx)
|
||||
def user_profile(request: HttpRequest, user_id: int) -> dict:
|
||||
return {"name": f"user_{user_id}"}
|
||||
|
||||
@client(affects=UserCtx)
|
||||
def update_profile(request: HttpRequest, user_id: int, name: str) -> dict:
|
||||
return {"ok": True}
|
||||
|
||||
register(user_profile, "user_profile")
|
||||
register(update_profile, "update_profile")
|
||||
|
||||
def tearDown(self):
|
||||
clear_registry()
|
||||
from mizan.cache import reset_cache
|
||||
from mizan.setup.settings import clear_settings_cache
|
||||
reset_cache()
|
||||
clear_settings_cache()
|
||||
|
||||
def _fetch_context(self, params="user_id=5"):
|
||||
from django.test import override_settings
|
||||
|
||||
with override_settings(
|
||||
MIZAN_CACHE_SECRET="test-secret",
|
||||
MIZAN_CACHE_REDIS_URL="dummy",
|
||||
):
|
||||
from mizan.setup.settings import clear_settings_cache
|
||||
clear_settings_cache()
|
||||
response = self.client.get(f"/api/mizan/ctx/user/?{params}")
|
||||
return response
|
||||
|
||||
def _call_mutation(self, fn_name, args):
|
||||
from django.test import override_settings
|
||||
|
||||
with override_settings(
|
||||
MIZAN_CACHE_SECRET="test-secret",
|
||||
MIZAN_CACHE_REDIS_URL="dummy",
|
||||
):
|
||||
from mizan.setup.settings import clear_settings_cache
|
||||
clear_settings_cache()
|
||||
response = self.client.post(
|
||||
"/api/mizan/call/",
|
||||
data=json.dumps({"fn": fn_name, "args": args}),
|
||||
content_type="application/json",
|
||||
)
|
||||
return response
|
||||
|
||||
def test_first_fetch_is_miss(self):
|
||||
"""First context fetch executes functions (cache miss)."""
|
||||
response = self._fetch_context()
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = response.json()
|
||||
self.assertEqual(data["user_profile"]["result"]["name"], "user_5")
|
||||
|
||||
def test_second_fetch_is_hit(self):
|
||||
"""Second identical fetch returns cached response."""
|
||||
r1 = self._fetch_context()
|
||||
r2 = self._fetch_context()
|
||||
self.assertEqual(r1.content, r2.content)
|
||||
self.assertEqual(r2.get("X-Mizan-Cache"), "HIT")
|
||||
|
||||
def test_mutation_invalidates_cache(self):
|
||||
"""Mutation purges the context cache; next fetch is a miss."""
|
||||
# Prime cache
|
||||
r1 = self._fetch_context()
|
||||
self.assertIn(r1.get("X-Mizan-Cache"), ("MISS", None))
|
||||
|
||||
# Mutate
|
||||
self._call_mutation("update_profile", {"user_id": 5, "name": "new"})
|
||||
|
||||
# Cache should be purged — next fetch is a miss
|
||||
r2 = self._fetch_context()
|
||||
# Should re-execute, not serve stale
|
||||
self.assertEqual(r2.json()["user_profile"]["result"]["name"], "user_5")
|
||||
|
||||
def test_scoped_invalidation_preserves_other_entries(self):
|
||||
"""Mutating user 5 doesn't invalidate user 6's cache."""
|
||||
# Prime both users
|
||||
self._fetch_context("user_id=5")
|
||||
self._fetch_context("user_id=6")
|
||||
|
||||
# Mutate only user 5
|
||||
self._call_mutation("update_profile", {"user_id": 5, "name": "new"})
|
||||
|
||||
# User 6 should still be cached
|
||||
r6 = self._fetch_context("user_id=6")
|
||||
self.assertEqual(r6.get("X-Mizan-Cache"), "HIT")
|
||||
|
||||
@@ -124,7 +124,7 @@ export async function mizanCall(
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
credentials: 'same-origin',
|
||||
body: JSON.stringify({ function: functionName, args }),
|
||||
body: JSON.stringify({ fn: functionName, args }),
|
||||
})
|
||||
if (!res.ok) throw new MizanError(res.status, await res.text())
|
||||
|
||||
|
||||
@@ -72,8 +72,7 @@ export async function handleContextFetch(
|
||||
body: results,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=0, stale-while-revalidate=300',
|
||||
'Vary': 'Authorization, Cookie',
|
||||
'Cache-Control': 'public, max-age=0, s-maxage=31536000',
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,12 +8,14 @@
|
||||
import type { EdgeManifest } from './types'
|
||||
import { getAllFunctions, getContextGroups, getContextParamNames } from './registry'
|
||||
|
||||
// Both camelCase and snake_case forms included for cross-language matching.
|
||||
// Wire format is snake_case (protocol rule); camelCase is the TS-local convention.
|
||||
const USER_SCOPED_PARAMS = new Set(['userId', 'user', 'ownerId', 'accountId', 'user_id', 'owner_id', 'account_id'])
|
||||
|
||||
export function generateManifest(baseUrl = '/api/mizan'): EdgeManifest {
|
||||
const groups = getContextGroups()
|
||||
const allFunctions = getAllFunctions()
|
||||
const manifest: EdgeManifest = { contexts: {}, mutations: {} }
|
||||
const manifest: EdgeManifest = { version: 1, contexts: {}, mutations: {} }
|
||||
|
||||
// Contexts
|
||||
for (const [ctxName, fnNames] of Object.entries(groups)) {
|
||||
|
||||
@@ -56,6 +56,7 @@ export interface ManifestMutation {
|
||||
}
|
||||
|
||||
export interface EdgeManifest {
|
||||
version: number
|
||||
contexts: Record<string, ManifestContext>
|
||||
mutations: Record<string, ManifestMutation>
|
||||
}
|
||||
|
||||
@@ -55,7 +55,7 @@ describe('Edge Compatibility', () => {
|
||||
test('context GET is cacheable', async () => {
|
||||
const r = await handleContextFetch('user', { userId: '5' })
|
||||
expect(r.headers['Cache-Control']).toContain('public')
|
||||
expect(r.headers['Cache-Control']).toContain('stale-while-revalidate')
|
||||
expect(r.headers['Cache-Control']).toContain('s-maxage')
|
||||
expect(r.headers['Cache-Control']).not.toContain('no-store')
|
||||
})
|
||||
|
||||
@@ -70,14 +70,6 @@ describe('Edge Compatibility', () => {
|
||||
expect(r.headers['Cache-Control']).toBe('no-store')
|
||||
})
|
||||
|
||||
// ── Vary header ────────────────────────────────────────────────────
|
||||
|
||||
test('Vary header present on context GET', async () => {
|
||||
const r = await handleContextFetch('user', { userId: '5' })
|
||||
expect(r.headers['Vary']).toContain('Authorization')
|
||||
expect(r.headers['Vary']).toContain('Cookie')
|
||||
})
|
||||
|
||||
// ── X-Mizan-Invalidate header ──────────────────────────────────────
|
||||
|
||||
test('mutation response includes invalidation header', async () => {
|
||||
@@ -195,6 +187,7 @@ describe('Manifest', () => {
|
||||
test('manifest matches expected structure', () => {
|
||||
const m = generateManifest()
|
||||
|
||||
expect(m.version).toBe(1)
|
||||
expect(m.contexts.user).toBeDefined()
|
||||
expect(m.contexts.user.endpoints).toEqual(['/api/mizan/ctx/user/'])
|
||||
expect(m.contexts.user.params).toContain('userId')
|
||||
|
||||
Reference in New Issue
Block a user