Dev#19
Merged
Merged
Conversation
…#179) Following the research summary at tareas/research_225/_proposal.html, this is a draft implementation of the six modality schemas for the APIPod common layer. Pushing as a proposal for review — happy to throw it away or rewrite. What it adds: - 12 Pydantic classes in apipod/common/schemas.py (Request + Response for image generation, video generation, audio, 3D, vision, multimodal embedding). - arbitrary_types_allowed on OpenAIBaseModel so media_toolkit's ImageFile / AudioFile / VideoFile can be used as field annotations. APIPod's file_handling_mixin converts URL / base64 inputs into those types before validation; the schemas don't need to do their own conversion. - 6 entries in _BaseLLMMixin._llm_configs mapping each new request to (response, endpoint_type). Endpoint type strings: image_generation, video_generation, audio, generation_3d, vision, embedding_multimodal. - 6 elif branches in _wrap_llm_response that build the right response object per endpoint_type. The chat / completion / embedding branches are the template. Two bugs in the existing auto-wrap path fixed on the same pass — without these the wrap never reaches the new branches: - FRICTION #15 (line 87): wrap was passing object='embedding' to EmbeddingResponse, but the schema declares object: Literal['list']. This is why qwen-models's /embed endpoint builds the response by hand. After the fix the auto-wrap path is safe; qwen-models can drop its manual EmbeddingResponse construction once this lands. - Line 81: int(datetime.now(timezone.utc)) raises TypeError on every call (datetime is not int-castable). Replaced with int(...timestamp()). The wrap path was unreachable before this fix, so I can't tell from git history when it broke — flagging in case it's intentional. Tests in test/llm/test_six_schemas.py (21 tests, all passing). No torch / transformers required. Covers: - Registry shape (9 entries in the tuple and in _llm_configs). - Required-field validation per new schema. - Documented defaults per new schema. - Response Literal['...'] validation per new modality. - Full wrap path for each new endpoint_type. - FRICTION #15 regression (outer object == 'list', data[i].object == 'embedding'). - Unknown endpoint_type still raises ValueError. What's out of scope for this PR (deliberate): - Generalising _BaseLLMMixin to _BaseModalityMixin (these new forms aren't LLM; the mixin name doesn't quite fit anymore). Architectural call. - Consolidating SUPPORTED_LLM_REQUEST_SCHEMAS (in schemas.py) and _llm_configs (in the mixin) into a single source of truth. Cleanup PR. - NSFW (#164) wiring — its own backlog item. - Wiring any of these schemas to a real model service. Questions for review: 1. Field set per schema — yes / change-this-field / split-into-two? 2. Class names — happy to swap. 3. The two existing bugs (FRICTION #15 + line 81 datetime). Want them in this PR or split out? Refs: tareas/research_225/_proposal.html
Adds two files under examples/ to validate the new six Pydantic schemas end-to-end as Matthias requested on PR #15: - examples/test_service_six_schemas.py: APIPod service exposing /vision (VisionRequest, image required) and /audio (AudioRequest, audio optional). Each endpoint echoes media properties so we can see if the file actually got deserialized into ImageFile/AudioFile. - examples/test_clients_six_schemas.py: runs three checks against the running service: HTTP request with base64, HTTP request with URL, and a fastSDK call via APISeex. Result on this branch (server starts cleanly): - HTTP base64: 422 "Input should be an instance of ImageFile" - HTTP URL: 422 same error - fastSDK: does not dispatch the request without a registered service (no ad-hoc URL path in FastClient / create_sdk) Confirms Matthias's hypothesis that media-file integration with the new schemas does not work out of the box. Root cause is that the fastapi file_handling_mixin converts URL/base64 to ImageFile for *loose endpoint parameters*, not for ImageFile fields declared inside a Pydantic body model. Fix is out of scope of this PR.
Per Carlos's review on PR #15: the previous name suggested the schemas were tied to OpenAI's own models, when the intent is the opposite. The shape mirrors OpenAI's API for client compatibility, but any provider (Flux, Stable Diffusion, ElevenLabs, Whisper, etc.) can sit behind it. Renames the base class and adds a module-level + class-level docstring making the provider-agnostic intent explicit. No behaviour change. The related concerns about apipod/engine/llm/_BaseLLMMixin (used for non-LLM endpoints like image/audio/video generation) are not addressed here on purpose, because that rename touches the runpod and fastapi backends and their tests. Tracked for a separate follow-up PR.
…es inside Pydantic bodies Matthias asked on PR #15 to make media-file integration actually work for the new schemas with JSON callers sending base64 or URLs. file_handling_mixin converts those for loose endpoint parameters but not for ImageFile / AudioFile / VideoFile / MediaFile fields declared inside a Pydantic body model, where Pydantic validates the string before the mixin gets a chance. This adds a model_validator(mode='before') on APIPodSchemaBase that walks model_fields, picks up media-typed fields, and converts string values: URLs and data URIs go through media_from_any, bare base64 strings go through the file class's from_base64 (media_from_any does not recognise bare base64). Non-string values pass through untouched so existing callers that already hand over an ImageFile instance keep working. E2E (examples/test_service_six_schemas.py + test_clients_six_schemas.py): HTTP base64 PASS, HTTP URL PASS, both return 200 with ImageFile in the echo. fastSDK still requires Registry setup as called out previously, no change. The 21 unit tests in test/llm/test_six_schemas.py still pass. examples/test_service_six_schemas.py: param renamed to 'payload' to match the router convention, response dicts use the actual VisionData / AudioData shapes (labels: List[VisionLabel] for vision, audio fields for audio).
Previous test file had 21 tests but coverage was uneven across modalities. Vision in particular had no wrap test, so the vision elif branch in _wrap_llm_response was never executed. Adds 10 tests to bring video / audio / 3d / vision to parity with image: - video missing-prompt validation - response object Literal["..."] check + wrong_object_raises for video, audio, 3d, vision - vision_wrap covering the previously unexercised elif branch Total: 21 -> 31, all passing.
Matthias pointed out that media_from_any already detects base64 without
a data: prefix. My earlier sniff was confused by a 1x1 test PNG (~67
bytes / 92 b64 chars), which sits below media_toolkit's sniffer length
threshold (it bails on short payloads to avoid false positives on
identifier-looking strings). With a realistic payload the heuristic
kicks in and a single media_from_any call covers URL, data URI, and
bare base64.
- pre-validator on APIPodSchemaBase now always routes through media_from_any.
The branch on ("http://", "https://", "data:") and the from_base64
fallback are gone.
- test_clients_six_schemas.py: build a 32x32 PNG (~100 bytes / ~140 b64
chars) instead of the 1x1, so the E2E base64 path exercises the same
shape a realistic JSON caller sends.
- test_six_schemas.py: replace the inline 1x1 _TINY_PNG_B64 constant with a
helper that builds the 32x32 PNG; documents the threshold reason inline.
E2E still passes (HTTP base64 200, HTTP URL 200, fastSDK still gated by
Registry setup as flagged before). All 31 unit tests pass.
…coraters to support media-toolkit file parsing in nested pydantic models. In schema endpoints the result is also wrapped into the corresponding response format if the user lazy returns just for example an AudioFile.
…o specify the fields of the LLM Chunks anymore
…ipod mimicking. Implented streaming capabilities for media files. Implemented auto-chunking.
… metrics - Implemented backend-neutral streaming with and centralized serialization logic. - Unified sync/async function execution across FastAPI and RunPod backends in . - Centralized endpoint planning and streaming detection into . - Refactored and to remove platform-specific leaks and unify progress reporting. - Introduced a dedicated class to consolidate job timing (created, queued, started, finished) and simplify lifecycle tracking. - Simplified to be self-describing, removing the need for external status mapping.
…ommand. apipod --simulate serverless-runpod. Introduced the simulate and --direct flags for developer flexibility. Introduced a cert for overwriting settings when managed by socaity
…ifferent build configs, endpoints, media support and streaming
…eeper than simply checking the string. This bug prevented newer python version to correclty render streaming endpoints in configuration mode local serverless with a local jobqueue and stream store in place
Test suite
…/SocAIty/socaity_router into feature/six-pydantic-schemas-179
…ctly wrapped. Added a debug file for local test debugging especially when working together with fastsdk.
…xecution plan. Only return the stream url if the endpoint supports streaming
… stream attribute from the pydantic model
…for generator expressions
Parse FileModel JSON strings in multipart fields and MediaList generic annotations. Coerce dict/JSON payloads into pydantic FileModels on input. Extend JobResult serialization and prefix serverless job hypermedia links. Serialize tuple returns like lists, media files to FileModel, and omit JobProgress from responses with a warning. Pass router mount prefix into status/cancel/stream links so prefixed services resolve correctly. Add include_router with mount prefix normalization on FastAPI and RunPod backends. Mirror FastAPI include_router for SocaityFastAPIRouter; normalize mount prefixes and propagate them to job link generation on both backends. Join binary stream chunks in aggregate_plain. Return concatenated bytes from plain stream aggregation instead of a chunk list. Update debug test services for prefixed router mounting and mixed_media JSON. Switch to app.include_router(child, prefix=...); return mixed_media as a JSON dict with base64 fields instead of a tuple. docs: note router mounting and job link prefix behavior in TECHNICAL_README
…plan Gateway-generated handlers are not async generators, so signature inspection alone cannot detect streaming endpoints. Expose GATEWAY_SUPPORTS_STREAMING_ATTR and let build_plan() treat handlers marked by registry metadata as streaming. Co-authored-by: Cursor <cursoragent@cursor.com>
Skip Security, Query, Body, and other FastAPI-injected parameters when validating schema endpoint signatures so dynamically built gateway handlers can carry auth context alongside the request model. Co-authored-by: Cursor <cursoragent@cursor.com>
Remote job queues such as the gateway RedisJobQueue build JobResult from ServiceJob records without a job_function. Use the queue override when present and guard the default path when job_function is missing. Co-authored-by: Cursor <cursoragent@cursor.com>
…rcular import Move the policies import inside _validate_schema_endpoint_signature so schema_resolve and signatures.policies do not import each other at module load time. Co-authored-by: Cursor <cursoragent@cursor.com>
Remote job queues invoke wrapped handlers at submit time and return submission metadata instead of inference output. Introduce a generic EnqueuePayload marker so integrators opt out of response-model coercion without coupling APIPod to platform-specific job param types. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep APIPod free of gateway/worker normalization and legacy aliases. Expose STREAM_WAIT_STATUSES for local /stream attach checks and clarify JobResult.status as a string that platform deployments may extend. Co-authored-by: Cursor <cursoragent@cursor.com>
… building Streaming is now detected purely from the function contract (generator, Iterator return annotation, or schema stream AST). Gateway handlers declare an Iterator return type instead of tagging a platform-specific attribute. is_streaming_endpoint resolves the schema binding itself when not provided. Co-authored-by: Cursor <cursoragent@cursor.com>
Remove gateway-specific introspection from the FastAPI router: no get_job_result override sniffing, no cancel_gateway_job getattr, no private _add_job calls. JobQueueInterface.add_job/get_job_result accept supports_streaming and link_prefix so queues own JobResult construction, and BaseJob carries supports_streaming set at enqueue time. Co-authored-by: Cursor <cursoragent@cursor.com>
The enum values and stream-wait set are exercised end to end by the streaming tests; the standalone constant checks added no coverage. Co-authored-by: Cursor <cursoragent@cursor.com>
Fix: gateway integration for remote job queues and dynamic endpoints There's open todos' please check them and make a task in the scrum board. I merge now to move on
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.