feat(gateway): typed DTO contract for HTTP responses and OpenAPI schema#403
Draft
bburda wants to merge 51 commits into
Draft
feat(gateway): typed DTO contract for HTTP responses and OpenAPI schema#403bburda wants to merge 51 commits into
bburda wants to merge 51 commits into
Conversation
Name the unnamed std::index_sequence tag parameters in variant_schema() and collect_impl() to satisfy readability-named-parameter. These are the only two clang-tidy findings in the new dto/ headers.
Adds entities.hpp with 8 entity DTOs (list-item + detail pairs for Area, Component, App, Function), the Collection<T> wrapper with per-instantiation dto_name specializations, and populates AllDtos in registry.hpp with all entity, x-medkit, error, and collection DTOs. Extends test_dto_contract.cpp with EveryRegisteredDtoRoundTrips, which exercises every registered DTO through SchemaWriter, JsonWriter, and JsonReader.
Include ros2_medkit_gateway/dto/registry.hpp and convert the static-const map in SchemaBuilder::component_schemas() to an IIFE so the DTO-generated schemas can be merged in at initialisation time. The DTO version wins on name collisions (currently only GenericError); hand-written factory calls are preserved intact so existing $ref targets remain valid until the per-domain migration tasks remove them. Store dto::collect_component_schemas() in a named local variable before iterating - calling .items() directly on the returned temporary creates a dangling reference inside the nlohmann iteration_proxy, which would cause invalid_iterator.214 at runtime.
Template overloads on RouteEntry delegate to the existing raw 3-arg overloads, generating a $ref to the DTO's components/schemas name via dto::dto_name<T>. Non-template calls are unaffected (non-templates win overload resolution).
Replace hand-built nlohmann::json + XMedkit fluent builder with typed dto::Collection<dto::AreaListItem> and dto::XMedkitArea structs. Adds "type": "area" field per item (deliberate per issue #338 - entities must carry their type). Wire behavior otherwise preserved field-for-field.
Migrate handle_get_area, handle_list_components, handle_get_component, handle_list_apps, handle_get_app, handle_list_functions, and handle_get_function to typed dto:: structs with send_dto(). Entity list items now carry a required "type" discriminator field per issue #338 intent. All 2169 gateway unit tests pass.
Replace all EntityList/EntityDetail $ref usages with typed DTO schema names (AreaList, ComponentList, AppList, FunctionList, AreaDetail, ComponentDetail, AppDetail, FunctionDetail). Each top-level collection, entity detail, and sub-collection route now references the concrete DTO schema that matches what the handler actually emits. Delete SchemaBuilder::entity_detail_schema() and entity_list_schema() which are fully superseded by the DTO layer. Remove their entries from component_schemas() - the DTO-generated schemas win on all collisions. Add entity_type_to_list_name/detail_name helpers in path_builder.cpp so build_entity_collection/build_entity_detail emit $ref to the correct DTO schema for each entity keyword. Update test_schema_builder and test_path_builder: remove tests for the deleted factories and replace with assertions against the DTO registry. AllRefsResolveToRegisteredSchemas passes with zero dangling $ref.
Add FaultListItem, FaultStatus, FaultItem, FaultEnvironmentData, FaultXMedkit, FaultDetail and Collection<FaultListItem> (named FaultList) DTOs matching the exact wire shapes of fault_msg_conversions.cpp and FaultHandlers::build_sovd_fault_response. Register all new types in AllDtos; EveryRegisteredDtoRoundTrips passes.
- generic_error() now delegates to SchemaWriter<dto::GenericError>::schema()
(DTO is source of truth; deleted the redundant hand-built implementation)
- build_sovd_fault_response() returns dto::FaultDetail instead of nlohmann::json;
handle_get_fault calls HandlerContext::send_dto(res, detail)
- path_builder: build_faults_collection emits ref("FaultList") instead of
inline fault_list_schema()
- Deleted fault_list_item_schema(), fault_detail_schema(), fault_list_schema()
from SchemaBuilder; FaultListItem/FaultDetail/FaultList are now emitted by
dto::collect_component_schemas() via AllDtos
- Updated test_schema_builder, test_path_builder, test_fault_handlers to
assert against DTO-generated schemas and the dto::FaultDetail struct
- AllRefsResolveToRegisteredSchemas and EveryRegisteredDtoRoundTrips pass;
91/91 gateway unit tests green
…t handlers Add FaultListXMedkit and FaultListAggXMedkit typed DTO structs covering the five fault-list response shapes (global, per-app, function/component/area aggregated). Replace all XMedkit fluent builder usages in fault_handlers.cpp with typed struct construction serialized via dto::JsonWriter; use the json-overload of merge_peer_items for fan-out partial/failed_peers injection. Remove the #include of core/http/x_medkit.hpp from fault_handlers.cpp. Register the two new structs in AllDtos so EveryRegisteredDtoRoundTrips and AllRefsResolveToRegisteredSchemas cover them.
Introduce typed DTO structs for the CONFIGURATIONS domain: - ConfigXMedkitItem: per-item x-medkit in list responses - ConfigurationMetaData: list item (id, name, type, x-medkit) - Collection<ConfigurationMetaData> named "ConfigurationList" - ConfigListXMedkit: x-medkit on list response root - ConfigValueXMedkit: x-medkit on GET/PUT value responses - ConfigurationReadValue: GET/PUT response shape - ConfigurationWriteRequest: PUT request body - ConfigurationDeleteResultItem: 207 multi-status result entry - ConfigurationDeleteMultiStatus: 207 multi-status response body Provide dto_sample specializations for the two DTOs that carry a non-optional nlohmann::json field (data), ensuring EveryRegisteredDtoRoundTrips passes. Register all new types in AllDtos in registry.hpp.
Replace all XMedkit fluent builder usages in config_handlers.cpp with typed DTO structs from dto/config.hpp: - handle_list_configurations: ConfigListXMedkit + ConfigXMedkitItem per item - handle_get_configuration: ConfigurationReadValue + ConfigValueXMedkit - handle_set_configuration: parse_body<ConfigurationWriteRequest> for body parsing; ConfigurationReadValue + ConfigValueXMedkit for response - handle_delete_all_configurations: ConfigurationDeleteMultiStatus for 207 Remove all four hand-written config schema factories from schema_builder (configuration_metadata_schema, configuration_read_value_schema, configuration_write_value_schema, configuration_delete_multi_status_schema). DTO-generated schemas in collect_component_schemas() now own these names. Retype config routes in rest_server.cpp and path_builder.cpp to use $ref to DTO names (ConfigurationList, ConfigurationReadValue, ConfigurationWriteRequest, ConfigurationDeleteMultiStatus). Update test_schema_builder.cpp and test_path_builder.cpp to assert against DTO-generated schemas via component_schemas().
Add XMedkitDataItem, DataItem, XMedkitDataList, DataWriteRequest and Collection<DataItem> (named DataList) to the DTO contract layer. All new types are registered in AllDtos and pass EveryRegisteredDtoRoundTrips.
Migrate handle_list_data and handle_put_data_item off the legacy XMedkit
fluent builder. handle_get_data_item keeps JSON passthrough since its
payload shape is only known at runtime (live ROS 2 message).
- handle_list_data: builds Collection<DataItem> via JsonWriter; per-item
x-medkit typed as XMedkitDataItem; collection x-medkit typed as
DataListXMedkit; fan-out uses the JSON overload of merge_peer_items
- handle_put_data_item: uses parse_body<DataWriteRequest> for the
request body; write-response x-medkit typed as XMedkitDataItem
- Remove data_item_schema() and data_write_request_schema() factory
methods; DataItem, DataList, DataWriteRequest now come from DTO registry
- path_builder: data collection route uses ref("DataList") instead of
the inline items_wrapper(data_item_schema()) call
- rest_server: data collection route updated from DataItemList to DataList
- Add direction field to XMedkitRos2 for per-topic data direction
- Update test_schema_builder to assert against DTO-generated schema
Add Lock, AcquireLockRequest, ExtendLockRequest typed DTOs in dto/locks.hpp. Register all three (plus Collection<Lock> as LockList) in AllDtos so the EveryRegisteredDtoRoundTrips contract test covers them. Wire shapes match the existing lock_schema / acquire_lock_request_schema / extend_lock_request_schema factories exactly.
Migrate lock_handlers.cpp to typed DTOs: - handle_acquire_lock uses parse_body<dto::AcquireLockRequest> for JSON body parsing; custom validation (positive expiration, valid scope strings) follows parse_body on the typed struct fields - handle_extend_lock uses parse_body<dto::ExtendLockRequest> - handle_list_locks returns dto::Collection<dto::Lock> via send_dto - handle_get_lock and handle_acquire_lock return dto::Lock via send_dto Remove lock_schema / acquire_lock_request_schema / extend_lock_request_schema factory definitions and declarations; Lock, LockList, AcquireLockRequest, ExtendLockRequest in component_schemas() now come from the DTO registry. Update test_schema_builder to verify lock schemas via component_schemas() instead of the removed factory calls. Update AcquireLockWithMissingExpiration test: parse_body uses ERR_INVALID_REQUEST for missing required fields. make format_expiration public to allow file-scope DTO helper access.
Add dto/triggers.hpp with TriggerCondition, Trigger, TriggerCreateRequest, TriggerUpdateRequest structs and co-located dto_fields/dto_name. The trigger_condition wire field is a free-form merged JSON object (condition_type + additionalProperties), so Trigger and TriggerCreateRequest carry it as nlohmann::json with dto_sample specializations for round-trip safety. Register all new types and Collection<Trigger> (TriggerList) in registry.hpp AllDtos so EveryRegisteredDtoRoundTrips passes.
Replace manual JSON body parsing and response construction in
trigger_handlers.cpp with typed DTO operations:
- handle_create uses parse_body<dto::TriggerCreateRequest>; static
trigger_info_to_dto helper builds the response DTO; send_dto replaces
send_json
- handle_list builds dto::Collection<dto::Trigger> and calls send_dto
- handle_get and handle_update likewise use trigger_info_to_dto + send_dto
- handle_update uses parse_body<dto::TriggerUpdateRequest>
- handle_events (SSE stream) is left untouched
Delete the 4 legacy trigger factory functions (trigger_schema,
trigger_condition_schema, trigger_create_request_schema,
trigger_update_request_schema) from schema_builder.{cpp,hpp}; their
schemas now come from the DTO registry.
Rewrite the three trigger factory assertions in test_schema_builder.cpp
against SchemaWriter<dto::Trigger*> so they verify the same wire contract
through the DTO path.
Add dto/cyclic_subscriptions.hpp with three typed structs: - CyclicSubscription: CRUD response (id, observed_resource, event_source, protocol, interval enum) - CyclicSubscriptionCreateRequest: POST body (resource, interval enum, duration, protocol optional) - CyclicSubscriptionUpdateRequest: PUT body (interval optional enum, duration optional int) Collection<CyclicSubscription> named "CyclicSubscriptionList". All four types added to AllDtos in registry.hpp. test_dto_contract EveryRegisteredDtoRoundTrips passes.
Replace manual JSON body parsing and response construction in
cyclic_subscription_handlers.cpp with typed DTO operations:
- handle_create uses parse_body<dto::CyclicSubscriptionCreateRequest>;
static subscription_to_dto helper builds the response DTO; send_dto
replaces send_json
- handle_list builds dto::Collection<dto::CyclicSubscription> and calls
send_dto
- handle_get and handle_update likewise use subscription_to_dto + send_dto
- handle_update uses parse_body<dto::CyclicSubscriptionUpdateRequest>
- handle_events (SSE stream) is left untouched
- subscription_to_json preserved as a thin wrapper for test compatibility
Delete the 2 legacy cyclic subscription factory functions
(cyclic_subscription_schema, cyclic_subscription_create_request_schema)
from schema_builder.{cpp,hpp}; their schemas now come from the DTO
registry.
Repoint path_builder.cpp to use SchemaBuilder::ref for cyclic
subscription list, create request, and create response schemas.
Fix rest_server.cpp PUT route to reference CyclicSubscriptionUpdateRequest
instead of CyclicSubscription for the request body.
Rewrite the factory assertion in test_schema_builder.cpp against
SchemaWriter<dto::CyclicSubscriptionCreateRequest> and update
test_path_builder.cpp to check $ref instead of inline properties.
Add LogContext, LogEntry, Collection<LogEntry>, LogListXMedkit, and LogConfiguration DTO structs with co-located dto_fields/dto_name specializations. LogListXMedkit covers all x-medkit fields emitted by handle_get_logs across FUNCTION/AREA/COMPONENT/APP entity branches. Register all five types in AllDtos via registry.hpp. EveryRegisteredDtoRoundTrips: green.
Replace the legacy XMedkit fluent builder in log_handlers.cpp with typed dto::LogListXMedkit. Migrate handle_get_logs_configuration to send_dto<LogConfiguration> and handle_put_logs_configuration to parse_body<LogConfiguration>, keeping semantic range validation for max_entries and severity after parsing. Remove log_entry_schema, log_entry_list_schema, and log_configuration_schema factories from SchemaBuilder; LogEntry, LogEntryList, LogContext, LogListXMedkit, and LogConfiguration are now emitted by the DTO registry. Update path_builder to reference LogEntryList via $ref. Update test_schema_builder and test_path_builder to assert against the registered DTO schemas. grep -rn "log_entry_schema|log_configuration_schema" src/ shows only comment references, zero code invocations.
Adds dto/updates.hpp with UpdateList (items-wrapper of bare strings), UpdateSubProgress (nested sub-step struct), XMedkitUpdate (typed x-medkit extension with field_enum on phase), and UpdateStatus (status response with required x-medkit). Registers all four types in AllDtos.
Migrates handle_list_updates to dto::UpdateList and handle_get_status to dto::UpdateStatus via a new to_update_status_dto() converter. Removes the static status_to_json() method from UpdateHandlers. Adds update_status_to_string() helper alongside the existing update_phase_to_string() in update_types.hpp. Deletes update_list_schema and update_status_schema from SchemaBuilder. Schemas are now generated from the DTO types in AllDtos.
Add AuthCredentials and AuthTokenResponse typed DTOs for the auth domain. AuthCredentials captures the OAuth2 request body fields (grant_type, client_id, client_secret, refresh_token, scope). AuthTokenResponse captures the token success response fields (access_token, token_type, expires_in, scope, refresh_token). Both are registered in AllDtos; EveryRegisteredDtoRoundTrips passes.
Switch handle_auth_authorize and handle_auth_token to use parse_body<AuthCredentials> for request parsing and send_dto with AuthTokenResponse for success responses. The 401 credential-check path (invalid_client, invalid_grant) is preserved via the existing AuthManager result handling. The OAuth2 error format (error + error_description) is preserved for all handler-level validations (unsupported_grant_type, invalid_request field checks). Delete auth_token_response_schema and auth_credentials_schema factory functions from SchemaBuilder; both schemas now come from dto/auth.hpp via collect_component_schemas. Zero remaining references to the deleted factories. Routes in rest_server.cpp already use SchemaBuilder::ref() for both auth schemas, so no route changes needed.
Add dto/health.hpp with typed DTOs for the health/root domain: Health, HealthDiscovery, HealthDiscoveryLinking, HealthAggregationWarning (for GET /health), VersionInfo, VersionInfoEntry, VersionInfoVendor, XMedkitVersionInfo (for GET /version-info), RootOverview, RootCapabilities, RootAuth, RootTls (for GET /). Register all in AllDtos. EveryRegisteredDtoRoundTrips passes for all new types.
Migrate handle_health, handle_root, and handle_version_info to build typed DTOs (Health, RootOverview, VersionInfo) and call send_dto instead of constructing raw nlohmann::json. The x-medkit-data-provider and x-medkit-subscription-executor endpoint-level vendor extension keys are modelled as optional<nlohmann::json> fields with hyphened wire keys. The handle_root dynamic endpoint-list computation and the handle_version_info fan-out logic are preserved verbatim. Remove the three legacy hand-written schema factories (health_schema, version_info_schema, root_overview_schema) from schema_builder; the DTO registry now generates HealthStatus, VersionInfo, and RootOverview along with all sub-DTOs. Update test_schema_builder to assert against the DTO-generated schema shapes (component_schemas() and $ref linkage) instead of the removed factory methods. health_handlers.cpp no longer includes core/http/x_medkit.hpp. Zero code references to the three deleted factory names remain. All 2186 gateway unit tests pass.
Delete core/http/x_medkit.hpp, src/core/http/x_medkit.cpp and their test (test_x_medkit.cpp). All 13 handler domains now use typed dto::XMedkit* structs + dto::JsonWriter instead of the fluent builder. The one straggler was fan_out_helpers.hpp which kept a merge_peer_items(XMedkit&) overload alongside the already-migrated nlohmann::json& overload. The legacy overload is removed; the json& overload is the only one remaining. test_fan_out_helpers.cpp is updated to use plain json objects in place of XMedkit for the ext parameter. component_schemas() is simplified: all domain schemas now come from dto::collect_component_schemas(); the only surviving hand-written entry is OperationExecutionList (a thin items-wrapper without a dedicated DTO type). The earlier GenericError and per-domain factory calls that were superseded by the DTO registry are gone. send_json in handler_context.hpp is documented as an escape-hatch for dynamic-payload, fan-out, and spec-blob callers; prefer send_dto<T> for typed responses. CMakeLists.txt: remove ament_add_gtest(test_x_medkit ...) block and the test_x_medkit entry from the coverage target list. Build: clean. Unit suite: 90 tests, 0 failures (2632 subtests).
Add two new assertions to TestOpenApiCallability: 1. test_x_medkit_sub_schemas_present_in_spec: verifies that components/schemas in the live spec contains the DTO-generated schemas XMedkitArea (with ros2 $ref to XMedkitRos2) and AreaListItem (with x-medkit $ref to XMedkitArea). This directly confirms the issue #338 DTO-contract wiring to the spec builder. 2. test_live_entity_responses_conform_to_spec: fetches live responses from /areas, /components, /apps, /health, /version-info, /apps/{id}, /apps/{id}/faults, /apps/{id}/operations, and /apps/{id}/data, then validates each against the jsonschema declared in the 200 response of the same path in the runtime spec. Schema drift causes test failure. Also fix two pre-existing issues surfaced by the callability test: - CyclicSubscriptionCreateRequest.interval used plain field() instead of field_enum() - the schema lacked the enum constraint, causing the payload generator to send "test_value" which the handler rejects. Fixed by using field_enum with kCyclicSubscriptionIntervalValues. - Added ConfigurationWriteRequest and TriggerCreateRequest runtime validations to _KNOWN_BUSINESS_400_PATTERNS: both are genuine spec limitations (oneOf-across-optionals and free-form JSON sub-field constraints) that cannot be expressed in OpenAPI without breaking client ergonomics.
Add a design doc explaining the typed DTO contract introduced in the ros2_medkit_gateway: the Field descriptor, dto_fields/dto_name constexpr tuples, the three visitors (JsonWriter, SchemaWriter, JsonReader), the AllDtos registry, and the workflow for adding new typed endpoints. Wire the new design doc into the per-package design index toctree. Update the OpenAPI tutorial to explain that schemas are generated from the DTO registry rather than hand-written factories. Update the README Docs row to mention the DTO-backed schema accuracy.
…ates ExecutionUpdateRequest.capability was registered with field_enum(..., kExecutionCapabilityValues), which caused parse_body to reject any out-of-vocabulary value with a generic 400 ERR_INVALID_REQUEST before the handler ran. This killed the handler's own capability-validation path that supports custom x-* capabilities and returns the richer ERR_INVALID_PARAMETER 'Unknown capability' response with a supported_capabilities array. Change the registration to plain field() so missing capability is still caught by parse_body (ERR_INVALID_REQUEST), but an out-of-vocab value flows to the handler's if/else chain as intended. Update the test comment to clarify that parse_body only catches absence, not bad values.
lock_to_json and subscription_to_json had zero production callers after the DTO migration - handlers now build responses directly via JsonWriter. Delete both helpers (declaration + definition) and their test suites. REQ_INTEROP_089 was tagged on two of the subscription_to_json tests; that requirement remains covered by test_subscription_manager.cpp and test_transport_registry.cpp.
…gregated-fault x-medkit Add a static_assert to field_enum requiring M to be std::string or std::optional<std::string>. JsonReader::check_enum only fires for those types; applying field_enum to any other member type would silently accept out-of-vocabulary values at runtime. Also document FaultListAggXMedkit in the Known Limitations section of dto_contract.rst - it is the fifth collection x-medkit struct registered in AllDtos but not $ref-linked from its list response schema.
…apability field change The capability field is now registered as plain field() rather than field_enum(), so the generated schema no longer carries an enum array. Update the schema builder test to assert the field is an unrestricted string, matching the deliberate design to allow custom x-vendor-* capabilities through to the handler.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
The gateway's HTTP payloads were maintained as three independent, hand-written representations - the handler's JSON construction, the OpenAPI schema factories, and the
XMedkitextension builder - with nothing keeping them in sync. They drifted: entity list responses advertised a minimal schema while the gateway actually emitted richx-medkitmetadata, so generated clients could not see those fields.This introduces a typed DTO contract. Each payload is a plain C++17 struct described once by a
constexprfield list. Three visitors fold over that single description:JsonWriter- struct to wire JSONSchemaWriter- type to OpenAPI schemaJsonReader- request body to struct, with validationWire output and OpenAPI schema are therefore generated from one source and cannot drift. All HTTP handler domains (entities, faults, operations, configurations, data, locks, triggers, cyclic-subscriptions, bulk-data, logs, scripts, updates, auth, health) were migrated to the contract, and
components/schemasis now generated from the DTO registry. The legacyXMedkitfluent builder and the ~50 hand-written schema factories were removed.Client-observable changes
components/schemasis regenerated from the DTOs; several schema names changed (for example the entity list/detail schemas). Clients generated from the previous spec need to be regenerated.typediscriminator (previously absent on list items).invalid-request(HTTP status 400 is unchanged).Known limitation
The fault/config/data/log list endpoints emit a richer collection-level
x-medkitthan the genericCollection<T>wrapper schema types it as. The wire payload stays a valid instance of the published schema; this is a schema-precision gap, documented indesign/dto_contract.rst.Issue
Part of #338.
This PR makes the gateway spec accurate. #338 stays open until generated clients are regenerated against the corrected spec and downstream consumers updated, which is follow-up work in separate repositories.
Type
Testing
test_dto_contractunit suite round-trips every registered DTO through all three visitors and validates each generated schema.x-medkitsub-schemas are asserted present in the spec.ros2_medkit_gatewayandros2_medkit_integration_testssuites pass (unit, integration, linters); clang-tidy is clean.Reviewers can verify by building
ros2_medkit_gateway, runningcolcon test, and hittingGET /api/v1/docsto confirmcomponents/schemascontains the typed entity andx-medkitschemas.Checklist