[6/9] Select the ROS 2 DDS middleware at runtime via --rmw - #9813
Draft
JArmandoAnaya wants to merge 14 commits into
Draft
[6/9] Select the ROS 2 DDS middleware at runtime via --rmw#9813JArmandoAnaya wants to merge 14 commits into
JArmandoAnaya wants to merge 14 commits into
Conversation
The carla-ros2-native ExternalProject configures a separate CMake process, so it never inherited CMake/Common.cmake's CMAKE_CXX_STANDARD and compiled at the Clang default (C++17) while the rest of the project is C++20. Set the standard explicitly in the inner project and pass it through the ExternalProject CMAKE_ARGS so every consumer of the ROS 2 headers (carla-server, libcarla-ros2-native.so, the LibCarla test suites) builds at the same language level. Vendor ExternalProjects (fastdds, foonathan_memory) keep their own standard; only our code is raised. Validated: all TUs of the shared lib compile clean at -std=gnu++20 with 0 errors and 0 warnings under UE's libc++, and a full package-development build runs to BUILD SUCCESSFUL. FastDDS 2.11.2 and Fast-CDR 1.1 headers are C++20-clean.
Introduce the middleware-neutral type layer of the ROS 2 middleware decoupling series, ported from ue4-dev: - types/msg/*.h: 31 plain C++ structs, one per ROS 2 message type, no DDS dependency, standard library headers only, all members value-initialized (upstream ue4-dev carla-simulator#9612). - types/CdrSerialization.h: serialize_to_cdr(), deserialize_from_cdr() and cdr_serialized_size() for all msg::* types using Fast-CDR (classic CDR, little-endian, DDS encapsulation header). The buffers are wire-compatible with every ROS 2 distribution and can be handed directly to FastDDS write() paths or CycloneDDS dds_writecdr(), removing the need for per-vendor generated type files. A kMaxCdrSequenceElements cap rejects hostile sequence lengths during deserialization (upstream ue4-dev carla-simulator#9643). - types/CdrTopicInfo.h: per-type type_name(), REP-2011 RIHS01 type hash and max_serialized_size(); the hashes let ROS 2 Iron and newer RMWs parse the type hash CARLA advertises via USER_DATA (upstream ue4-dev carla-simulator#9681). - types/UserDataFormat.h: build_user_data() / build_user_data_for<T>() helpers producing the REP-2016 "typehash=RIHS01_<hex>;" key-value payload (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so CdrSerialization.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) instead of the Fast-CDR 2.x forms the ue4-dev tip carries since its Fast-DDS 2.14.6 upgrade (ue4-dev carla-simulator#9789). Five lines differ; the wire format is identical either way and is pinned by the golden-bytes test added in the follow-up test commit. The FastDDSConversions.h / FastDDSTypeMap.h files from carla-simulator#9612 are deliberately not ported; they were superseded by unified CDR upstream. The new headers are not referenced by any build target yet; they start compiling when the middleware abstraction lands in the next PR of the series. (adapted from ue4-dev 542959a) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4) (adapted from ue4-dev b865088)
Add LibCarla/source/test/server/test_ros2_serialization.cpp with the cdr_topic_info (2 cases) and cdr_serialization (22 cases) GTest groups from the ue4-dev middleware suite: POD round-trips for every message family, a golden-bytes guard that pins the classic CDR little-endian encapsulation so any drift to XCDRv2 or big-endian is caught deterministically, truncated / corrupt-encapsulation / hostile-sequence-length rejection paths exercising the kMaxCdrSequenceElements cap, and the contract that cdr_serialized_size(msg) equals serialize_to_cdr(msg).size(). The tests exercise only Fast-CDR plus the new foundation headers; no DDS participant is created and libfastrtps is not linked. Build wiring, gated on ENABLE_ROS2: - libcarla_test_server gains the Ros2Native install include dir as a SYSTEM include and links libfastcdr.so from the fastdds ExternalProject install tree; add_dependencies orders the test compile and link after the fastdds install step. - The fastdds ExternalProject declares libfastcdr.so via INSTALL_BYPRODUCTS so Ninja accepts the full-path link input on a clean tree. - Without ENABLE_ROS2 the test source is removed from the server test glob and the suite builds exactly as before. (adapted from ue4-dev f53144c) (adapted from ue4-dev b865088)
Introduce the vendor-neutral middleware strategy layer of the ROS 2 middleware decoupling series and its FastDDS implementation, compiled into libcarla-ros2-native.so. Ported from ue4-dev: - middleware/Middleware.h: the Middleware enum plus the string and ROS 2 type-name helpers; the CycloneDDS value and its availability branches are present but stay compiled out until the CycloneDDS middleware lands (upstream ue4-dev carla-simulator#9608). - middleware/IPublisherMiddleware.h, ISubscriberMiddleware.h: the type-erased publisher/subscriber strategy interfaces. Subscribers write received samples straight into caller-owned storage to avoid a copy (upstream ue4-dev carla-simulator#9608). - middleware/MiddlewareFactory.h: creates the active middleware for a traits type; each vendor arm is double-gated on its CARLA_ROS2_MIDDLEWARE_* macro and CARLA_ROS2_MIDDLEWARE_TESTING so the suite exercises the availability logic without linking DDS (upstream ue4-dev carla-simulator#9608). - middleware/ActiveMiddleware.{h,cpp}: a DDS-free bridge (SetActiveMiddleware) so ROS2.cpp, the only ROS 2 translation unit in carla-server, can select the middleware without any DDS header crossing the shared-library boundary. Nothing calls it until the cutover; the definition ships now so the shared lib has a translation unit that compiles MiddlewareFactory.h and the FastDDS headers with the real vendor macros. - middleware/fastdds/GenericCdrPubSubType.h: one FastDDS TopicDataType that serializes every carla::ros2::msg::* struct through the unified CdrSerialization.h path, replacing the generated per-type PubSubType classes; getSerializedSizeProvider reports the actual instance size so variable-length payloads (camera frames, point clouds) are not bounded by the static max size (upstream ue4-dev carla-simulator#9643). - middleware/fastdds/FastDDS{Publisher,Subscriber}Middleware.h: the FastDDS strategy implementations. Each endpoint advertises the REP-2016 "typehash=RIHS01_<hex>;" USER_DATA so Jazzy RMWs match on the REP-2011 type hash (upstream ue4-dev carla-simulator#9681). - middleware/fastdds/FastDDSSharedParticipant.{h,cpp}: a refcounted process-wide DomainParticipant shared across all FastDDS endpoints, avoiding the discovery storm that destroying N participants back to back caused on shutdown (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so GenericCdrPubSubType.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) matching CdrSerialization.h from the previous PR of the series. The wire format is classic CDR little-endian and is unchanged. The existing publishers keep using the generated FastDDS types; the cutover to this abstraction and the PublisherImpl/SubscriberImpl rewrite land in the next PR of the series. No behavior change. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev 02a83ef) (adapted from ue4-dev c64e8f4)
Extend the ROS 2 serialization suite with the middleware abstraction cases and rename it to test_ros2_middleware.cpp to match the layer it now covers. 32 new cases, all pure logic with no DDS participant: - middleware_to_string / middleware_from_string / middleware_available / middleware_type_name: the Middleware enum string and type-name helpers, and compile-time availability reporting. - MiddlewareFactoryFixture: set/get, strict resolution, and that creating an unavailable vendor middleware returns nullptr. - generic_cdr_pubsubtype: GenericCdrPubSubType round-trips a fixed-size and a string-bearing message through a real FastDDS SerializedPayload_t, name matches CdrTopicInfo, createData/deleteData and getKey behave. - generic_cdr_pubsubtype_large_payload: getSerializedSizeProvider reports the actual instance size so a full camera frame and a large point cloud serialize past the static max size. The file defines CARLA_ROS2_MIDDLEWARE_FASTDDS and CARLA_ROS2_MIDDLEWARE_TESTING so the factory availability branches are active while the real DDS entities stay compiled out; only GenericCdrPubSubType pulls actual Fast-DDS/Fast-CDR headers. When ENABLE_ROS2 is ON, libcarla_test_server now also links libfastrtps.so (added as an INSTALL_BYPRODUCTS on the fastdds ExternalProject) for the TopicDataType base and SerializedPayload_t, with -rpath-link / -rpath / --disable-new-dtags so its private foonathan_memory dependency resolves at link and run time. When OFF, the renamed source is removed from the glob and the suite builds exactly as before. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4)
…bstraction Cut the ROS 2 publishers and subscribers over to the middleware strategy layer added earlier in the series. PublisherImpl and SubscriberImpl stop talking to FastDDS directly and delegate to an IPublisherMiddleware / ISubscriberMiddleware created by MiddlewareFactory, so the transport is chosen at runtime with no vendor header reaching these templates. Ported from ue4-dev: - publishers/PublisherImpl.h, subscribers/SubscriberImpl.h: rewritten from raw FastDDS plumbing (participant/writer/reader/listener) to hold a unique_ptr<I*Middleware> from MiddlewareFactory plus one owned message. The LIBCARLA_WITH_GTEST seams (SetMiddlewareForTesting, and the subscriber's SimulateMessageReceiptForTesting) let the suite inject fakes without a live DDS participant. The traits contract is reduced to a single msg_type typedef. - publishers/Carla*Publisher.cpp, subscribers/*Subscriber.cpp: every publisher and subscriber traits now binds a carla::ros2::msg::* POD instead of the generated sensor_msgs::msg::X + XPubSubType pair, and the message bodies use direct member access in place of the generated field accessors. The DDS type name, hash and size come from the CdrTopicInfo<msg_type> specialization consumed inside the middleware. - ROS2.cpp: ROS2::Enable selects the middleware through the DDS-free SetActiveMiddleware(Middleware::FastDDS) bridge before creating any publisher or subscriber. FastDDS is the only middleware today; the runtime --rmw= selection lands later in the series. The developer BasicPublisher / BasicSubscriber / BasicListener demo path (WITH_ROS2_DEMO) also moves off the generated std_msgs::msg::String: BasicPublisher through its traits, and the raw-FastDDS BasicSubscriber and BasicListener by registering GenericCdrPubSubType<msg::String>, so the generated type files can be removed cleanly in the next PR. FastDDS keeps publishing exactly as before; the wire format is unchanged. (adapted from ue4-dev f2c0760)
Add the PublisherImpl and SubscriberImpl cases to the ROS 2 middleware suite, 14 new cases covering the cutover. Ported from ue4-dev: - MockPublisherMiddleware / MockSubscriberMiddleware: minimal IPublisherMiddleware / ISubscriberMiddleware fakes that record the calls the impls make. - publisher_impl (7): GetMessage returns the owned buffer, Init and Publish delegate to the injected middleware, Publish before Init fails, IsAlive and GetTopicName delegate, and the owned message pointer flows through Publish unchanged. - subscriber_impl (7): the new-message flag starts false, Init threads the storage and flag pointers to the middleware and propagates its result, GetMessage clears the flag, and SimulateMessageReceipt drives the receive path without a live participant. The impls are injected through the LIBCARLA_WITH_GTEST seams, so no DDS participant is created. The cases reuse the TestMsg traits already defined for the middleware groups. (adapted from ue4-dev f2c0760)
The fastcdrgen-generated message classes under carla/ros2/types/ are dead after the publisher/subscriber cutover: everything now routes through the carla::ros2::msg::* PODs and the hand-written CDR path. Delete the 120 legacy files (60 headers + 60 sources), keeping only the CDR foundation (CdrSerialization.h, CdrTopicInfo.h, UserDataFormat.h) and the msg/ PODs. (adapted from ue4-dev ff5e490)
Add CycloneDDS as a second DDS middleware behind the existing abstraction. It publishes and receives the same classic little-endian CDR bytes that the shared CdrSerialization path already produces for FastDDS, plugging a custom ddsi_sertype/ddsi_serdata (raw CDR passthrough, no IDL-generated types and no cyclonedds-cxx) into the IPublisherMiddleware/ISubscriberMiddleware contract. The publisher Init uses a fixed reliable, keep-last depth 1 QoS matching the FastDDS defaults, and the shared participant uses domain 0 to match FastDDSSharedParticipant. The Middleware enum and MiddlewareFactory already dispatch to these classes, so no change to those files is needed. (adapted from ue4-dev 02a83ef)
Build CycloneDDS 0.10.5 as a shared library through a new ExternalProject, link libddsc.so into libcarla-ros2-native.so, define CARLA_ROS2_MIDDLEWARE_CYCLONEDDS on that target, and stage libddsc.so* into the plugin Binaries as a runtime dependency. (adapted from ue4-dev 02a83ef)
Make the native ROS 2 connector choose its DDS middleware at server launch. Both FastDDS and CycloneDDS are compiled into libcarla-ros2-native.so; the new --rmw=<fastdds|cyclonedds> flag selects between them. FastDDS stays the default, so --ros2 alone is unchanged. Extend the DDS-free ActiveMiddleware bridge so availability resolves inside the shared library, where the CARLA_ROS2_MIDDLEWARE_* macros are defined (they are not visible in carla-server): SetActiveMiddleware returns whether the requested middleware is available, and a new GetAvailableMiddleware exposes the compiled-in list. ROS2::Enable takes the selected middleware and fails when it is unavailable. The plugin parses --rmw= into CarlaSettings, and CarlaEngine disables ROS 2 with a logged error on an unrecognized or uncompiled middleware, listing the available ones, with no silent fallback. Document the flag in Docs/ros2_native.md and add the consolidated decoupling entry to CHANGELOG.md.
Reconcile the rviz example with the runtime middleware selection: add --distro=<humble|jazzy> and --rmw=<fastdds|cyclonedds> to run_rviz.sh, build a per-combination image via a new Dockerfile, set the matching RMW environment variables, and add the CycloneDDS UDP transport config. Document the usage in the example README. Upstream commit message: Update ROS2 Example (carla-simulator#9672) * Added `--distro` and `--rmw` arguments to `run_rviz.sh` to be able to run the example with ROS2 distros `Humble` and `Jazzy` and RMW `FastDDS` and `CycloneDDS` * add udp transport to cyclonedds config file (adapted from ue4-dev 7fd91eb) Co-Authored-By: Luis Poveda Cano <luispovedacano@gmail.com>
Add PythonAPI/test/smoke/test_ros2.py exercising the server-side native ROS 2 publish path: the enable_for_ros/disable_for_ros API, camera and LiDAR publishing, the radar/DVS/semantic-LiDAR paths, an enable and disable cycle, and a multi-sensor stress run with sequential teardown. The suite is middleware-agnostic, so it validates whichever --rmw= the server was launched with. It overrides tearDown to reload Town10HD_Opt because Town03 is not present in the packaged build. Register it in smoke_test_list.txt.
This was referenced Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sixth PR of the ROS 2 middleware decoupling series for ue5-dev (tracking issue #9762): make the DDS middleware selectable at server launch. Both FastDDS and CycloneDDS are compiled into
libcarla-ros2-native.so(PR #9811), but until nowROS2::Enablealways chose FastDDS. This PR adds the--rmw=<fastdds|cyclonedds>command-line flag, wires it through the UE5 plugin, ports the rviz example and docs, adds a native ROS 2 smoke test, and lands the single consolidated CHANGELOG line for the whole decoupling. Ported from the merged ue4-dev series (#9644, #9692, #9672, #9665).An unrecognized
--rmwvalue, or one naming a middleware not compiled into the binary (for examplecycloneddson Windows), logs an error listing the available middlewares and leaves ROS 2 disabled for that session. There is no silent fallback. FastDDS stays the default, so--ros2alone behaves exactly as before.Availability is resolved inside the shared library: the
CARLA_ROS2_MIDDLEWARE_*macros are defined only on thecarla-ros2-nativetarget, so the existing DDS-freeSetActiveMiddlewarebridge was extended to return whether the requested middleware is available and to expose the compiled-in list, rather than checking it incarla-server.Depends on #9811; review after it merges. The diff below includes the earlier PRs' commits until then.
What this PR changes
carla/ros2/middleware/ActiveMiddleware.{h,cpp}:SetActiveMiddleware(Middleware)now returnsbool(resolves availability and selects inside the.so), and a newGetAvailableMiddleware()returns the compiled-in middleware list.ROS2::Enable(bool)becomesbool Enable(bool, Middleware = FastDDS); it logs and returns false when the requested middleware is unavailable.RmwNameUPROPERTY toCarlaSettingsparsed from--rmw=(lowercased), and wiresCarlaEngine::NotifyInitGameto parse the value, select the middleware, and disable ROS 2 with a logged error on an unrecognized or uncompiled middleware.config/cyclonedds.xmlandDockerfile, and arun_rviz.shrewrite with--distroand--rmwswitches that build a per-combination image and set the matching RMW environment variables.--rmw=inDocs/ros2_native.mdand adds the consolidated decoupling entry toCHANGELOG.md.PythonAPI/test/smoke/test_ros2.py(native ROS 2 publish-path smoke tests), registered insmoke_test_list.txt. It overridestearDownto reloadTown10HD_Optbecause Town03 is not in the packaged build, and is middleware-agnostic so it runs against a server launched with either--rmwvalue.PR series
types/msg/*POD structs,CdrSerialization/CdrTopicInfo/UserDataFormat, CDR round-trip GTestsGenericCdrPubSubType, shared participant,ActiveMiddlewarebridgePublisherImpl/SubscriberImplto delegate through the factory, migrate publisher traits to the POD types,ROS2.cppcallsSetActiveMiddlewaretypes/*set--rmw=selection, plugin integration, examples, docsCarlaSettings/CarlaEnginewiring, launch flag, examples, docs, consolidated CHANGELOG--rmw=zenoh)middleware/zenoh/*, factory branch, zenoh-c build inRos2Native, rviz example, tests--ros-domain-id)MiddlewareConfig, resolution--ros-domain-id>ROS_DOMAIN_ID> 0, honored by all middlewares, docs + example + testsRelated: #9762, #9811
Where has this been tested?
libcarla_test_server210/210,libcarla_test_client193/193,package-developmentgreen. On the Development package,--ros2 --rmw=fastddsand--ros2 --rmw=cycloneddsboth publish, verified from ROS 2 Humble containers (/clockGID vendor prefix01.0fand01.10respectively), and--rmw=<invalid>disables ROS 2 with a logged error and no crash.smoke.test_ros2passes 5/5 under both middlewares.Possible Drawbacks
--rmw=cycloneddsdisables ROS 2 with a logged error instead of falling back to FastDDS; this is intentional so a misconfiguration is visible rather than silently ignored.smoke.test_ros2requires a server launched with--ros2; it is not run in CI and is skipped by the standard smoke run against a non-ROS 2 server.This change is