Support the UE 5.8 City Sample and PCG based city generation in the UE5 branch

[M-Colley]

Title: Support the UE 5.8 City Sample and PCG based city generation in the UE5 branch

Is your feature request related to a problem? Please describe.

The UE5 branch of CARLA ships a small set of hand authored towns. Environment variety is therefore a hard limit for perception training sets and for driving simulator studies, where most work is collected in Town 10. Epic released an updated City Sample on 27 August 2026 that removes the main obstacle to generating new urban environments inside the editor. The new levels are built entirely in engine with the Procedural Content Generation framework, and the project ships the CitySamplePCG plugin, PCG Primitive subgraphs for roads, buildings, forests and parks, shape grammar definition assets for buildings and roads, PCG Assembly data assets, Nanite Foliage and Megaplants. The listing is at https://www.fab.com/listings/4898e707-7855-404b-af0e-a505ee690e68 and the announcement is at https://www.unrealengine.com/en-US/blog (City Sample update, 27 August 2026).

This content requires Unreal Engine 5.8. CARLA's UE5 branch builds against a fork of Unreal Engine 5.5, so the sample cannot be opened in the CARLA editor and none of the PCG primitives are available. The engine upgrade is already tracked in #9516. The existing procedural path in CARLA is the Digital Twins tool, which is limited to OpenStreetMap geometry and depends on Houdini for building generation. A PCG based path would remove the external dependency and give parametric control over road layout, block structure, building profiles and vegetation from within the editor.

Describe the solution you'd like

The request is a supported workflow that turns a PCG generated level into a fully functional CARLA map. The following items are ordered by dependency.

1. Engine upgrade to Unreal Engine 5.8. PCG Primitives, Nanite Foliage, Megaplants and the Unreal MCP server are 5.8 features. This blocks everything below and extends Upgrade Carla's UnrealEngine 5 fork to 5.6+ #9516.

2. OpenDRIVE binding. CARLA derives lane topology, waypoints, junctions and traffic manager behaviour from the .xodr file. PCG road splines carry no OpenDRIVE semantics, so a PCG city loaded as geometry alone gives no routing, no waypoint API and no traffic manager. Two directions are possible. In the first, an .xodr is the authoritative input and a PCG node reads it through CARLA's existing OpenDRIVE parser and emits road centrelines, lane boundaries, junctions and sidewalks as PCG splines and point sets that the city graphs consume. In the second, road splines authored in PCG are exported to OpenDRIVE 1.4. The first direction is cheaper to implement, keeps the client API contract unchanged, and guarantees that geometry and logical road network agree. I would prioritise it.

3. Semantic and instance segmentation tags for PCG spawned geometry. PCG spawns through instanced static mesh components. CARLA assigns semantic tags per actor and per mesh. A per instance semantic tag attribute needs to propagate from the PCG graph into the instanced components so that the semantic and instance segmentation sensors label PCG buildings, vegetation and street furniture correctly. Without this, PCG geometry is invisible to the segmentation ground truth and the maps are unusable for perception work.

4. CARLA actors emitted from PCG. Traffic lights, traffic signs and street lights should be spawned as CARLA actors bound to the corresponding OpenDRIVE junction and road ID, so that world.get_traffic_lights() and signal state control work on generated maps.

5. Pedestrian navigation. Sidewalk output from PCG should feed the Recast navigation mesh build so that walker spawning works without a manual .bin export step.

6. Reproducibility metadata. The PCG seed, graph asset version and plugin version should be written into the map metadata and exposed through the Python API. A generated map can then be shared as a graph, a seed and an .xodr. These files are small and version controllable, which makes an environment used in a published experiment reproducible by third parties. Shipping a packaged level is not a workable substitute here because of file size and licence terms.

7. World Partition and Large Map interaction. The City Sample levels rely on World Partition streaming. Its interaction with CARLA's Large Map tile system and with sensor determinism in synchronous mode needs to be specified and tested.

Licence constraint

City Sample content is distributed under Epic's licence terms, and the Fab listing states that usage with AI is not permitted. CARLA specific assets are CC BY and CARLA specific code is MIT. Vendoring City Sample assets into the CARLA repository is therefore out of the question. The deliverable should be CARLA's own PCG graphs plus an import and tagging path, with the City Sample treated as an optional user supplied dependency and used as a reference implementation. Users then obtain the sample from Fab themselves under Epic's terms.

Describe alternatives you've considered

• Digital Twins with OpenStreetMap input. This produces valid OpenDRIVE but depends on Houdini for buildings, offers little control over block and facade variation, and is not available in the UE5 branch in the same form.
• RoadRunner. It produces OpenDRIVE and geometry, and it carries a commercial licence that many academic groups cannot fund.
• Manual .fbx and .xodr import per map. This requires authoring each environment by hand and provides no parametric variation.
• Backporting City Sample content to Unreal Engine 5.5. The PCG Primitives plugin and Nanite Foliage are 5.8 features, so the backport fails at the plugin level rather than at the asset level.

Acceptance criteria for a first milestone

• A PCG generated level loads in the UE5 branch of CARLA and the server starts.
• world.get_map().to_opendrive() returns a valid OpenDRIVE description of the generated road network.
• The traffic manager spawns and routes vehicles on the generated network without manual waypoint fixes.
• Semantic and instance segmentation sensors label PCG spawned buildings and vegetation.
• Walkers spawn and navigate the generated sidewalks.
• Regenerating the map from the recorded seed and graph version produces an identical level.

Additional context

• Fab listing: https://www.fab.com/listings/4898e707-7855-404b-af0e-a505ee690e68
• Related issues: Upgrade Carla's UnrealEngine 5 fork to 5.6+ #9516 (upgrade of the UE5 fork to 5.6 or newer), Automating Procedural map generation with full control over assets #4332 (procedural map generation with asset control), Epic: Procedural road generation #579 (procedural road generation).
• I run driving simulator studies on CARLA 0.10.0 and can test candidate builds on Ubuntu 22.04 and Windows 11, including traffic manager behaviour, segmentation ground truth and walker navigation on generated maps, and report results back here.

[M-Colley]

I have this working, and the tooling is public:
https://github.com/M-Colley/citysample-carla (MIT).

It runs Epic's City Sample as a CARLA 0.10.0 server against the CARLA fork of
UE 5.8, and separately exports the city's road network to OpenDRIVE so it can
be driven in any CARLA. No Epic content is redistributed — the tools regenerate
everything from your own City Sample install in a few minutes.

Road network. City Sample has no OpenDRIVE, and its road data is not in a
form CARLA reads: the drivable layout lives in Epic's ZoneGraph. A small UE
plugin exports it (-run=ExportZoneGraph), and Python converts lanes → roads +
junctions → .xodr. On Small_City_LVL that is 3,337 roads, 172 junctions,
145 km, 2,861 spawn points, with paramPoly3 endpoint continuity checked to
2.45 × 10⁻⁵ m across 5,452 joins. The traffic manager routes on it.

Two things mattered more than expected. One road per lane, not per lane group
— grouping them made adjacent roads' generated meshes overlap and trapped
vehicles, costing about half the spawn points. And lane-level <link> elements,
not just road-level: road links alone made things worse (34/40 vehicles
despawned); adding <lane><link><successor> took survival to 41/58.

Hosting the level itself needed rather more, because several CARLA features
assume a map CARLA authored:

• Ray-cast sensors returned nothing. CARLA hardcodes ECC_GameTraceChannel2,
  which in City Sample is a different channel with DefaultResponse=Ignore, so
  every ray passed through the world. Resolving the channel by name gives
  232,584 lidar points where there were 0, and 1,980 radar detections where
  there were 0.
• Semantic segmentation labelled almost nothing. City Sample streams its
  geometry through Epic's FastGeoStreaming as FFastGeoPrimitiveComponent —
  plain C++ objects, not UPrimitiveComponent UObjects, not owned by actors —
  so ATagger cannot reach them by construction. A census inside ATagger
  found seven static mesh components in the entire city. Walking the
  FastGeo containers directly took the camera from 98.8 % Unlabeled to
  0.5 %.
• Semantic LiDAR was still unlabeled for a related but distinct reason: a
  ray-cast sensor reads the hit's UPrimitiveComponent tag, and FastGeo owns
  its FBodyInstances through IPhysicsBodyInstanceOwner instead.
  FFastGeoPhysicsBodyInstanceOwner::FromHitResult resolves it. 99.1 % → 3.3 %.
• No traffic lights. The signal layout is not in the ZoneGraph; it is a
  separate data asset. Parsing it yields 353 lights across 106 junctions,
  phased so opposing approaches share a green, plus 1,096 crosswalks. The
  other 66 junctions are stop-sign intersections — MassTraffic's own
  bHasTrafficLights distinguishes them — so those get 229 stop signs.
• world.get_actors() could not see the traffic. City Sample's vehicles and
  pedestrians are Mass ECS entities, not actors, so the camera renders hundreds
  of cars the API cannot see. A bridge mirrors the nearest ones in as dormant
  proxies.
• Pedestrians could not spawn or move, silently, for three independent
  reasons — see the issues below.

Bugs found on the way. Four are already filed (#9863, #9864, #9865, #9866).
The rest are written up in the repo, and two are worth a maintainer's attention
because together they make walker navigation fail silently on any pre-existing
map:

• HasServerSideNavigation() returns true for a RecastNavMesh with zero
  tiles, and the client cannot fall back — so a perfectly valid
  Saved/Nav/<Map>.bin becomes unreachable and every walker API returns
  nothing, with no log line on either side.
• FNavigationMesh::Load and get_required_files("Nav") search different
  directories, so a navmesh in the place the first documents is ignored by the
  path that actually runs.

Also: Navigation::GetRandomLocation filters on CARLA_TYPE_SIDEWALK alone,
so a navmesh built from an OpenDRIVE file without type="sidewalk" lanes is
valid, has hundreds of tiles, and still cannot place a single pedestrian.

Happy to open a PR for any of the CARLA-side fixes if that is useful — they are
in the repo as a single patch against ue58-dev-carla, and none of them are
City Sample-specific except by motivation.

[germanros1987]

This is nice, but I am concerned about the license. Have you talked to EpicGames? Sometimes they are willing to "relax" their language.

[M-Colley]

No, I have not contacted Epic. I could ask them but I would think that ask lands better coming from the project than from me, what do you thin? Before that, it is worth separating the three layers and what the project actually does:

The tooling. MIT, and it contains no Epic content. The ZoneGraph exporter plugin, the lane-to-OpenDRIVE converter, the signal layout parser and the tagging code all regenerate their output from the user's own City Sample install. Nothing from Epic is vendored, and the repo does not ship a packaged level.

The CARLA-side patches. These are independent of the City Sample. Resolving the ray-cast channel by name instead of hardcoding ECC_GameTraceChannel2, walking the FastGeo containers in ATagger, FFastGeoPhysicsBodyInstanceOwner::FromHitResult, and the three navmesh issues are fixes to CARLA against UE APIs. They apply to any FastGeo or World Partition map, and the navmesh ones apply to any pre-existing map at all. CARLA already builds against a UE fork under the UE EULA, so there is no new exposure here.

The generated artifacts (.xodr, signal layout, navmesh) are derived from Epic data. They are generated locally from the user's own install and I do not publish them. I would keep that rule whatever else is decided.

Still, there might be three questions worth putting to Epic, in this order:

1. Does the restriction cover models trained on rendered output of the content, or only generative models trained on the assets themselves? Perception training on simulator renders is the main CARLA use case, so this one decides whether the path is viable at all.
2. May a user publish an OpenDRIVE description of the road network generated from their own install, as a reproducibility artifact attached to a paper? These are a few MB of geometry with no assets in them.
3. May renders be published as figures in papers?