-
Notifications
You must be signed in to change notification settings - Fork 732
Swastika/docs/navigation relocalization #2764
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 5 commits
2c1b729
a7f0221
804eb71
6c337f0
23fe1bf
814c28f
52d3b0f
13af95e
05d926f
b89be20
35d146d
286becf
dc85349
d0f741e
f0a7907
be32ac5
e606650
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -19,6 +19,7 @@ | |||||||
| docs/capabilities/memory/assets/** filter=lfs diff=lfs merge=lfs -text | ||||||||
| docs/capabilities/memory/assets/.gitattributes -filter -diff -merge text | ||||||||
| docs/capabilities/mapping/assets/** filter=lfs diff=lfs merge=lfs -text | ||||||||
| # Mintlify deploy does not fetch Git LFS; keep site branding as normal blobs. | ||||||||
| # Mintlify deploy does not fetch Git LFS; keep published docs assets as normal blobs. | ||||||||
| docs/assets/dimensional-logo-master-transparent.png -filter -diff -merge | ||||||||
| docs/assets/favicon.png -filter -diff -merge | ||||||||
| docs/capabilities/navigation/assets/** -filter -diff -merge | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. removed |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,20 @@ | ||
| --- | ||
| title: "Go2 Non-ROS Navigation" | ||
| title: "Go2 Navigation Deep Dive" | ||
| --- | ||
|  | ||
|
|
||
| The Go2 navigation stack runs entirely without ROS. It uses a **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations. | ||
|
|
||
|  | ||
|
|
||
| For return visits to a known space, use [premap relocalization](/docs/capabilities/navigation/relocalization.md) instead of relying on live mapping alone. | ||
|
|
||
| ## Data Flow | ||
|
|
||
| <details> | ||
| <summary>diagram source</summary> | ||
|
|
||
| ```pikchr fold output=assets/go2nav_dataflow.svg | ||
| color = white | ||
| color = #1e293b | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| fill = none | ||
|
|
||
| Go2: box "Go2" rad 5px fit wid 170% ht 170% | ||
|
|
@@ -38,29 +41,29 @@ text "Twist" italic at (M4.x, Nav.s.y - 0.45in) | |
|
|
||
| </details> | ||
|
|
||
|  | ||
|  | ||
|
|
||
| ## Pipeline Steps | ||
|
|
||
| ### 1. LiDAR Frame — [`GO2Connection`](/dimos/robot/unitree/go2/connection.py) | ||
| ### 1. LiDAR Frame ([`GO2Connection`](/dimos/robot/unitree/go2/connection.py)) | ||
|
|
||
| We don't connect to the LiDAR directly — instead we use Unitree's WebRTC client (via [legion's webrtc driver](https://github.com/legion1581/unitree_webrtc_connect)), which streams a heavily preprocessed 5cm voxel grid rather than raw point cloud data. This allows us to support stock, unjailbroken Go2 Air and Pro models out of the box. | ||
| We do not connect to the LiDAR directly. Instead we use Unitree's WebRTC client via [legion's webrtc driver](https://github.com/legion1581/unitree_webrtc_connect), which streams a heavily preprocessed 5cm voxel grid rather than raw point cloud data. This lets us support stock, unjailbroken Go2 Air and Pro models out of the box. | ||
|
|
||
|  | ||
|
|
||
| ### 2. Global Voxel Map — [`VoxelGridMapper`](/dimos/mapping/voxels.py) | ||
| ### 2. Global Voxel Map ([`VoxelGridMapper`](/dimos/mapping/voxels.py)) | ||
|
|
||
| The [`VoxelGridMapper`](/dimos/mapping/voxels.py) maintains a sparse 3D occupancy grid using Open3D's `VoxelBlockGrid` backed by a hash map. Each voxel is a 5cm cube by default. | ||
|
|
||
| Voxel hash map provides O(1) insert/erase/lookup, so this is efficient even with millions of voxels. The grid runs on **CUDA** by default for speed, with CPU fallback. | ||
|
|
||
| Each incoming LiDAR frame is spliced into the global map via column carving. We consider any previously mapped voxels in the space of a received LiDAR frame stale, by erasing entire Z-columns in the footprint, we guarantee: | ||
| Each incoming LiDAR frame is spliced into the global map via column carving: any previously mapped voxels in the space of a received LiDAR frame are considered stale, and entire Z-columns in its footprint are erased before the new frame is written. This guarantees: | ||
|
|
||
| - No ghost obstacles from previous passes | ||
| - Dynamic objects (people, doors) get cleared automatically | ||
| - The latest observation always wins | ||
|
|
||
| We don't have proper loop closure and stable odometry, we trust the data go2 odom reports, which is surprisingly stable but does drift eventually, You will reliably map and nav through very large spaces (500sqm in our tests) but you won't go down the street to a super market. | ||
| Live column-carving has no loop closure. We trust Go2 odometry, which is stable but drifts over distance. You can reliably map and navigate large spaces around 500 m² in our tests, but not kilometer-scale outdoor routes. For return visits with loop-closed maps, use [premap relocalization](/docs/capabilities/navigation/relocalization.md) and build the premap offline with `dimos map global --export`. | ||
|
|
||
|
|
||
| #### Configuration | ||
|
|
@@ -75,11 +78,11 @@ We don't have proper loop closure and stable odometry, we trust the data go2 odo | |
|
|
||
|  | ||
|
|
||
| ### 3. Global Costmap — [`CostMapper`](/dimos/mapping/costmapper.py) | ||
| ### 3. Global Costmap ([`CostMapper`](/dimos/mapping/costmapper.py)) | ||
|
|
||
| The [`CostMapper`](/dimos/mapping/costmapper.py) converts the 3D voxel map into a 2D occupancy grid. The default algorithm (`height_cost`) maps rate of change of Z, with some smoothing. | ||
|
|
||
| algo settings are in [`occupancy.py`](/dimos/mapping/pointclouds/occupancy.py) and can be configured per robot | ||
| Algorithm settings live in [`occupancy.py`](/dimos/mapping/pointclouds/occupancy.py) and can be configured per robot. | ||
|
|
||
|
|
||
| #### Configuration | ||
|
|
@@ -103,33 +106,37 @@ class HeightCostConfig(OccupancyConfig): | |
|
|
||
|  | ||
|
|
||
| ### 4. Navigation Costmap — [`ReplanningAStarPlanner`](/dimos/navigation/replanning_a_star/module.py) | ||
| ### 4. Navigation Costmap ([`ReplanningAStarPlanner`](/dimos/navigation/replanning_a_star/module.py)) | ||
|
|
||
| The planner will process the terrain gradient and compute it's own algo-relevant costmap, prioritizing safe free paths, while be willing to path aggressively through tight spaces if it has to | ||
| The planner processes the terrain gradient and computes its own planning costmap, preferring safe free paths but willing to path aggressively through tight spaces when it has to. | ||
|
|
||
| We run the planner in a constant loop so it will dynamically react to obstacles encountered. | ||
| We run the planner in a constant loop so it dynamically reacts to obstacles as they appear. | ||
|
|
||
|  | ||
|
|
||
| ### 5. All Layers Combined | ||
|
|
||
| All visualization layers shown together | ||
| All visualization layers shown together: | ||
|
|
||
|  | ||
|
|
||
| ## Frontier Exploration | ||
|
|
||
| The [`WavefrontFrontierExplorer`](/dimos/navigation/frontier_exploration/wavefront_frontier_goal_selector.py) drives autonomous exploration of unknown space. It scans the costmap for frontiers, the boundaries between mapped and unmapped cells, picks the best candidate with a wavefront BFS from the robot's position, and publishes it as a navigation goal. When a goal is reached (or fails), it selects the next frontier until the space is fully mapped. Like patrolling below, it is exposed as an agent skill: an LLM agent can call `explore` and `end_exploration`. | ||
|
|
||
| ## Patrolling | ||
|
|
||
| The patrolling system drives the robot to systematically cover a **known** area. It is exposed as an agent skill. An LLM agent can call `start_patrol` and `stop_patrol` to control it. Note that the area has to be explored first. | ||
|
|
||
| ### How it works | ||
|
|
||
| 1. **Visitation tracking** — As the robot moves, a visitation grid (aligned to the costmap) marks cells around the robot's position as visited. This gives the system a running picture of where the robot has and hasn't been. This expires over time, and has to be visited again. | ||
| 1. **Visitation tracking:** As the robot moves, a visitation grid aligned to the costmap marks cells around the robot's position as visited. This gives the system a running picture of where the robot has and has not been. Visits expire over time and cells must be covered again. | ||
|
|
||
| 2. **Goal selection** — A *patrol router* picks the next goal. The default strategy is **coverage**: it samples a handful of candidate points from unvisited, obstacle-free cells, plans a path to each one, and picks the candidate whose path would cover the most new ground. Candidates are weighted by a Voronoi skeleton so goals are more likely to be spread evenly across the map, rather than clustering in large open areas. | ||
| 2. **Goal selection:** A patrol router picks the next goal. The default strategy is coverage: it samples candidate points from unvisited, obstacle-free cells, plans a path to each one, and picks the candidate whose path would cover the most new ground. Candidates are weighted by a Voronoi skeleton so goals spread evenly across the map rather than clustering in large open areas. | ||
|
|
||
| 3. **Navigation loop** — The module sends each goal to the planner and waits for a `goal_reached` signal before requesting the next one. If no valid goal is available (e.g. the map hasn't loaded yet), it retries after a short delay. | ||
| 3. **Navigation loop:** The module sends each goal to the planner and waits for a `goal_reached` signal before requesting the next one. If no valid goal is available, for example when the map has not loaded yet, it retries after a short delay. | ||
|
|
||
| 4. **Stopping** — When patrol is stopped, the module cancels in-progress navigation by publishing the robot's current pose as the goal, then re-enables the planner's normal replanning behavior. | ||
| 4. **Stopping:** When patrol is stopped, the module cancels in-progress navigation by publishing the robot's current pose as the goal, then re-enables the planner's normal replanning behavior. | ||
|
|
||
| ### Patrol router strategies | ||
|
|
||
|
|
@@ -141,7 +148,7 @@ The patrolling system drives the robot to systematically cover a **known** area. | |
|
|
||
| ### Safety | ||
|
|
||
| Goal candidates are filtered through a **safe mask** — the free-space region eroded by the robot's clearance radius — so the robot is never sent to a position too close to walls or obstacles. The planner's safe-goal clearance is also tightened while patrolling to ensure the robot can rotate in place at every goal. | ||
| Goal candidates are filtered through a safe mask, which is the free-space region eroded by the robot's clearance radius, so the robot is never sent too close to walls or obstacles. The planner's safe-goal clearance is also tightened while patrolling so the robot can rotate in place at every goal. | ||
|
|
||
| ### Router comparison | ||
|
|
||
|
|
@@ -178,4 +185,4 @@ unitree_go2 = autoconnect( | |
|
|
||
| to_svg(unitree_go2, "assets/go2_blueprint.svg") | ||
| ``` | ||
|  | ||
|  | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,19 @@ | ||
| --- | ||
| title: "Navigation" | ||
| title: "Go2 Navigation Overview" | ||
| description: "Live column-carving navigation and premap relocalization for the Unitree Go2." | ||
| --- | ||
|
|
||
|  | ||
| The Go2 navigation stack uses a simple **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations. Map live as you drive, or return to a known space using a saved premap and relocalization. | ||
|
|
||
| The Go2 navigation stack uses simple **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations. | ||
|  | ||
|
|
||
| [Navigation Deep Dive](/docs/capabilities/navigation/deep_dive.md) | ||
| ## Choose your workflow | ||
|
|
||
| We also have a simple relocalization system on previously stored and reconstructed maps. | ||
| | Workflow | When to use | Blueprint | Docs | | ||
| |----------|-------------|-----------|------| | ||
| | **Live mapping** | Explore a new space where the map updates every frame | `unitree-go2` | [Navigation deep dive](/docs/capabilities/navigation/deep_dive.md) | | ||
| | **Premap + relocalization** | Return to a known space and plan on a loop-closed map | `unitree-go2-relocalization` | [Relocalization](/docs/capabilities/navigation/relocalization.md) | | ||
|
|
||
| [Relocalization](/docs/capabilities/navigation/relocalization.md) | ||
| Live column-carving maps are fast and reactive, but odometry drifts over long distances. For spaces you revisit, record once, run pose-graph optimization (PGO) offline, then relocalize against the exported premap at runtime. | ||
|
|
||
| For hardware setup, simulation, and the full blueprint list, see the [Go2 platform guide](/docs/platforms/quadruped/go2/index.md). |




Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the team concluded that we should not include blobs in github repo as it eventually significantly slows down all github checkouts. we have CI checks against this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May need to store direct blobs somewhere becuase yes mintlify cant pull LFS. maybe seperate repo, or we just bite the bullet and put them in the main repo, i think theyre small anyway. Just want the docs site to look good and not reuqire a ton of maintance overhead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@leshy @spomichter
If we were to include all the assets directly, it would quickly balloon the repo.
For example, in the current checkout we have 1510 text files, totaling 11MiB. But Just
assets/5-all.pngalone is 1.1 MiB, and some assets are larger.I think one option would be to link directly to the GitHub hosted link, For example,
docs/capabilities/navigation/assets/5-all.pngwould be https://media.githubusercontent.com/media/dimensionalOS/dimos/main/docs/capabilities/navigation/assets/5-all.png .Another would be for us to host the assets somewhere ourselves.
If we do include the assets directly into Git, we'll have to carry those files forever!...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
main branch can have assets in LFS, i can create a new docs branch with git blobs and deploy mintlify docs on that new branch.
will remove the git blobs from this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@swstica I don't think that's a long term solution. That branch will have to be merged into main eventually, no? So it's just postponing things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@paul-nechifor agreed. will either host them somewhere as you suggested or move some assets to git blob.