Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
docs/capabilities/memory/assets/** filter=lfs diff=lfs merge=lfs -text
docs/capabilities/memory/assets/.gitattributes -filter -diff -merge text
docs/capabilities/mapping/assets/** filter=lfs diff=lfs merge=lfs -text
# Mintlify deploy does not fetch Git LFS; keep site branding as normal blobs.
# Mintlify deploy does not fetch Git LFS; keep published docs assets as normal blobs.

@leshy leshy Jul 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the team concluded that we should not include blobs in github repo as it eventually significantly slows down all github checkouts. we have CI checks against this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May need to store direct blobs somewhere becuase yes mintlify cant pull LFS. maybe seperate repo, or we just bite the bullet and put them in the main repo, i think theyre small anyway. Just want the docs site to look good and not reuqire a ton of maintance overhead

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leshy @spomichter

If we were to include all the assets directly, it would quickly balloon the repo.
For example, in the current checkout we have 1510 text files, totaling 11MiB. But Just assets/5-all.png alone is 1.1 MiB, and some assets are larger.

I think one option would be to link directly to the GitHub hosted link, For example, docs/capabilities/navigation/assets/5-all.png would be https://media.githubusercontent.com/media/dimensionalOS/dimos/main/docs/capabilities/navigation/assets/5-all.png .

Another would be for us to host the assets somewhere ourselves.

If we do include the assets directly into Git, we'll have to carry those files forever!...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main branch can have assets in LFS, i can create a new docs branch with git blobs and deploy mintlify docs on that new branch.

will remove the git blobs from this PR.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@swstica I don't think that's a long term solution. That branch will have to be merged into main eventually, no? So it's just postponing things.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paul-nechifor agreed. will either host them somewhere as you suggested or move some assets to git blob.

docs/assets/dimensional-logo-master-transparent.png -filter -diff -merge
docs/assets/favicon.png -filter -diff -merge
docs/capabilities/navigation/assets/** -filter -diff -merge

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 WebP Uses Text Auto-Detection

reloc_and_nav_to.webp is now a published docs asset, but .webp is not covered by any binary or non-LFS attribute here. If Git's text=auto misclassifies this large WebP during checkout or archive creation, line-ending conversion can corrupt the image and the relocalization page renders a broken asset.

Suggested change
docs/capabilities/navigation/assets/** -filter -diff -merge
docs/capabilities/navigation/assets/** -filter -diff -merge
*.webp -text binary

Binary file modified docs/capabilities/navigation/assets/1-lidar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/2-globalmap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/3-globalcostmap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/4-navcostmap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/5-all.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/coverage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/frontier.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 21 additions & 20 deletions docs/capabilities/navigation/assets/go2nav_dataflow.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/noros_nav.gif

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you getting this as well:

Image

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/patrol_path.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/random.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/capabilities/navigation/assets/reloc_and_nav_to.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 28 additions & 21 deletions docs/capabilities/navigation/deep_dive.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
---
title: "Go2 Non-ROS Navigation"
title: "Go2 Navigation Deep Dive"
---
![output](assets/noros_nav.gif)

The Go2 navigation stack runs entirely without ROS. It uses a **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations.

![Live Go2 navigation in Rerun](assets/noros_nav.gif)

For return visits to a known space, use [premap relocalization](/docs/capabilities/navigation/relocalization.md) instead of relying on live mapping alone.

## Data Flow

<details>
<summary>diagram source</summary>

```pikchr fold output=assets/go2nav_dataflow.svg
color = white
color = #1e293b

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't mintlify also dark mode? this makes it look much worse, idk if there is a better solution, mintlify can't render transparent svg?

Image Image

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does render transparent svg, but mintlify theme is default to system/browser. so it was breaking for light theme. lmk if this works.
Screenshot 2026-07-08 at 8 19 44 PM

fill = none

Go2: box "Go2" rad 5px fit wid 170% ht 170%
Expand All @@ -38,29 +41,29 @@ text "Twist" italic at (M4.x, Nav.s.y - 0.45in)

</details>

![output](assets/go2nav_dataflow.svg)
![Go2 navigation data flow](assets/go2nav_dataflow.svg)

## Pipeline Steps

### 1. LiDAR Frame [`GO2Connection`](/dimos/robot/unitree/go2/connection.py)
### 1. LiDAR Frame ([`GO2Connection`](/dimos/robot/unitree/go2/connection.py))

We don't connect to the LiDAR directly — instead we use Unitree's WebRTC client (via [legion's webrtc driver](https://github.com/legion1581/unitree_webrtc_connect)), which streams a heavily preprocessed 5cm voxel grid rather than raw point cloud data. This allows us to support stock, unjailbroken Go2 Air and Pro models out of the box.
We do not connect to the LiDAR directly. Instead we use Unitree's WebRTC client via [legion's webrtc driver](https://github.com/legion1581/unitree_webrtc_connect), which streams a heavily preprocessed 5cm voxel grid rather than raw point cloud data. This lets us support stock, unjailbroken Go2 Air and Pro models out of the box.

![LiDAR frame](assets/1-lidar.png)

### 2. Global Voxel Map [`VoxelGridMapper`](/dimos/mapping/voxels.py)
### 2. Global Voxel Map ([`VoxelGridMapper`](/dimos/mapping/voxels.py))

The [`VoxelGridMapper`](/dimos/mapping/voxels.py) maintains a sparse 3D occupancy grid using Open3D's `VoxelBlockGrid` backed by a hash map. Each voxel is a 5cm cube by default.

Voxel hash map provides O(1) insert/erase/lookup, so this is efficient even with millions of voxels. The grid runs on **CUDA** by default for speed, with CPU fallback.

Each incoming LiDAR frame is spliced into the global map via column carving. We consider any previously mapped voxels in the space of a received LiDAR frame stale, by erasing entire Z-columns in the footprint, we guarantee:
Each incoming LiDAR frame is spliced into the global map via column carving: any previously mapped voxels in the space of a received LiDAR frame are considered stale, and entire Z-columns in its footprint are erased before the new frame is written. This guarantees:

- No ghost obstacles from previous passes
- Dynamic objects (people, doors) get cleared automatically
- The latest observation always wins

We don't have proper loop closure and stable odometry, we trust the data go2 odom reports, which is surprisingly stable but does drift eventually, You will reliably map and nav through very large spaces (500sqm in our tests) but you won't go down the street to a super market.
Live column-carving has no loop closure. We trust Go2 odometry, which is stable but drifts over distance. You can reliably map and navigate large spaces around 500 m² in our tests, but not kilometer-scale outdoor routes. For return visits with loop-closed maps, use [premap relocalization](/docs/capabilities/navigation/relocalization.md) and build the premap offline with `dimos map global --export`.


#### Configuration
Expand All @@ -75,11 +78,11 @@ We don't have proper loop closure and stable odometry, we trust the data go2 odo

![Global map](assets/2-globalmap.png)

### 3. Global Costmap [`CostMapper`](/dimos/mapping/costmapper.py)
### 3. Global Costmap ([`CostMapper`](/dimos/mapping/costmapper.py))

The [`CostMapper`](/dimos/mapping/costmapper.py) converts the 3D voxel map into a 2D occupancy grid. The default algorithm (`height_cost`) maps rate of change of Z, with some smoothing.

algo settings are in [`occupancy.py`](/dimos/mapping/pointclouds/occupancy.py) and can be configured per robot
Algorithm settings live in [`occupancy.py`](/dimos/mapping/pointclouds/occupancy.py) and can be configured per robot.


#### Configuration
Expand All @@ -103,33 +106,37 @@ class HeightCostConfig(OccupancyConfig):

![Global costmap](assets/3-globalcostmap.png)

### 4. Navigation Costmap [`ReplanningAStarPlanner`](/dimos/navigation/replanning_a_star/module.py)
### 4. Navigation Costmap ([`ReplanningAStarPlanner`](/dimos/navigation/replanning_a_star/module.py))

The planner will process the terrain gradient and compute it's own algo-relevant costmap, prioritizing safe free paths, while be willing to path aggressively through tight spaces if it has to
The planner processes the terrain gradient and computes its own planning costmap, preferring safe free paths but willing to path aggressively through tight spaces when it has to.

We run the planner in a constant loop so it will dynamically react to obstacles encountered.
We run the planner in a constant loop so it dynamically reacts to obstacles as they appear.

![Navigation costmap with path](assets/4-navcostmap.png)

### 5. All Layers Combined

All visualization layers shown together
All visualization layers shown together:

![All layers](assets/5-all.png)

## Frontier Exploration

The [`WavefrontFrontierExplorer`](/dimos/navigation/frontier_exploration/wavefront_frontier_goal_selector.py) drives autonomous exploration of unknown space. It scans the costmap for frontiers, the boundaries between mapped and unmapped cells, picks the best candidate with a wavefront BFS from the robot's position, and publishes it as a navigation goal. When a goal is reached (or fails), it selects the next frontier until the space is fully mapped. Like patrolling below, it is exposed as an agent skill: an LLM agent can call `explore` and `end_exploration`.

## Patrolling

The patrolling system drives the robot to systematically cover a **known** area. It is exposed as an agent skill. An LLM agent can call `start_patrol` and `stop_patrol` to control it. Note that the area has to be explored first.

### How it works

1. **Visitation tracking** As the robot moves, a visitation grid (aligned to the costmap) marks cells around the robot's position as visited. This gives the system a running picture of where the robot has and hasn't been. This expires over time, and has to be visited again.
1. **Visitation tracking:** As the robot moves, a visitation grid aligned to the costmap marks cells around the robot's position as visited. This gives the system a running picture of where the robot has and has not been. Visits expire over time and cells must be covered again.

2. **Goal selection** — A *patrol router* picks the next goal. The default strategy is **coverage**: it samples a handful of candidate points from unvisited, obstacle-free cells, plans a path to each one, and picks the candidate whose path would cover the most new ground. Candidates are weighted by a Voronoi skeleton so goals are more likely to be spread evenly across the map, rather than clustering in large open areas.
2. **Goal selection:** A patrol router picks the next goal. The default strategy is coverage: it samples candidate points from unvisited, obstacle-free cells, plans a path to each one, and picks the candidate whose path would cover the most new ground. Candidates are weighted by a Voronoi skeleton so goals spread evenly across the map rather than clustering in large open areas.

3. **Navigation loop** The module sends each goal to the planner and waits for a `goal_reached` signal before requesting the next one. If no valid goal is available (e.g. the map hasn't loaded yet), it retries after a short delay.
3. **Navigation loop:** The module sends each goal to the planner and waits for a `goal_reached` signal before requesting the next one. If no valid goal is available, for example when the map has not loaded yet, it retries after a short delay.

4. **Stopping** — When patrol is stopped, the module cancels in-progress navigation by publishing the robot's current pose as the goal, then re-enables the planner's normal replanning behavior.
4. **Stopping:** When patrol is stopped, the module cancels in-progress navigation by publishing the robot's current pose as the goal, then re-enables the planner's normal replanning behavior.

### Patrol router strategies

Expand All @@ -141,7 +148,7 @@ The patrolling system drives the robot to systematically cover a **known** area.

### Safety

Goal candidates are filtered through a **safe mask** — the free-space region eroded by the robot's clearance radiusso the robot is never sent to a position too close to walls or obstacles. The planner's safe-goal clearance is also tightened while patrolling to ensure the robot can rotate in place at every goal.
Goal candidates are filtered through a safe mask, which is the free-space region eroded by the robot's clearance radius, so the robot is never sent too close to walls or obstacles. The planner's safe-goal clearance is also tightened while patrolling so the robot can rotate in place at every goal.

### Router comparison

Expand Down Expand Up @@ -178,4 +185,4 @@ unitree_go2 = autoconnect(

to_svg(unitree_go2, "assets/go2_blueprint.svg")
```
![output](assets/go2_blueprint.svg)
![unitree_go2 blueprint module graph](assets/go2_blueprint.svg)
18 changes: 12 additions & 6 deletions docs/capabilities/navigation/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
---
title: "Navigation"
title: "Go2 Navigation Overview"
description: "Live column-carving navigation and premap relocalization for the Unitree Go2."
---

![output](assets/noros_nav.gif)
The Go2 navigation stack uses a simple **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations. Map live as you drive, or return to a known space using a saved premap and relocalization.

The Go2 navigation stack uses simple **column-carving voxel map** strategy: each new LiDAR frame replaces the corresponding region of the global map entirely, ensuring the map always reflects the latest observations.
![Live Go2 navigation in Rerun](assets/noros_nav.gif)

[Navigation Deep Dive](/docs/capabilities/navigation/deep_dive.md)
## Choose your workflow

We also have a simple relocalization system on previously stored and reconstructed maps.
| Workflow | When to use | Blueprint | Docs |
|----------|-------------|-----------|------|
| **Live mapping** | Explore a new space where the map updates every frame | `unitree-go2` | [Navigation deep dive](/docs/capabilities/navigation/deep_dive.md) |
| **Premap + relocalization** | Return to a known space and plan on a loop-closed map | `unitree-go2-relocalization` | [Relocalization](/docs/capabilities/navigation/relocalization.md) |

[Relocalization](/docs/capabilities/navigation/relocalization.md)
Live column-carving maps are fast and reactive, but odometry drifts over long distances. For spaces you revisit, record once, run pose-graph optimization (PGO) offline, then relocalize against the exported premap at runtime.

For hardware setup, simulation, and the full blueprint list, see the [Go2 platform guide](/docs/platforms/quadruped/go2/index.md).
Loading
Loading