Skip to content

Commit 578e7cf

Browse files
feat(crashdump): add Crashdump pillar browsing erl_crash.dump files (#59)
* feat(crashdump): add Crashdump pillar browsing erl_crash.dump files Adds a web crash dump browser - the last big item from the observer_cli / OTP-observer gap-closing plan. Pick a dump, watch the parse progress live, then read the crash slogan, the VM's counters at crash time, and every dumped process: sortable by memory, reductions or message queue, searchable, with a drill-down showing the full per-process record including the stack dump and message queue. Parsing reuses OTP's own crashdump_viewer gen_server (observer app) instead of reimplementing the dump format - no GUI/wx is ever started. A dedicated ObserverWeb.Crashdump.Server owns the parser: it starts it on demand, serializes loads, and receives the parser's progress reports through a small linked relay process registered under the name the parser reports to (a process can only carry one registered name), re-broadcast over PubSub so the page renders a live progress bar. Records are decoded by zipping documented field orders over the tuples, degrading gracefully across OTP record evolution, and pids round-trip through the parser's own textual form. Safety and availability follow the established patterns: there is no free path input - only files in explicitly allowlisted crashdump_dirs are browsable, and with no configuration the page stays off (dumps hold everything the VM had at crash time; documented with a warning). Hosts without the observer application get a graceful notice; dev and test load it via Mix.ensure_application!/1. Dialyzer cannot see the pruned observer app, so the crashdump_viewer calls are ignore-listed with a comment explaining why they are safe. Tests exercise the full pipeline against a real crash dump generated by halting a peer node - listing, allowlist rejection, progress, general info (slogan), process summaries, details round-trip, and the page flow from LOAD to the stack dump panel. * Updating image and toolbar * feat(crashdump): explicit enable flag and browser upload Addresses the concern about the :observer dependency by making the whole feature - including its navigation tab - opt-in behind a single flag, and by adding file upload so a dump can be inspected without any server-side directory: * config :observer_web, crashdump: true gates the pillar. Off by default the tab never appears; the page (if reached directly) explains how to enable it. When on but :observer isn't in the release, it explains that instead. * Upload an erl_crash.dump straight from the browser (e.g. one pulled off a crashed Nerves device) - it is copied to a temporary path the parser re-reads for the session and the previous upload is cleaned up. The directory listing is kept as a second, optional path and no longer required. The crashdump_dirs allowlist still governs host-side browsing (no free path input); uploads are size-capped and consumed to a controlled temp file. The nav also moves Crashdump ahead of Metrics so Metrics stays last. Enabled in the dev server for convenience. Real-upload parsing is covered end to end at the context level (load_upload/1 against a dump from a crashed peer node); the page test covers the upload UI and the validate/empty-submit handlers, since simulating the upload channel through the nested LiveComponent hits a test-harness limitation. * Removing crashdump option, it is always available, checking the observer availability
1 parent 7f23100 commit 578e7cf

18 files changed

Lines changed: 1310 additions & 9 deletions

File tree

.dialyzer_ignore.exs

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11
[
22
{"lib/observer_web/macros.ex", :unknown_function},
3-
{"lib/observer_web/monitor/process_port.ex", :pattern_match}
3+
{"lib/observer_web/monitor/process_port.ex", :pattern_match},
4+
# :crashdump_viewer (observer app) is invisible to the PLT - Mix prunes undeclared OTP
5+
# applications from the code path. Every call is behind ObserverWeb.Crashdump.available?/0
6+
# and exercised against a real crash dump in the test suite.
7+
{"lib/observer_web/crashdump/server.ex", :unknown_function}
48
]

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ library is part of the [DeployEx][dye] project.
3030
### Busiest network connections ranked by throughput, including NIF sockets
3131
![Network Dashboard](./guides/static/network_dash.png)
3232

33+
### Crash dump browser (upload or host dirs): slogan, VM state and every dumped process
34+
![Crashdump Dashboard](./guides/static/crashdump_dash.png)
35+
3336
### Application topology view with per-app summary, processes, ports, links and relations.
3437
![Applications Dashboard](./guides/static/applications_tree.png)
3538

dev.exs

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# Development server for Observer Web
22

3-
# :mnesia is not part of the application's declared dependencies (host releases decide whether
4-
# they ship it), and Mix prunes undeclared OTP applications from the code path (Elixir 1.15+).
5-
# Load it here so the Mnesia browser can be exercised: :mnesia.start() from a remsh, then browse.
3+
# :mnesia and :observer are not part of the application's declared dependencies (host releases
4+
# decide whether they ship them), and Mix prunes undeclared OTP applications from the code path
5+
# (Elixir 1.15+). Load them here so the Mnesia browser and the Crashdump viewer can be
6+
# exercised. Comment out the :observer line to preview the Crashdump "not available" notice.
67
Mix.ensure_application!(:mnesia)
8+
Mix.ensure_application!(:observer)
79

810
# Phoenix
911

@@ -76,6 +78,9 @@ Application.put_env(:observer_web, WebDev.Endpoint,
7678
]
7779
)
7880

81+
# The crashdump_dirs listing is optional now that dumps can be uploaded straight from the
82+
# browser. Point it at a real directory to exercise the host-dump listing.
83+
Application.put_env(:observer_web, :crashdump_dirs, ["/path/to/dumps/"])
7984
Application.put_env(:phoenix, :serve_endpoints, true)
8085
Application.put_env(:phoenix, :persistent, true)
8186

guides/installation.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,35 @@ config :observer_web,
294294
> this where dashboard access is appropriately restricted (see the authentication resolver
295295
> section).
296296
297+
### Crash dump browser
298+
299+
The Crashdump page opens `erl_crash.dump` files with OTP's own `crashdump_viewer` parser (part
300+
of the `:observer` application).
301+
302+
Parsing needs the `:observer` application on the node, so add it to your release's applications.
303+
Enabling `:observer` only puts its modules on the code path - it has no application callback and starts no processes (and never any GUI/wx); Observer Web only ever calls its dump parser.
304+
When it is missing, the page shows a notice explaining what to add instead of the viewer.
305+
306+
There are two ways to open a dump:
307+
308+
- **Upload** an `erl_crash.dump` straight from the browser (for example, one pulled off a
309+
crashed Nerves device). It is parsed on the dashboard node and the temporary copy is
310+
discarded.
311+
- **Browse** dumps already present on the dashboard host. To avoid free path input, only files
312+
in explicitly allowlisted directories are listed:
313+
314+
```elixir
315+
config :observer_web,
316+
crashdump_dirs: ["/var/log/my_app/crashdumps"]
317+
```
318+
319+
> #### Production data exposure {: .warning}
320+
>
321+
> A crash dump includes process states, message queues and application data at crash time.
322+
> Anyone with access to the Observer Web dashboard can read the dumps they upload or browse -
323+
> make sure dashboard access is appropriately restricted (see the authentication resolver
324+
> section).
325+
297326
### Usage with Web and Clustering
298327

299328
The Observer Web provides observer ability for the local application as well as any other that is

guides/overview.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,13 @@ plus the NIF-based socket module sockets that port listings miss.
5454

5555
![Observer Network Dashboard](./static/network_dash.png)
5656

57+
- **💥 Crash Dump Browser** - Upload an `erl_crash.dump` from your machine, or open one from
58+
an allowlisted directory on the host, then read the crash slogan, the VM's state at crash time
59+
and every dumped process - stacks and message queues included - parsed with OTP's own
60+
crashdump_viewer. Requires the `:observer` application in your release.
61+
62+
![Observer Crashdump Dashboard](./static/crashdump_dash.png)
63+
5764
- **🔬 Process/Port Inspection** - View processes and ports details as well as their status and
5865
connectivity (and much more), with a per-application summary table: process/port counts,
5966
version and on-demand memory/reductions totals.

guides/static/crashdump_dash.png

265 KB
Loading

lib/observer_web/application.ex

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ defmodule ObserverWeb.Application do
1414
Observer.Web.Telemetry,
1515
ObserverWeb.Tracer.Server,
1616
ObserverWeb.Monitor.ProcessPort,
17+
ObserverWeb.Crashdump.Server,
1718
{Phoenix.PubSub, [name: ObserverWeb.PubSub]}
1819
] ++ telemetry_servers()
1920

lib/observer_web/crashdump.ex

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
defmodule ObserverWeb.Crashdump do
2+
@moduledoc """
3+
Browses `erl_crash.dump` files in the web dashboard - the web equivalent of OTP's
4+
`crashdump_viewer` GUI, reusing its battle-tested parser (`:crashdump_viewer`, part of the
5+
`:observer` application) instead of reimplementing the dump format.
6+
7+
## Requirements
8+
9+
Parsing needs OTP's `:crashdump_viewer` (part of the `:observer` application, which ships with
10+
OTP but is commonly excluded from releases). No GUI/wx is ever started; only the parsing
11+
`gen_server` is used. When `:observer` isn't in the release, the page explains what to add.
12+
13+
## Ingesting a dump
14+
15+
Two ways:
16+
17+
* **Upload** an `erl_crash.dump` straight from the browser (e.g. one pulled off a crashed
18+
Nerves device) - it's parsed on the dashboard node and the temporary copy is discarded.
19+
* **Browse** dumps already present on the dashboard host, from directories the host
20+
explicitly allowlists (never a free path):
21+
22+
config :observer_web, crashdump_dirs: ["/var/log/my_app/crashdumps"]
23+
24+
## Security
25+
26+
Crash dumps contain everything the VM held at crash time: process states, message queues,
27+
application data. Directory browsing is limited to the allowlisted directories, uploads are
28+
size-capped, and dumps are parsed on the dashboard node only - never fetched from remote nodes.
29+
"""
30+
31+
alias ObserverWeb.Crashdump.Server
32+
33+
@doc """
34+
Whether the `:crashdump_viewer` parser is available on this node.
35+
"""
36+
@spec available? :: boolean()
37+
def available?, do: Code.ensure_loaded?(:crashdump_viewer)
38+
39+
@doc """
40+
Lists the crash dump files found in the configured `:crashdump_dirs` (newest first).
41+
Returns `{:error, :not_configured}` when no directory is configured - the feature's off state.
42+
"""
43+
@spec list_dumps() :: {:ok, [map()]} | {:error, :not_configured}
44+
def list_dumps do
45+
case Application.get_env(:observer_web, :crashdump_dirs, []) do
46+
[] ->
47+
{:error, :not_configured}
48+
49+
dirs when is_list(dirs) ->
50+
dumps =
51+
dirs
52+
|> Enum.flat_map(&list_dir/1)
53+
|> Enum.sort_by(& &1.mtime, :desc)
54+
55+
{:ok, dumps}
56+
end
57+
end
58+
59+
defp list_dir(dir) do
60+
dir
61+
|> Path.join("*.dump")
62+
|> Path.wildcard()
63+
|> Kernel.++(Path.wildcard(Path.join(dir, "erl_crash.dump")))
64+
|> Enum.uniq()
65+
|> Enum.flat_map(fn path ->
66+
case File.stat(path, time: :posix) do
67+
{:ok, %File.Stat{type: :regular, size: size, mtime: mtime}} ->
68+
[%{path: path, name: Path.basename(path), size: size, mtime: mtime}]
69+
70+
_unreadable ->
71+
[]
72+
end
73+
end)
74+
end
75+
76+
@doc """
77+
Starts loading a dump. Progress and completion are broadcast on the `"crashdump"` PubSub
78+
topic as `{:crashdump_progress, percent | {:error, reason} | :done}` - subscribe via
79+
`subscribe/0`. Only paths returned by `list_dumps/0` are accepted.
80+
"""
81+
@spec load(String.t()) :: :ok | {:error, term()}
82+
def load(path) do
83+
with :ok <- check_available(),
84+
{:ok, dumps} <- list_dumps(),
85+
%{path: ^path} <- Enum.find(dumps, &(&1.path == path)) || {:error, :unknown_dump} do
86+
Server.load(path)
87+
end
88+
end
89+
90+
@doc """
91+
Loads a dump from an uploaded file's already-consumed temporary path. Unlike `load/1` this
92+
skips the directory allowlist: the path isn't user-supplied text but a temp file LiveView
93+
produced from a size-capped upload (see `Observer.Web.Crashdump.Page`).
94+
"""
95+
@spec load_upload(String.t()) :: :ok | {:error, term()}
96+
def load_upload(path) do
97+
with :ok <- check_available() do
98+
Server.load(path)
99+
end
100+
end
101+
102+
defp check_available do
103+
if available?(), do: :ok, else: {:error, :crashdump_viewer_unavailable}
104+
end
105+
106+
@doc """
107+
The server's current state: `:idle`, `{:loading, path, percent}` or `{:loaded, path}`.
108+
"""
109+
@spec status() :: :idle | {:loading, String.t(), non_neg_integer()} | {:loaded, String.t()}
110+
defdelegate status, to: Server
111+
112+
@doc """
113+
Subscribes the caller to load progress broadcasts.
114+
"""
115+
@spec subscribe() :: :ok | {:error, term()}
116+
def subscribe do
117+
Phoenix.PubSub.subscribe(ObserverWeb.PubSub, "crashdump")
118+
end
119+
120+
@doc """
121+
General information about the loaded dump.
122+
"""
123+
@spec general_info() :: {:ok, map()} | {:error, :no_dump_loaded}
124+
defdelegate general_info, to: Server
125+
126+
@doc """
127+
The loaded dump's process summaries (pid, name, state, current function, message queue
128+
length, reductions, memory) - the integer fields are sortable.
129+
"""
130+
@spec processes() :: {:ok, [map()]} | {:error, :no_dump_loaded}
131+
defdelegate processes, to: Server
132+
133+
@doc """
134+
Full details of one process in the loaded dump, by its pid string (e.g. `"<0.42.0>"`),
135+
including the stack dump and message queue as expandable strings.
136+
"""
137+
@spec proc_details(String.t()) :: {:ok, map()} | {:error, :no_dump_loaded | :not_found}
138+
defdelegate proc_details(pid_string), to: Server
139+
end

0 commit comments

Comments
 (0)