Skip to content

symbolizer: add logging for symbolizer errors#5354

Open
g-ortuno wants to merge 1 commit into
google:masterfrom
g-ortuno:symbolizer-logs
Open

symbolizer: add logging for symbolizer errors#5354
g-ortuno wants to merge 1 commit into
google:masterfrom
g-ortuno:symbolizer-logs

Conversation

@g-ortuno

@g-ortuno g-ortuno commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Spawns a separate thread to observer stderr and include its output when logging symbolization errors. We use a separate thread to avoid potential deadlock situations if llvm-symbolizer writes more than 64KB.

This will help diagnose symbolizer issues like https://crbug.com/532093354 more easily in the future.

@g-ortuno g-ortuno requested a review from a team as a code owner July 7, 2026 21:34
@g-ortuno g-ortuno changed the title add logging for symbolizer errors symbolizer: add logging for symbolizer errors Jul 8, 2026
@g-ortuno

g-ortuno commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

@dylanjew mind taking a look? Progression tasks haven't run since 8am for some reason[1]. Unfortunately no progression or regression tasks were spawned while the changes were in dev[2]. A few fuzzing tasks were spawned but I couldn't find one that found a crash. There are a lot of regression tasks that successfully symbolized stack traces during that window, but I'm not sure they were running the new code.

The symbolization errors are only happening on macOS, so we won't see any of the new logs on dev.

@g-ortuno

g-ortuno commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

@javanlacerda PTAL! Would be good to include this in the next release so we can understand the macOS failures: https://crbug.com/532093354. Since this is in the symbolizer code, it shouldn't affect OSS, right?

@dylanjew

dylanjew commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

@dylanjew mind taking a look? Progression tasks haven't run since 8am for some reason[1]. Unfortunately no progression or regression tasks were spawned while the changes were in dev[2]. A few fuzzing tasks were spawned but I couldn't find one that found a crash. There are a lot of regression tasks that successfully symbolized stack traces during that window, but I'm not sure they were running the new code.

The symbolization errors are only happening on macOS, so we won't see any of the new logs on dev.

The schedule progression cron runs at 7 AM every day. Maybe there's something with the dev/prod sync mechanism or with releases that clears the progression tasks queue.

@g-ortuno

g-ortuno commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

With @dylanjew 's help, I confirmed that symbolization is still working http://shortn/_v1dlN1tmsO on dev so this should be safe to land.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants