Troubleshooting method
A repeatable, evidence-driven approach to finding root causes across software, hardware, and networks. No guesswork, no cargo-cult fixes, no restarting-and-hoping.
brian@dev:~$ ./isolate.sh --logic
Evidence-first
no guesswork, only data
One variable at a time
clean, testable changes
Break it down
binary search the stack
Verify the fix
survive the original failure
Document & harden
never let it recur
Across the stack
from silicon to SQL
The Process
Five steps to a root cause
Reproduce & Observe
Never fix what you can't reproduce. Capture the exact failure state, logs, and conditions so every hypothesis has a baseline to test against.
Isolate Variables
Change one thing at a time. Binary-search the problem space to shrink it from 'the whole system is broken' to 'this exact component is at fault'.
Form a Hypothesis
Root causes, not symptoms. I ask why until the answer can't be answered again, and frame each theory as a testable prediction.
Test & Validate
Prove the theory with evidence, not assumption, then confirm the fix holds under the original failing conditions and beyond.
Verify & Harden
The fix is only done when it survives. I verify under load, document what happened, and harden the system so it doesn't recur.
Field Notes
Real problems, root causes
Linux / Storage
Mystery Disk-Full Server
The Problem
A server kept filling its disk overnight with no obvious culprit.
Approach
Checked df first, then traced the biggest offenders with du while the culprit rotated. Monitored inotify and cron to catch the writer red-handed.
Root Cause
A misconfigured log-rotation job was growing an unbounded log each night.
Resolution
Fixed rotation, added a disk-usage alert, and confirmed two weeks of stable capacity.
Networking
Dropped Wi-Fi, Working Router
The Problem
Intermittent drops on one device while everything else stayed connected.
Approach
Isolated the physical layer first, then checked channels, power output, and the specific adapter's driver and power management.
Root Cause
The laptop's Wi-Fi power-save mode was dropping the link during idle windows.
Resolution
Disabled power-saving on the adapter; connectivity stabilised permanently.
Hardware / Firmware
Silent Reboot Loop
The Problem
A PC rebooted moments after POST with no error on screen.
Approach
Isolated hardware component by component, tested RAM and PSU, then checked thermal behaviour under load.
Root Cause
A failing RAM stick with marginal errors that only surfaced after warm-up.
Resolution
Identified the faulty module, replaced it, and ran a full memory test to confirm.
Hardware
Thermal Throttle Slump
The Problem
A build degraded to a crawl under load despite adequate specs.
Approach
Monitored core temps and clock speeds in real time, then inspected mounting and airflow.
Root Cause
Dried-out thermal paste and a clogged cooler causing aggressive throttling.
Resolution
Re-applied paste, cleaned the loop, and reclaimed full sustained performance.
Software
Application Crash, No Stack Trace
The Problem
A release crashed intermittently in production with an empty-looking trace.
Approach
Reproduced locally with instrumented builds, enabled verbose logging, and reproduced on a staging copy with identical inputs.
Root Cause
A race condition between two async writes to shared state.
Resolution
Serialized the writes, added a regression test, and shipped a clean fix.
Security / Network
Encrypted Traffic Riddle
The Problem
A trusted app kept flagging data corruption across the wire.
Approach
Captured and decrypted the session, then compared every byte end-to-end across the full path.
Root Cause
A middlebox was transparently re-encrypting and corrupting handshakes.
Resolution
Bypassed the culprit, verified integrity end-to-end, and hardened the TLS config.
My Diagnostic Kit
Continue
Put the method to work
The same discipline shows up across engineering and the homelab.