| Episodes | 51 |
| Mentions | 70 |
| Cited here | 32 |
| First — last | #4 — #711 |
| Top guests | Dave Taylor, Dave Young, Ben Eater |
| Related | soldering · pcb · transistor · capacitor |
Troubleshooting is the practice of narrowing and isolating the elements of a system in turn until a fault is confined to one of them.[494] No general recipe transfers unchanged between circuits, which is why requests for a universal troubleshooting guide cannot be satisfied.[169] What does transfer is a discipline: start with the elementary confirmations, change one variable at a time, record what has been tested, and treat even an experienced engineer’s immediate diagnosis as a hypothesis to be confirmed by a test before it is acted on.[169][428][494][574] Understanding the layers of a system above and below one’s own is not critical for design work but is for troubleshooting, because faults surface exactly where those abstractions break down.[616]
Opening a diagnosis
The first measurement in any troubleshooting sequence is the supply voltages and power rails.[4] The opening questions are the elementary ones: whether power is applied, whether the unit is connected, whether the reference is present, and whether it is at the correct value.[169] Checking that the expected signal is present at all belongs in the same first pass.[396] Confirming the part number marked on a board itself, rather than trusting the bag or box it arrived in, forecloses a class of fault in which the entire debugging effort is applied to the wrong hardware.[428]
Scope is established early. A fault inherent to a widely used part or design would be seen by every user, so a symptom that only one user reports points at something specific to that user’s setup.[288] Establishing what fraction of a batch shows the fault — one unit, ten, or all of them — separates a design fault from a unit-specific one, and being able to swap the suspect item and watch the symptom follow it confirms which variable is responsible.[288] Substituting a suspect module with an identical known-good one and observing that the symptom persists demonstrates that the fault lies outside the substituted module.[494] A suspected design-level defect in a mass-produced instrument is worth searching for online before further probing, because the same failure is frequently already documented on forums or in teardown videos.[643]
Method and its failure modes
Applying the scientific method to debugging means stating a hypothesis before probing and writing it down; without a written record the activity is thrashing rather than experiment.[460] On an effort running for weeks, tests get repeated because the engineer cannot recall what has already been checked, so a written log of what was tested and what it showed is what keeps the search from circling.[494] Changing many things at once prevents any single result from being interpreted, because no observation can be tied to one change.[428]
Several biases work against the search. An engineer troubleshooting their own design tends to rule out simple causes on the strength of familiarity and to look for complex explanations first.[4] A plausible first idea can grow into a chain of reasoning that is followed to its end before it is ever tested, while the actual cause is the simple one ruled out at the outset.[94] Assuming by default that one’s own hardware is the cause produces a concrete list of things to test first, at the cost of biasing the search towards it.[494] Handing a stalled fault to someone who does not know the design often exposes an obvious cause quickly, because that person carries none of the designer’s assumptions about what cannot be wrong.[4] Deliberately leaving a fault alone for a period is a recognised tenet for the same reason, since continuing to stare at it reinforces the hypothesis already held.[4]
Substituting components one after another until a circuit works locates a bad part without identifying the mechanism, so it gives no protection against the same fault recurring.[4]
Hard cases
An intermittent fault that behaves correctly most of the time and hiccups occasionally defeats a static check of voltages and rails, which is why it resists the standard opening procedure.[53] Repeating an identical test many times is justified when the object is to reproduce such a fault reliably rather than to obtain a different result.[68] A circuit that partly works — for example running at one third of the expected current — is harder to diagnose than one that fails outright, because the expected discrete pass-or-fail signal is absent.[94]
Faults inside a monolithic analog part such as an op amp or a DC-DC controller cannot be probed directly, so deciding whether the part or the surrounding circuit is at fault depends on understanding the discrete building blocks the part implements.[4] Oscillation or erratic analog behaviour can come from operating a part near a marginal specification rather than from a defective component, for example signals coupling through a supply because of poor power-supply rejection.[4] In vintage equipment repair the fault found quickly is usually dried-out electrolytic capacitors; the hard cases are parametric, such as a single transistor whose beta is slightly off sitting in the most sensitive part of the circuit.[227]
Symptoms frequently point away from their cause. A single corrupted memory bit rarely produces an obvious memory symptom; it more often produces behaviour that directs the investigator at some unrelated part of the circuit.[68] Windowed EPROMs covered with paint rather than an opaque label can still be erased by ultraviolet light passing through the paint, producing bit failures that appear spontaneous.[68] Leakage currents across the surface of a PCB are treated by cleaning the board and removing flux residue.[100] A board recreated from artwork rather than from a verified netlist carries connections that are missing or wrong, and these surface as wiring faults to be chased during bring-up.[609]
Techniques and tools
Digital signals are analog waveforms attempting to reach a logic level, ringing and passing through intermediate voltages on the way; treating them that way rather than as clean ones and zeros is what allows digital faults to be diagnosed.[222]
Thermal stimulus is a common localisation technique with an important limitation. Freeze spray localises a thermally sensitive fault but does not identify it; confirming the cause requires further work such as reflowing the suspect device or testing traces, and interpreting what the thermal response means.[148] Heating or cooling a single component with a hot-air pencil or freeze spray can move a circuit in and out of tolerance even when the responsible part is elsewhere on the board, so a response to local thermal stimulus does not by itself convict the stimulated component.[326]
Documentation determines what is possible. Without a schematic and board layout, diagnosis degrades to measuring nodes whose function is unknown or removing components at random, unless the board is first reverse-engineered.[561] Board-repair software that imports a schematic and overlays it on an image of the board lets a technician click a node and highlight the corresponding trace across the layout, which is what makes component-level repair of dense assemblies efficient.[561] Hewlett-Packard signature analyzers attempted to mechanise the same work: dedicated test software was run against a known-good board and a logic probe returned a short four-character signature at each node, which a technician could compare against the faulty unit.[169]
Development work requires instruments in the engineer’s own lab, because troubleshooting has to happen at the moment the fault appears, which is a different equipment requirement from manufacturing.[312] Standardising a group on one recommended instrument matters more than which instrument is best, because shared troubleshooting depends on everyone being able to start from the same first step.[190] Storing a board under investigation in a tray with its debug programmer and connections left attached preserves the setup between sessions and avoids reconnecting the wrong things when work resumes.[559]
Designing to reduce diagnosis
Wrapping a design around an existing supported standard or module, instead of a from-scratch equivalent, removes that block from the set of things that must be diagnosed when the system does not work.[25] Testing every shipped unit to specification with a purpose-built jig lets that unit be excluded as a variable when a customer later troubleshoots a measurement that looks wrong.[210] Learning a development board’s configuration options, such as its on-board DIP switches, before they are needed pays off when problems have to be diagnosed on that board later.[221]
Buying a module without understanding it moves the difficulty rather than removing it: a 12 V to 5 V switching converter is serviceable until it fails or becomes unstable, at which point a buyer who does not know how a switcher works cannot diagnose it.[353]
Remote and reported faults
In remote troubleshooting, asking for a photograph of the setup before any verbal description catches gross errors — a missing device, a disconnected battery, reversed wires — that the reporter’s own account omits because they are not aware of them.[477] A set of instruments reported as drifting badly was traced to the 10 MHz reference being connected reference-port to reference-port rather than output to input; the fault survived a verbal confirmation that the reference was connected and was found only by inspecting the installation in person.[533]
Economics
A fixed-price contract quote transfers the troubleshooting risk to the contractor, because debugging time is unbounded at the moment the estimate is made, and electronics work contains more of these traps than software work.[70] In flat-rate consumer repair paid at roughly ten dollars per set, a novel fault taking an hour is unprofitable while a previously seen fault fixed in ten minutes is profitable, which puts direct economic value on remembering fault signatures.[222]
Language models as an aid
Language-model assistance has proved useful for structural configuration debugging such as Zephyr device tree files, where the risk is that a fast-moving project’s changes make a suggestion stale — an error the compiler then reports.[683] It does not extend to everything: pasting compiler or runtime errors from privately written code into a general language model does not yield a diagnosis, because the model has no knowledge of the specific code that produced them.[693] Using a model as the primary debugging aid also leaves no capability for the cases it cannot answer, so the underlying skill of synthesising evidence remains the thing that has to be maintained.[683]