Firmware testing for hardware prototypes: HIL test benches, automation, and AI-generated test cases

Firmware testing is the practice of verifying the software that runs directly on a product's microcontroller or processor: that it reads sensors correctly, drives outputs correctly, communicates reliably, and recovers safely from faults such as power loss, bad input, or a failed update. It differs from app or web software testing because the code depends on real hardware, real timing, and real electrical signals.

On a prototype, firmware testing answers practical questions. Does the device start the same way every time? What happens at low battery? Does it recover if the Bluetooth link drops mid-transfer? Can a field update be interrupted without bricking the unit?

We do not pick one kind of test. We stack cheap, fast tests under slower, more realistic ones, so most bugs are caught where they are cheapest to fix.

A few of the tools above, so you know what they are: Unity is a unit-testing framework written in ANSI C that is designed to be portable from 8-bit microcontrollers to 64-bit systems (ThrowTheSwitch, Unity).

Zephyr's Ztest provides assertion macros and a test structure that can be used for unit testing modules or for integration testing (Zephyr Project, Ztest).

Renode is Antmicro's open-source framework for emulating CPUs, peripherals, sensors and multi-node systems so teams can develop and test without the physical hardware (Renode).

pytest-embedded is an open-source pytest plugin from Espressif designed for embedded testing, with services for serial ports and emulators (pytest-embedded on GitHub).

Layer 5 is a different job, covered in our guide to hardware beta testing. Where all of these sit in a program is in stages of product testing and their purposes.

Short answer: usually both, at different times. SIL runs your compiled code in a simulated environment, so it is fast, cheap, and available before boards arrive. HIL runs the code on the real controller hardware connected to a simulated environment through real analog and digital signals, so it catches what SIL cannot.

MathWorks describes SIL as testing the compiled control code in a simulated environment, and HIL as connecting the real controller hardware to a real-time simulated plant using actual analog and digital signals, which gives higher-fidelity testing of I/O, timing, and communication (MathWorks, Hardware-in-the-Loop).

NI adds that running the code on the actual controller is what exposes timing, synchronization, and execution flaws that are typically not present in a SIL environment (NI, What Is Hardware-in-the-Loop).

For a simple Class B product (one mechanism or simple electronics), unit tests plus a few on-target tests are often enough. For Class C and D products (several mechanisms, a custom board with firmware, or several subsystems working together), a HIL bench usually earns its keep.

A HIL test bench does not have to be an automotive-grade rack. For most prototypes it is a bench that can power the board, pretend to be its sensors and loads, inject faults, and run a script.

NI lists the core elements of a HIL system as the device under test, I/O and bus interfaces to connect to it, real-time compute, application software to run and automate test cases, and simulation models, with optional fault insertion, signal conditioning, and emulated loads (NI). Here is how we scale that down for a prototype.

List what the firmware must do and what must never happen. If you ran a design FMEA, its failure modes and detection controls are a ready-made test list (see DFMEA for prototypes). Each test gets a measurable pass limit tied to a requirement.

Keep the controller board real. Simulate what is slow, dangerous, expensive, or hard to control by hand: a battery that sags, a motor that stalls, a sensor that reads out of range, a radio link that drops. MathWorks notes that HIL lets engineers test fault scenarios and edge cases without risking expensive equipment (MathWorks).

A programmable power supply lets the script control voltage and current, which is how you test brownout and low battery.

Add a way to reset and reflash the board from the script, inputs that stand in for sensors (DAC outputs, signal generators, or a second microcontroller that emulates a sensor bus), and measurement of the board's outputs.

For repeatable contact to test pads, the bench often sits on a pogo pin fixture; our test fixture design guide covers that part.

Use a test runner that sets up each scenario, measures, compares against limits, and saves a record per run. OpenHTF is an open-source Python hardware testing framework originally created at Google that structures tests as phases with measurements and limits (OpenHTF on GitHub). Pytest-embedded is another option for serial-connected devices. Keep limits in one file and version the scripts with the firmware and board revision.

Run the suite on a known-good board several times before trusting it. A bench that fails good boards wastes more time than no bench.

If the product has mains voltage, large batteries, or moving parts, add current limits, interlocks, and guards, and treat the bench as lab equipment. MathWorks notes that typical controller HIL uses low-voltage signals, and that power HIL with amplifiers is a separate, higher-energy setup (MathWorks).

The real value of a bench shows up when it runs without anyone babysitting it. Hardware test automation means every firmware change is built, flashed to a board on the bench, and run through the same suite, with results stored next to the commit. NI lists test automation up to continuous integration pipelines as an extension of a HIL system (NI).

What we automate first on a prototype, in order:

Automated testing of embedded systems does not replace an engineer at the bench. It makes sure the bugs you already fixed stay fixed.

AI tools are genuinely useful in firmware testing, mostly for writing and thinking, not for judging. Our split:

The best evidence we know of for AI test generation comes from software, not firmware, but the lesson transfers.

Meta reported that its TestGen-LLM tool, which uses large language models to improve existing human-written tests, filters every generated test so that only tests that build, pass reliably, and increase coverage are kept. In its evaluation on Instagram's Reels and Stories products, 75% of generated test cases built correctly, 57% passed reliably, and 25% increased coverage (Alshahwan et al., Meta, arXiv 2402.09171).

In other words, a large share of raw AI output was not usable until it was filtered by actually running it.

NIST's Generative AI Profile names confabulation, confidently stated but false content, as a core generative AI risk, and also warns about over-reliance on AI output and about exposing sensitive data (NIST AI 600-1).

For firmware tests that gives us three rules: an AI-generated test only counts after it runs on the bench and is reviewed. Pass limits come from the requirements, never from the model. And schematics, source code, and test data only go into tools whose data terms we have checked.

For the wider tool picture, see AI tools for hardware prototyping in 2026, and for on-device AI products, edge AI hardware prototyping.

If your product is a medical device, plan the firmware testing to fit the software life cycle standard from the start.

IEC 62304 defines life cycle requirements for medical device software and applies when software is itself a medical device or an embedded or integral part of one (IEC 62304:2006, IEC Webstore).

For automotive and aerospace controllers, MathWorks notes that HIL testing supports requirements-based testing under standards such as ISO 26262 and DO-178 (MathWorks).

Retrofitting traceability later is much harder than keeping requirement IDs on tests from day one. See also design verification and validation.

LA NPDT does design, engineering, and prototyping in-house at our facility in Ruston, Louisiana: electronics and firmware, plus 3D printing, CNC machining, urethane casting, and bridge tooling.

The same team that writes the firmware builds the prototype and the bench, so when a test fails we can tell whether it is the code, the board, or the mechanics, and fix it in the same build. We use AI to speed up test ideas, test code, and log reviews, and our engineers set the limits and sign off the results.

Our view is practical: test in layers, automate the tests you will run more than a few times, and build a HIL bench when faults are risky, slow to create by hand, or will keep coming back with every update. If a product is simple enough that a short on-target test list covers it, we say so.

How much firmware testing a project needs depends on its complexity class.

We sort every product into four classes: Class A, a single simple part with no moving parts or electronics (for example, a plastic or rubber kitchen utensil); Class B, one mechanism or simple electronics; Class C, multiple mechanisms or medium electronics with firmware. And Class D, complex products with multiple subsystems.

Typical costs for each class are in one place on our homepage: product development costs by complexity. As with all our work, equipment and machine time are part of labor, and materials are billed separately.

If you want the plan first, our Stage 1: Review/Analysis + Plan is a fixed $800 single payment. Or see our electronic design services and rapid prototyping, or book a free 20-minute initial consultation. Phone: +1 318-200-0526.