Capstone: Beacon Pilot Deployment and Post-Mortem
Thirty days in a real workshop taught more than six stages — which is exactly why the six stages existed.
🎯 By the end you'll understand
What twenty-five Beacons actually did in thirty days in a working machine shop, reported against a failure criterion written before deployment — which the pilot failed on two clauses and could not evaluate on the third. Why the six-month battery requirement is still a projection rather than a measurement, and precisely why thirty days could never have settled it. How every one of Beacon's three known Rev A defects behaved in the field, including the root cause of the 1.8 µA sleep leak, found on day 22 and traceable to a gate that a 3.3 V pin cannot switch off against a source sitting on the cell. How to grade a system honestly as bench-proven, pilot-proven, or still-research, and how to trace every requirement from the sentence in Lesson 1.9 to the evidence that does or does not support it. Rev B's change list with the evidence attached to each item, priced. And the uncomfortable list of what a real product would still need before anyone could sell one.
KEY INSIGHTThe pilot failed, and failing was the point
A pilot without a written failure criterion cannot fail, which means it cannot inform anything. Beacon's criterion was written before deployment. Two of its three clauses failed, and the third turned out to be unfalsifiable — which was the most useful finding of the three.
The most common way a hardware pilot goes wrong is that it succeeds. Twenty-five units go out, most of them work most of the time, everybody agrees it went well, and the programme moves to production carrying every defect it started with. The mechanism is simple: with no criterion agreed in advance, the result is evaluated against whatever expectation survives contact with the data, and that expectation adjusts itself downwards at exactly the rate needed to produce a pass.
So before a single unit was deployed, three sentences went into the log, dated and unedited afterwards. One: no more than 2 of 25 units may require physical intervention during the thirty days. Two: fleet-wide report delivery must be at least 90 %. Three: no unit's projected battery life may fall below six months. Clause one failed — five units needed hands, across six interventions. Clause two failed — delivery was 84.5 %. Clause three could not be evaluated at all, because thirty days consumes under three per cent of the cell's capacity, which is smaller than the measurement's own uncertainty. A criterion you cannot evaluate is not a criterion, and discovering that is worth more than a passing grade would have been.
An honest post-mortem has four parts and they must appear in this order: what was measured, unadjusted; what it is attributed to, so a reader can disagree with your reasoning rather than your arithmetic; what will change, with the evidence for each item; and what remains unknown, stated as explicitly as the knowns. Reports that omit the fourth part read better and are worth less, because the unknowns are where the next failure lives.
HOW IT WORKSHow the pilot was designed
Twenty-five units, provisioned through the jig from Lesson 6.3 in about two minutes each, went into a working machine shop measuring roughly 22 by 14 metres: steel benches, a CNC mill in a metal enclosure, a partition clad in sheet steel dividing the space, coolant mist near the mill, and people moving through all day. Each unit was attached to a piece of equipment — a tool cart, a measuring instrument, a rack of stock — using the printed enclosure from Lesson 6.5 with its adhesive pad. A Raspberry Pi Zero 2 W at one end of the shop scanned for advertisements and published each report to MQTT; a forty-line Python subscriber wrote a CSV row per report and served a one-page status table.
Three deliberate choices in that setup are worth naming, because they determined what the pilot could and could not learn. The shop was chosen because it is hostile — vibration, metal, coolant, and people — rather than because it was convenient; a pilot in a quiet office would have produced a cleaner result and less information. The gateway was deliberately singular, one receiver at one end, because the point was to find out what the radio link does in a real building rather than to guarantee coverage. And the units shipped with the known Rev A defects unfixed, because the canon's three defects were understood but their consequences were not, and consequences are exactly what a pilot buys.
EXAMPLEThe honest numbers
Delivery: 84.5 %, and three quarters of the loss is geometry
Twenty-five units reporting once a minute for thirty days is 1,080,000 expected reports. The gateway logged 913,000, a delivery rate of 84.5 %, against a criterion of 90 %. The interesting part is the attribution: of the 167,000 missing reports, about 74 % are radio coverage — seven units that were behind a steel-clad partition or too far from the single gateway — and only 26 % are unit failures. The received signal strength tells the same story cleanly.
| Placement | Units | Typical RSSI at the gateway | Delivery |
|---|---|---|---|
| Line of sight, under 12 m | 18 | −62 to −84 dBm | ≥ 99 % |
| Behind the steel-clad partition | 5 | −96 to −99 dBm | 61–74 % |
| Far bench, 18 m plus the partition | 2 | below −99 dBm, intermittent | 34–41 % |
Requirement R2 asked for ten metres through one plaster wall. Measured against that sentence, Beacon passes comfortably: every unit within twelve metres of line of sight delivered better than ninety-nine per cent. The units that failed were behind a sheet-steel partition at eighteen metres, which is not a plaster wall and is not ten metres. R2 is met and R2 is wrong. The requirement described a building nobody was going to deploy in, and it was written that way in Lesson 1.9 because "one plaster wall" was a concrete-sounding phrase rather than because anyone had surveyed a workshop. That is the single most transferable finding in this post-mortem, and it is a requirements defect rather than an engineering one.
Battery: a projection, still
Every unit was deployed at between 4.05 and 4.12 V and read between 3.98 and 4.05 V on day thirty, each reading taken 300 ms after the divider was switched on — the 2 × 1 MΩ divider with its 100 nF cap has a 50 ms time constant, so anything under about 250 ms of settling reads low and reads low by a different amount every time. The mean drop was 62 mV; the two units on the CNC bench dropped 71 and 74 mV. Now the arithmetic that matters. Measured sleep current across all thirty jig-tested units was 9.5 to 10.1 µA, and the active share is 0.045 s × 40 mA / 60 s = 30 µA, so the average is about 40 µA. Over 720 hours that is 28.8 mAh — 2.9 % of a 1000 mAh cell. On a lithium-polymer discharge curve that region is nearly flat, and the battery-sense divider's measurement carries roughly ±15 mV of uncertainty after calibration.
So the observed 62 mV drop is consistent with 28.8 mAh, and it is equally consistent with anything from about 17 to about 41 mAh once the uncertainty is propagated — an error band near ±40 % on the implied consumption. The projection from the current measurement is 1000 mAh / 40 µA ≈ 25,000 hours ≈ 2.85 years, which is the nameplate figure, not the answer. Apply the canon's 6× real-world derating and it becomes 4,167 hours — 5.7 months, just under requirement R1's six. The leak did not merely eat margin; on the derated ledger it is the difference between passing and missing, which is the same conclusion Lesson 4.3 reached for this board from the bench alone. The pilot did not verify it and could not have. What the pilot does establish is that no unit exhibited an unexpected drain, which rules out a whole class of gross faults and is worth having. It is not the same claim, and the report says so in those words. Note also that the two CNC-bench outliers at 71 and 74 mV are consistent with the extra motion wakes described below — and are not statistically distinguishable from the 62 mV mean given ±15 mV, which is a second illustration of the same limit.
False motion: two units, a couple of hundred events a day
The two units on the bench beside the CNC mill logged between 180 and 260 motion events a day, against an expected count under ten. Neither tag moved at any point in the thirty days. The cause is straightforwardly mechanical: the LIS2DH12's interrupt threshold was set on a quiet bench to 2 LSB at ±2 g full scale — 32 mg — with the duration register at zero, so a single sample above threshold fires the interrupt. Machine vibration through a steel bench clears 32 mg easily.
In week four, four units were re-flashed with a raised threshold of 6 LSB (96 mg), a duration of three samples at the 10 Hz output rate, and a sixty-second firmware refractory after any motion report. Events fell to under six a day on the CNC-bench units, and a deliberate physical move of a tag was still detected in ten trials out of ten. The energy cost of the storms was modest — 220 extra wakes a day at about 0.5 µAh each is roughly 4.6 µA, about twelve per cent on top of a 40 µA average — but the reporting noise was not: a "moved" flag that is set on nearly every report is a flag carrying no information, which is the subject of Lesson 6.7's widget.
BEACONThe three canon defects, in the field
Rev A shipped with three known defects. All three were understood before deployment; none of their field consequences were.
Defect 1 — the button on GPIO9, a boot-strapping pin
Three units failed to start after a routine battery swap, on days 18, 21, and 24. In each case a technician held the button while clipping the cell in; GPIO9 low at power-up puts the ESP32-C3 into ROM download mode, where it waits for a host over serial, never runs the application, never advertises, and draws tens of milliamps indefinitely. Each unit was recovered by removing and refitting the battery without touching the button. The field consequence is therefore not damage but silent absence, which is the worst kind: the units were discovered because somebody scrolled the dashboard to a "last seen" column that existed by accident, and the mean time to discovery was about a day. Lesson 6.1 asked whether this would matter in the field and answered "we don't know yet." Now we know: it matters every time a unit is serviced by someone who has not been told which finger to keep off the button.
Defect 2 — the battery divider leaking 1.8 µA, and the transistor that could never turn off
OPEN-3 was raised on the first day of bring-up: 9.6 to 10.0 µA of deep-sleep current across five boards against a canon budget of 8.0 µA, identical on every board, therefore a design defect rather than workmanship. The log's hypothesis was recorded with its arithmetic: two 1 MΩ resistors across 3.7 V draw 1.85 µA, an almost exact match for the 1.8 µA excess, so the divider's switching PMOS was probably not fully off during sleep. The jig later measured 9.5 to 10.1 µA across all thirty units, inside a limit set from the same population — which, as Lesson 6.3 noted, is precisely why a production test cannot see a defect every unit shares.
The root cause was found on day 22, and it was on the schematic the whole time. The divider's switch is a P-channel MOSFET whose source sits on VBAT — 3.7 V nominal, up to 4.2 V charged — and its gate is driven by GPIO1, a 3.3 V digital output. Turning a PMOS fully off requires VGS = 0, which means putting the gate at the source. The highest that pin can reach is 3.3 V, so with the pin driven high the gate sits about 0.9 V below its own source. That is past the threshold of any small-signal PMOS you would fit here, so the switch is partially on, the two-megohm divider stays connected across the cell, and 3.7 V / 2 MΩ = 1.85 µA flows for fifty-nine seconds out of every sixty. The log's day-one arithmetic was exactly right; what nobody asked was why the switch could not be off.
No firmware state fixes this, and it is worth being precise about why, because the obvious idea is wrong. Driving the pin high is already the best a 3.3 V pad can do. Releasing it to high impedance is worse: the pad's ESD protection diode clamps the gate to roughly a diode drop above VDD, which still leaves VGS negative, and it back-feeds on the order of 64 nA into the 3.3 V rail through an ESD structure that is not a circuit element and was never characterised as one. The fix is hardware — a small NMOS (or NPN) as a level shifter with a 4.7 MΩ pull-up to VBAT, so the gate rests at VBAT and VGS is exactly zero. Bodged onto three bench boards with a transistor and a resistor, sleep current fell from 9.8 µA to 8.1 µA, against a canon budget of 8.0.
Beacon's LED rail uses a high-side PMOS too, driven directly from GPIO10 with a 100 kΩ pull-up. That one is right, and the difference is one word: its source is the 3.3 V rail, not the battery. A 3.3 V GPIO driven high therefore reaches VGS = 0 exactly, and the switch is genuinely off — while the pull-up holds it off through reset and through sleep with no firmware involvement at all. Two P-channel switches, the same schematic shape, opposite verdicts, and the only thing that decides it is which rail the source is tied to. If you take one habit from this defect, make it this: for any high-side switch, check the gate drive against the source rail, not against logic levels.
This was findable at schematic review, and the bring-up log pointed at it on day one with arithmetic that turned out to be exactly right. It went unresolved for three weeks — not because it was hard, but because the six-fold derating margin on R1 made it non-urgent, and because the hypothesis ("the switch is not fully off") felt like an answer. It was only half of one: it named the symptom in circuit terms and stopped one question short of the cause. The process lesson is worth more than the microamps: an open item with a confident hypothesis is more likely to be neglected than one without, because the hypothesis feels like a resolution. Two rules follow. Every open anomaly gets a date by which its hypothesis will be tested, written in the log beside the hypothesis. And a hypothesis that names a state rather than a mechanism — "not fully off" rather than "VGS cannot reach zero" — is not finished.
Defect 3 — the WS2812B-2020 footprint with pin 1 rotated
The twenty-five pilot units were correct, because the rotation was fixed in the placement file the assembler used rather than in the library, and the assembler's preview confirmed it. So this defect produced no pilot incidents — with one exception. Of the five hand-reworked bench boards from Lesson 6.2, one developed an intermittent LED in week three. That is exactly the outcome Lesson 6.2 predicted when it logged the reworked joints as the least-characterised joints on the board, and it is the only piece of the pilot that behaved precisely as forecast. The library footprint is still wrong, which means anyone who reorders from the repository repeats the defect. It is a latent defect with a field consequence of zero and a future consequence of one full batch.
COMPARISONPer-subsystem grades: bench-proven, pilot-proven, still-research
Three grades, defined so they cannot be fudged. Bench-proven means it was measured under controlled conditions and behaved. Pilot-proven means it survived thirty days in a hostile real environment across a population. Still-research means the evidence does not support a claim, whether because the test was never run, the test could not resolve it, or the result was negative.
| Subsystem | Grade | Evidence | Known ceiling |
|---|---|---|---|
| Power train — protection, reverse-battery, buck | Pilot-proven | 30 units through the jig, 25 units × 30 days, zero regulator or protection failures | Behaviour below 3.4 V was seen twice and never characterised; no unit was deeply discharged on purpose |
| Sleep architecture and the microamp budget | Bench-proven | 9.5–10.1 µA on 30 units; root cause of the excess found on day 22 and re-measured at 8.1 µA on 3 bodged boards | The fix is a hardware change, so no deployed unit could receive it. The fleet ran all thirty days at 40 µA |
| Six-month battery life (R1) | Still-research | Projection of 2.85 years from a 40 µA measurement; 30 days consumes 2.9 % of capacity, under the divider's noise floor | No measurement of this duration can resolve it. Needs a longer test, a coulomb counter, or an accelerated argument |
| Charging — MCP73831, USB-C, protection | Bench-proven | One full charge per unit at deployment; the jig skipped the charge-current test, so no per-unit charging data exists | Zero recharge cycles in the pilot. Cycle life, charge at temperature, and connector wear are all untested |
| Sensors — presence and function | Pilot-proven | 30/30 addressed correctly on the jig; 24 units reporting plausible values for 30 days | One unit contaminated by coolant mist; the recovery procedure worked but the exposure is a deployment constraint |
| Sensors — accuracy against R3 | Bench-proven | Agreement within 0.4 °C and 2.1 %RH against the jig's reference, at one temperature | No calibrated reference was ever in the field. ±0.3 °C and ±3 %RH remain datasheet claims, not measurements |
| Motion wake and report latency (R3, R4) | Bench-proven | 310 ms from interrupt to first advertising packet, measured on a scope — of which only the canon's 45 ms is radio-active at 40 mA; the rest is the deep-sleep wake and init, drawing far less. Had all 310 ms run at 40 mA the average would be 215 µA and R1 would be gone | Never measured in the field; and with 84.5 % delivery, the latency of a lost report is unbounded |
| BLE link and payload | Pilot-proven | 913,000 reports decoded correctly; CRC failures negligible; payload layout stable across the fleet | 84.5 % delivery in this building with one gateway. The link works; the coverage plan did not exist |
| Gateway and dashboard | Bench-proven | Collected and stored 30 days of data without loss on the receiving side | No liveness alerting and no analytics. Both of the incidents it should have caught were caught by a person looking at a table |
| OTA with rollback (R7) | Bench-proven | One successful update and one deliberate rollback after a failed self-test, on the bench | Never exercised deliberately in the field. Its one accidental activation drained a unit's battery in under three days |
| Enclosure — mechanical | Bench-proven | 25 shells, no cracks, no adhesive failures, one tolerance-stack failure on the button boss | The stack that killed unit 14 was in specification on every part. Worst-case stack was never computed |
| Enclosure — sensing behaviour | Still-research | Sealed v1 measured a ~90 minute humidity time constant; v2 with a membrane vent measured ~12 minutes | v2 was never deployed. The fleet ran the whole pilot reporting the enclosure's air, roughly an hour and a half late |
| EMC and certification readiness | Still-research | Pre-compliance only: passes on battery by about 13 dB, fails by 3.7 dB at 77 MHz with a 1 m USB cable | The product as built would not pass Part 15 Subpart B with a cable attached. No certification of any kind exists |
| Production test and provisioning | Pilot-proven | 30 units provisioned at about two minutes each; caught one real defect a functional test would have passed | Structurally blind to any defect shared by every unit; no thermal coverage, no charger test, no button test |
| Reliability characterisation | Still-research | 18,000 unit-hours, six incidents. By the rule of three, even a flawless pilot would support only ~6,000 hours at 95 % confidence | Every wear-out mechanism — flash, cells, connectors, corrosion — has a time constant longer than the test |
Count them: five pilot-proven, six bench-proven, four still-research. The pattern is worth naming. Everything that could be measured in a jig or on a bench got a grade; everything whose time constant exceeds thirty days, or whose verification needs equipment the project does not own, is still research. That is not a criticism of the pilot — it is what a pilot is for and what it is bounded by — but a report that presented eleven greens and stopped there would be lying by omission.
SEE ITRun the post-mortem — attribute the failures, then spend Rev B's budget
Two tasks. First, attribute each incident to the design decision that caused it — not to the component that broke, but to the choice, in Stages 1 through 5, that made the failure possible. Then choose Rev B's changes under a real budget: IDR 3,000 per unit of added bill of materials and 45 engineering hours. Taking everything costs IDR 3,200 and 51 hours, so something has to go. The board scores whether your change list addresses causes or symptoms, and shows which requirements your choices touch.
The instructive attribution is unit 14. The proximate cause is the enclosure's tolerance stack, and shortening the boss is a real fix — but the decision that made a dead battery possible was made in Stage 4, when the update path was gated on a button level rather than on an edge. A level can be held by a piece of plastic, by tape, by a shelf. An edge cannot. Fix the boss and you have fixed one enclosure; fix the gate and the entire class of stuck-button failures becomes impossible, for nothing, in firmware that reaches deployed units. When a mechanical variation causes a firmware state, the firmware is where the cheap, general fix lives.
IN PRACTICERequirements traceability, R1 through R9
Lesson 1.9 wrote nine requirements. Here is each one traced to where it was implemented, what evidence exists, and an honest status. "Met" means evidence supports the claim. "Bench-met" means evidence exists but only under controlled conditions. "Unverified" means the claim may well be true and nothing available tests it.
| # | Requirement | Implemented in | Evidence | Status |
|---|---|---|---|---|
| R1 | ≥ 6 months at 1 report/min on 1000 mAh | 3.7, 4.2, 4.3, 4.6 | 40 µA measured average → 2.85 year projection. 30-day cell drop of 62 mV against ±15 mV uncertainty | Unverified — the test cannot resolve it |
| R2 | ≥ 10 m indoor range through one plaster wall | 4.5, 5.7, 6.5 | ≥ 99 % delivery to 12 m line of sight; 2 of 25 units below 41 % behind a steel partition at 18 m | Met as written — and the requirement described the wrong building |
| R3 | Temp ±0.3 °C, RH ±3 %, motion wake ≤ 100 ms | 3.5, 3.6 | 0.4 °C and 2.1 %RH against the jig reference at one temperature; interrupt latency measured on a scope | Bench-met — no calibrated field reference ever existed |
| R4 | Report latency ≤ 2 s after motion | 3.4, 4.6 | 310 ms from interrupt to first advertising packet, on the bench | Bench-met — and for a lost report the latency is unbounded |
| R5 | BOM ≤ IDR 150,000 at 25 units, excluding enclosure | everywhere, costed in 6.8 | IDR 151,522 BOM; IDR 195,522 landed with assembly | Missed by 1.0 % — and it measured the wrong quantity |
| R6 | USB-C charging, 500 mA, with protection | 4.4 | 480–500 mA into a half-charged cell on 30 units; TVS and protection pair fitted; one charge per unit at deployment | Met |
| R7 | OTA updates with rollback | 3.2, 4.9 | One successful update and one deliberate rollback after a failed self-test, on the bench | Bench-met — never used deliberately in the field |
| R8 | PCB ≤ 35 × 25 mm, 4 layers | 5.4, 5.5 | 35.0 × 25.0 mm on the JLC04161H-7628 stackup, DRC and DFM clean | Met — and the constraint is what blocked second-sourcing the sensor |
| R9 | Indoor, 0–45 °C, IP54 enclosure optional | 6.5, 6.6 | Workshop range 24–34 °C over 30 days; enclosure IP54 achievable, never tested to IEC 60529 | Partially evidenced — neither end of the temperature range was exercised |
Read the status column as a whole and a pattern appears that no individual row shows. Two requirements are cleanly met. Three are bench-met, meaning the claim is probably true and the evidence is from a bench rather than from the world. One is missed. One is met against a sentence that described the wrong situation. One is partially evidenced. And one — the requirement that defines the entire product — is unverified and cannot be verified by any experiment this project has run. That distribution is normal for a good first pilot, and it is invisible unless somebody writes the table.
Rev A was brought up on a current-limited supply with a written prediction at every rung. A pogo-pin jig and a production-test image provisioned thirty units in about two minutes each. A pre-compliance sweep was run and produced a specific, actionable failure. Twenty-five units in printed enclosures ran for thirty days in a real workshop. The post-mortem below produces Rev B's change list, with evidence attached to every item. The milestone is met. Every open item in the status column above is the honest frontier, and the next section prices it.
IN PRACTICERev B's change list, with the evidence for each item
Twelve changes, each with the evidence that justifies it, its type, and its cost. Nothing on this list is here because it seemed like a good idea; every row points at a measurement or an incident.
| # | Change | Evidence | Type | IDR/unit | Hours |
|---|---|---|---|---|---|
| 1 | Move the button off GPIO9 to a wake-capable, non-strapping pin | Three lock-ups after battery swaps on days 18, 21, 24; canon defect flagged and accepted in 5.9 | HW + FW | 300 | 6 |
| 2 | Require a fresh button press after wake before entering the update path | Unit 14 flat on day 4; a held level averaged ≈ 17 mA against a 40 µA design | FW | 0 | 3 |
| 3 | Level-shift the divider PMOS gate: a small NMOS plus a 4.7 MΩ pull-up to VBAT, so the gate rests at the source | OPEN-3: 9.5–10.1 µA on 30 units against an 8.0 µA budget; VGS = −0.9 V with the pin driven high; root cause day 22; 8.1 µA re-measured on 3 bodged boards | HW | 300 | 4 |
| 4 | Correct the WS2812B-2020 footprint's pin 1 in the project library | 5 of 5 boards rotated in batch 1; CPL workaround in batch 2; the library is still wrong | Library | 0 | 1 |
| 5 | Set the buck to 3.0 V via VSET; add a 3.45 V firmware cutoff and safe state | Two brown-out resets in week 4, early morning, cells near end of discharge | HW + FW | 200 | 9 |
| 6 | Relieve the enclosure button boss by 0.4 mm; add a travel shoulder | Unit 14's tolerance stack closed with every part in specification | Mechanical | 0 | 2 |
| 7 | Raise the motion threshold to 96 mg, duration 3 samples, 60 s refractory | 180–260 events/day on two CNC-bench units; a four-unit trial in week 4 gave under 6/day with 10 of 10 real moves still detected | FW | 0 | 3 |
| 8 | Common-mode choke footprint on D±; stitching vias along the USB edge | Pre-compliance failed by 3.7 dB at 77 MHz with a 1 m cable; a clamp ferrite recovered only 1.1 dB of margin | HW | 1,900 | 5 |
| 9 | Re-allocate telemetry: motion count and wake source into spare bits; gateway liveness rule | One of six pilot incidents was diagnosed remotely; two were absences no payload could ever report | FW + gateway | 0 | 6 |
| 10 | Requirements: R5 to landed cost at a stated volume and rate; R2 to name a building and a delivery target; add a product-level humidity response time | R5 missed by 1 % while excluding 29 % of the real cost; R2 met while two units could not reach the gateway | Requirements | 0 | 2 |
| 11 | Second gateway plus store-and-forward retry in firmware | 74 % of all missing reports were coverage, concentrated in seven units | Deployment + FW | 0 | 8 |
| 12 | Adhesive strain relief on the battery lead; a retention rib in the lower shell | Unit 22 silent on day 11 — a friction-fit JST-PH on a bench a CNC mill vibrates all day | Mechanical | 500 | 2 |
| Total if everything is taken | 3,200 | 51 | |||
Two observations about that table. The first is how cheap it is. Three thousand two hundred rupiah per unit — about two per cent of the bill of materials — plus a little over a working week of engineering, removes five of the six pilot incidents, closes the sleep-current defect, and puts the board within reach of an electromagnetic-compatibility test it currently fails. Rev B is not a redesign. It is a list of small, specific things, each of which exists because something measurable happened.
The second is the type column. Four of the twelve changes — the fresh-press gate, the motion threshold, the telemetry re-allocation, and the store-and-forward retry — are firmware or gateway work that reaches units already in the field without a new board, a tool, or a lead time. That is what requirement R7 bought when it asked for over-the-air updates back in Lesson 1.9, and it is the strongest argument in this course for treating updateability as a first-class requirement rather than a convenience. A programme where every fix needs a respin has a different shape, a different budget, and a much worse Rev B.
The list also does not fit its own budget, which is worth recording rather than hiding. Rev B was allocated IDR 3,000 per unit and 45 engineering hours; the full twelve items are IDR 3,200 and 51 hours. Two decisions closed the gap. Item 11, the second gateway, was reclassified as a deployment task rather than an engineering change, removing eight hours — it needs a purchase and a site survey, not a firmware release. And the IDR 200 per-unit overrun was accepted explicitly, in writing, because it buys items 3 and 8 — the level shifter that closes the sleep-current defect and the common-mode choke that gates every commercial option the product has. An overrun accepted with a stated reason is a decision; an overrun discovered later is a surprise.
Three tempting changes were rejected, and the reasoning is part of the record. A bigger battery — proposed after unit 14 — treats a symptom of a firmware defect with a permanent increase in size and cost, and requirement R8 has no room for it anyway. Forcing the regulator into PWM mode to clean up the emission spectrum improves the margin by about three decibels, still fails with a cable attached, and destroys R1 outright; it is not a trade, it is a different product. Conformal coating after unit 07's contamination would not have helped, because the humidity sensor is the one part that must not be coated. Its fix is a mounting instruction and a fleet-analytics rule, neither of which is a hardware change. Writing down the rejected changes, with reasons, is what stops them being re-proposed every quarter.
DEFINITIONWhat a real product would still need
Beacon works. Twenty-five of them ran in a hostile building for a month and produced nine hundred thousand useful measurements. It is not a product, and the gap is worth stating precisely rather than gesturing at, because every item on this list is a specific piece of work with a specific cost.
Certification — and the fix that has to land first
The finished product needs FCC Part 15 Subpart B compliance in the United States, a CE declaration under the Radio Equipment Directive including immunity testing in the European Union, and SDPPI type approval held by an Indonesian legal entity for sale in Indonesia. The module's grant covers the radio and nothing else. Order of magnitude: IDR 174 million and eight to twelve weeks, and none of it can start until Rev B's common-mode choke fixes the 3.7 dB failure with a cable attached. Until then, units may be used internally and may not be sold.
Supply chain
Half the bill of materials — the module at 24 % and the humidity sensor at 25 % — is single-sourced with no footprint-compatible alternative. Beacon needs a second source designed in for the sensor, which requires relaxing R8's 35 × 25 mm limit; a purchase history with a distributor before allocation ever bites; lifecycle monitoring on every line so an end-of-life notice arrives with its last-time-buy window intact rather than after it closes; and a stated currency assumption, because a dollar-priced bill against a rupiah requirement moves with the exchange rate whether or not anyone changes the design.
Support and serviceability
A returns process. A documented way to recover a unit that has entered download mode without opening the case — which today does not exist, and which Rev B's button change makes unnecessary rather than possible. Spare cells and shells, and a decision about whether the cell is user-replaceable at all. A test specification a contract manufacturer can execute without you in the room. A serial-number registry linking each unit to its build, its firmware, and its test record. And somebody whose job it is to answer when a customer calls, which is not an engineering problem and is the one most often discovered last.
Reliability evidence that supports the claim you want to make
Eighteen thousand unit-hours support an MTBF claim of roughly six thousand hours at 95 % confidence — under a year, even had the pilot been flawless. A product sold with a multi-year expectation needs a larger population, a longer test, or an accelerated-life argument with assumptions you are prepared to defend. It also needs a deliberate answer on the wear-out mechanisms the pilot structurally could not see: cell calendar ageing, flash erase cycles, connector insertions, and corrosion through a wet season.
Security and privacy, currently absent
The advertising payload is unauthenticated: the CRC detects corruption and does nothing against a person in range transmitting whatever they like. The identifier is static, so a log of reports in a workplace is also a log of where people were. And the over-the-air update path will eventually face a network rather than a bench, at which point image signing and secure boot stop being optional. None of these mattered for an internal pilot. All of them matter the first time a unit is sold.
A volume that makes the arithmetic work
At 25 units with certification amortised, a Beacon costs around eleven and a half million rupiah. At 2,500 it is about two hundred and sixty thousand against a plausible four hundred and fifty thousand price. The product becomes viable somewhere between one and two thousand units, and that number should have been derived in Lesson 1.9 alongside the requirements rather than in Lesson 6.8 alongside the post-mortem. Knowing it changes which requirements are worth having.
WEAKNESSWhere this bites
Without a criterion agreed and dated before deployment, the result is graded against an expectation that adjusts downward exactly as far as it needs to. Six interventions across five units becomes "a few teething problems." Eighty-four per cent delivery becomes "mostly good coverage." Both statements are true and neither is a decision. The criterion is what converts a month of data into a yes or a no, and the discipline that makes it work is writing it down where it cannot be edited afterwards. Beacon's pilot failed two clauses of its own criterion, and that failure is why there is a change list at all.
The pilot's six incidents make an excellent slide. What that slide leaves out is that thirty days tested corrosion not at all, cell ageing for a twelfth of a year, cell cycling for zero cycles, flash wear for 0.4 % of its budget, and both ends of the specified temperature range for no time whatsoever. A report listing only what happened is a report about thirty days. A report that also lists what could not have happened is a report about the product. The second one is harder to write, less satisfying to present, and the only one that helps anyone decide what to do next.
Unit 14 died because a plastic boss held a button down. Shorten the boss and the incident cannot recur in that exact form — and tape, a shelf, a warped shell in a hot van, or a different print run all recreate it. The decision that made the failure possible was gating the update path on a button level rather than an edge, three stages earlier, and changing that costs nothing and reaches units already deployed. Proximate causes are easy to see and cheap to fix, which is exactly why they absorb the attention that root causes need. The test to apply: after your fix, can you describe a slightly different accident that produces the same failure? If yes, you fixed the instance.
KEY INSIGHTHow this connects — and what you built
- Requirements: Writing R1 Through R9 (1.9) — every row of the traceability table starts there, and two of the nine requirements turned out to be measuring the wrong thing. That is a normal result, and it is only visible because the requirements were written down as numbers.
- Beacon Rev A: Design Review Before Fab (5.9) — the review flagged the GPIO9 button and accepted it. The acceptance was reasonable and the field consequence was not knowable; what the review lacked was a way to record "accepted, consequence unknown, revisit after the pilot."
- Reliability and Field Failure Modes (6.6) — the projected column of that lesson's Pareto is the part of this post-mortem the pilot could not measure, and it is where Rev C's list will come from.
- Firmware in the Field: Logging, Telemetry, and Remote Diagnosis (6.7) — one incident in six was diagnosed remotely. Rev B's telemetry re-allocation is the direct response, and it costs no bits that were not already being transmitted.
- Cost, BOM, and Scaling From 25 to 25,000 (6.8) — Rev B's IDR 3,200 per unit is about two per cent of the bill, and the volume at which any of this becomes a business is between one and two thousand units.
Six stages ago, Beacon was a paragraph. It is now a board that has been fabricated, assembled, reworked, brought up on a current-limited supply, provisioned by a jig, swept against a regulatory limit line, enclosed, deployed, measured, and honestly criticised. The specific product matters less than the sequence, because the sequence is the transferable thing: predict, bound the damage, measure, compare against what you wrote down, and report the gap. That loop is what separates an engineer from someone who builds things that happen to work. It runs at every scale, from a single rung of a bring-up ladder to a thirty-day deployment, and it is the reason Rev B's change list is twelve specific items with evidence rather than a feeling that some things could be better.
The final honesty, which belongs at the end of a course rather than the start: Beacon is not finished and will not be. There is no version of this device that is done — only versions that have been characterised more carefully than the last one, with the remaining unknowns written down where the next person can find them.
GLOSSARYKey terms introduced
- Failure criterion
- A set of measurable conditions, agreed and dated before a trial begins, that determine whether it passed. Without one, results are graded against an expectation that quietly adjusts to produce a pass.
- Unfalsifiable criterion
- A pass condition the experiment cannot evaluate — like Beacon's six-month battery clause, which a thirty-day test cannot resolve because it consumes under three per cent of capacity, inside the measurement's own uncertainty.
- Bench-proven / pilot-proven / still-research
- Three grades for a subsystem's evidence: measured under controlled conditions, survived a real deployment across a population, or not supported by available evidence. The third grade is the one that makes the table honest.
- Requirements traceability
- An explicit mapping from each requirement to where it was implemented and to the evidence that supports or fails to support it. It surfaces the requirements that were met against the wrong sentence, which no architecture diagram can show.
- Proximate cause vs root cause
- The last thing that went wrong versus the decision that made it possible. Unit 14's proximate cause was a 0.3 mm tolerance stack; the root cause was gating an update path on a level rather than an edge. Fixing only the first leaves the class of failure intact.
- Latent defect
- A defect present in the design that has produced no field consequence yet. Beacon's rotated library footprint is latent: the pilot units were correct because of a placement-file workaround, and the next person to reorder repeats the defect.
- Change list with evidence
- A revision's changes, each attached to the measurement or incident that justifies it, plus the changes deliberately rejected and why. The rejected half is what stops the same proposals returning every quarter.
- Blind spot inventory
- The explicit list of mechanisms a test could not have exercised — for a thirty-day pilot: corrosion, cell cycling, flash wear, and both ends of the temperature range. Reporting it is the difference between a report about a test and a report about a product.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1The pilot's third failure criterion — "no unit's projected battery life below six months" — could not be evaluated. Explain why, and say what you would write instead.›
Because thirty days at a 40 µA average consumes 28.8 mAh, which is 2.9 % of a 1000 mAh cell, and that region of a lithium-polymer discharge curve is nearly flat. The observable is a 62 mV drop measured through a two-megohm divider whose uncertainty is about ±15 mV after calibration, so the implied consumption carries an error band near ±40 %. The criterion asks the experiment a question it does not have the resolution to answer, in either direction — it could neither confirm six months nor rule it out. A better criterion is one the test can actually settle, and there are two shapes. Either restate it in terms of the quantity you can measure precisely — "every unit's measured average current stays below 50 µA over the deployment," which the sleep and active measurements do resolve — or specify the instrument that would make the original question answerable, such as a coulomb counter on a subset of units. The general rule this teaches is worth more than the battery: when you write a pass condition, immediately ask what measurement would settle it and whether your setup can make that measurement. If it cannot, you have written an aspiration.
2Requirement R2 was met and the post-mortem still calls it a defect. How can both be true, and what does that imply about writing requirements?›
R2 asked for ten metres of indoor range through one plaster wall. Measured against that sentence, Beacon passed easily: every unit within twelve metres of line of sight delivered better than 99 %. But two of the twenty-five units delivered under 41 %, because they sat eighteen metres away behind a partition clad in sheet steel — a situation the requirement does not describe. The requirement was satisfied and the product did not work for those users, which means the requirement was measuring the wrong thing. It was written that way because "one plaster wall" sounds concrete and specific, and specificity is easily mistaken for correctness. The implication is that a requirement must be derived from the actual deployment environment, not from a plausible-sounding abstraction, and it should be stated as an outcome rather than a geometry — "at least 95 % report delivery from every mounting position in the target building" is testable, is what anyone actually wants, and would have forced a site survey and a coverage plan in Stage 1 rather than a surprise in Stage 6.
3OPEN-3's hypothesis was recorded correctly on day one and the root cause was not found until day 22. What went wrong in the process, and what rule prevents it?›
The arithmetic was right — two 1 MΩ resistors across 3.7 V draw 1.85 µA against a measured 1.8 µA excess — and so was the conclusion that the switching PMOS was not fully off. But the hypothesis named a state, not a mechanism, and it stopped one question short: why can this switch not be off? The answer was on the schematic. The PMOS has its source on VBAT at 3.7 V or more, and its gate is driven by a 3.3 V pad, so VGS is about −0.9 V even with the pin driven hard high, which is past threshold. No reachable firmware state helps, and releasing the pin to high impedance is worse because the pad's ESD diode clamps the gate and back-feeds current into the 3.3 V rail. The fix had to be a level shifter. Two failures compound here. The technical one is that gate drive was checked against logic levels instead of against the source rail — the same check, applied to the LED's PMOS whose source is the 3.3 V rail, correctly passes. The process one is that the item was neglected because it had a confident explanation attached: a plausible hypothesis stops the anomaly generating discomfort, and the six-fold margin on R1 made it non-urgent. Two rules follow, both cheap: every open anomaly gets a date by which its hypothesis will be tested, written beside the hypothesis; and a hypothesis phrased as a state rather than a mechanism is not finished. The cost of not having them was three weeks and twenty-five units deployed at 40 µA instead of 38.
4Rev B costs IDR 3,200 per unit and about 51 hours, and removes five of six pilot incidents. Why is that not evidence that the pilot was unnecessary?›
Because none of the twelve changes could have been written without it. Every row of the change list points at a measurement or an incident that only existed after twenty-five units spent thirty days in a hostile building. The GPIO9 button was reviewed in Stage 5 and deliberately accepted, because at that point nobody knew whether anyone would ever hold it during a battery swap; the pilot answered that with three occurrences. The motion threshold was chosen on a quiet bench and was correct there; only a steel bench beside a CNC mill revealed 32 mg to be far too sensitive. The button boss was in specification on every part. The 1.8 µA leak had a correct hypothesis and no proof. What the pilot bought is not a list of fixes — it is the attribution that makes the list defensible, which is the difference between a change you can justify to a reviewer and a change somebody thought would be nice. And the cheapness is the argument for pilots rather than against them: five million rupiah and a month converted twelve guesses into twelve evidenced decisions, at the only volume where a wrong guess costs almost nothing. The same twelve decisions discovered at 2,500 units would have cost inventory, re-certification, and a schedule.