# From 44 GB a day to 0.12

> How the unified sensor image stopped wearing out its own storage: what was writing, what we changed, the gates that keep it fixed, and what our Orb integration cost.

<svg viewBox="0 0 520 240" role="img" aria-labelledby="writes-title writes-desc" style="width:100%;max-width:600px;height:auto;font:17px system-ui,sans-serif">
<title id="writes-title">Storage writes per day at idle, by build</title>
<desc id="writes-desc">Before: 44 GB per day. Candidate 22: 4.2. Candidate 23: 2.7. Candidate 24: 0.83. Candidate 30, overnight steady state: 0.12.</desc>
<text x="0" y="32" fill="#586674">Before</text><rect x="150" y="12" width="260" height="28" rx="3" fill="#ab422e"/><text x="418" y="32" fill="#172536" font-weight="700">44 GB</text>
<text x="0" y="78" fill="#586674">Logs quieted</text><rect x="150" y="58" width="25" height="28" rx="3" fill="#087992"/><text x="183" y="78" fill="#172536" font-weight="700">4.2 GB</text>
<text x="0" y="124" fill="#586674">Sampler in RAM</text><rect x="150" y="104" width="16" height="28" rx="3" fill="#087992"/><text x="174" y="124" fill="#172536" font-weight="700">2.7 GB</text>
<text x="0" y="170" fill="#586674">MQTT in RAM</text><rect x="150" y="150" width="5" height="28" rx="3" fill="#087992"/><text x="163" y="170" fill="#172536" font-weight="700">0.83 GB</text>
<text x="0" y="216" fill="#586674">Overnight soak</text><rect x="150" y="196" width="2" height="28" rx="3" fill="#188267"/><text x="160" y="216" fill="#172536" font-weight="700">0.12 GB a day</text>
</svg>
<figcaption class="caption">Idle writes to the GL.iNet Slate 7 Pro&rsquo;s USB stick, 19&ndash;20 October 2026, read from the kernel&rsquo;s per-partition counters. Same device and stick, one build after another: the original image, then candidates 22, 23, 24 and 30. The last bar is the overnight steady state after boot; the others are 10-minute samples.</figcaption></figure>

- **What was writing:** the system journal took 88% of the writes. It was logging the phone app's connection checks, the screen's 5-second refresh and a 15-second health sampler. A messaging service no one had turned on was also saving its status file to disk every 2 seconds.
- **The fix:** sort every write into *durable*, which has to survive a power cut and goes to the Data partition, or *live*, which goes to RAM under `/run`. Write only when a value changes, never on a timer. Keep log noise out of the journal at the source.
- **What keeps it fixed:** the build now fails if any background task writes to storage more often than its budget allows. A hardware gate also fails a release candidate that writes more than 4 KiB/s at idle.
- **Our Orb wrapper's share:** with Orb switched *off*, WLAN Commander's own code for managing Orb was still about 30% of the health sampler's CPU, and it rewrote its settings file 239 times an hour. Those costs were ours, not Orb's. Details [below](#what-orb-cost).
- **Not yet measured:** the NanoPi's SD card on this image. Every rate on this page comes from the Slate's USB stick. Also not measured: how much Orb writes when it is switched on.

## How we measured

Two counters, read before and after every change: the kernel's per-partition write totals in `/sys/block/*/stat`, and each process's own count in `/proc/*/io`. Ten-minute samples at idle, with no tool running.

Measuring mattered because guessing failed twice. The first suspect for the steady 2-second write pattern was the automation worker reopening its database every second. We fixed that, and the pattern stayed: the database churn was real, but the kernel cancelled those writes before they reached the stick. The actual cause was the MQTT publisher's status file. Any time we name a writer below, it was confirmed by the counters, not by reading the code.

## What was writing, and what changed

| Writer | What it was doing | What changed |
|---|---|---|
| System journal | 461 KiB/s of log entries: SSH "Identity probe" handshakes from the app every 10 s, kiosk polls every 5 s, sampler runs every 15 s | Those lines are filtered at the source. 1.8 KiB/s afterwards, a 250-fold drop |
| Health sampler | Kept 24 hours of history on disk, while the console shows one hour. 160–290 KiB per 15-second run | One hour in a RAM ring, plus a small "last alive" note on Data every 5 minutes for diagnosing resets |
| MQTT publisher | Saved `status.json` with fsync and rename every 2 seconds, even with MQTT disabled. Each fsync forces a filesystem journal commit | The status file moved to RAM |
| Thermal history, clock, RADIUS progress, integrations | Small files saved on every tick: RADIUS progress up to 4 times a second, the clock's checkpoint copied every 5 minutes | Buffered in RAM. Thermal history is written once an hour, RADIUS only at the final phase, integration use at most every 10 minutes |
| Kiosk browser | Its profile and certificate database lived on storage | Its home moved to RAM |
| Journal memory mapping, Slate kernel only | About 70 minutes after boot the kernel merged the journal's mapping into one 2 MiB page, so every sync rewrote 6 MiB to change about 76 KiB | Huge pages limited to programs that ask for them. 44–48 KiB per sync afterwards |
| Access times, Slate only | The Slate has no clock battery. A boot a day after the last one rewrote the access time of every file boot reads, about 3,800 files | Data and root are remounted `noatime` early in boot. 5.06 → 2.61 MiB for that boot |

Two rules came out of this. We did not trade power-cut safety for wear: the filesystem still commits every 5 seconds, and the journal still syncs every 5 seconds. And we did not slow the app down: its fast discovery and connection checks stay as they are, and the sensor simply stopped logging them.

## How it stays fixed

**A budget for every background task.** A test runs each periodic task through a simulated hour and counts every file it creates, changes or removes on storage. Each task has a budget: the sampler 13 an hour, for its "last alive" notes; Orb holds, MQTT and storage health zero or one. Run against candidate 24's code, the harness counted an ACME certificate watcher at 720 changes an hour, RADIUS progress at 401, Orb holds at 239 and integration logins at 120. Each new test was first shown failing on that older code.

**A hardware gate before release.** Ten minutes at idle, starting at least 90 minutes after boot, must stay under **4 KiB/s**, about 337 MiB a day. Boot gets its own allowance of 18.75 MiB. We set both limits from the daily budget *before* measuring, and did not move them after a candidate failed. Candidate 25 failed the gate twice on journal commits, and the limit stayed where it was.

**A Storage health card in the console** shows each partition's lifetime writes and recent rate, so a regression is visible on any running sensor. The next section explains what it reads.

## How a sensor reports its drive's health

Open **Device** in the sensor console and find **Storage health**. For each drive, it shows what the drive and the filesystem actually report, and nothing else. A value the drive does not report reads *Not reported by this drive*. It is never estimated.

| What it shows | Where it comes from | Which drives |
|---|---|---|
| Drive type, model and size | The kernel's block-device information | All |
| Wear: life used in 10% steps, and reserve-block status (normal, 80% used, 90% used) | The drive's own estimate, read from sysfs or `mmc extcsd` once an hour | eMMC only, such as the WLAN Pi Go's internal storage |
| Lifetime writes per partition | The ext4 filesystem's own counter, since the filesystem was created | All ext4 partitions |
| Writes per day | Measured from those counters over at least 10 minutes, ignoring the first 20 minutes after boot. That stretch writes tens of MB once and would read as wear that is not happening | All ext4 partitions |
| Writes since the last boot | The same counters, compared across a reboot only when both readings had a trusted clock | All ext4 partitions |
| Filesystem errors | ext4's error count | All ext4 partitions |
| Read-only after an error | The filesystem remounted itself read-only, the usual sign that storage is failing. Shown with a warning badge | All |

**SD cards and USB sticks report no wear at all.** Neither interface carries a life-used figure, so on a NanoPi, a Raspberry Pi or the Slate the card shows writes, errors and read-only state, but no percentage. Writes per day is the number to watch there. The console also has code to read SMART data from NVMe and SATA drives, but the image does not include `smartctl`, so those drives show *Not reported* for now.

The card follows the same write rules as everything else on this page. It reads sysfs at most once a minute and runs drive tools at most once an hour. Its state lives in RAM, and the only thing it saves is the lifetime counters in the existing "last alive" note.

The WLAN Commander phone app does not show drive health yet; it is in the sensor console only.

## What it means for the drive

The Slate's stick reports nothing about its own wear, so this is an estimate. Assume ordinary TLC or QLC flash rated for 150–1,000 write cycles, and that the drive's controller multiplies small, fsync-heavy writes 3–20 times. At 44 GB a day, the pessimistic end of that range wears the stick out in **under two weeks**. At 0.83 GB a day, its life runs to **decades**. The range is wide because the drive publishes no endurance rating; the counters and the absence of errors are the only firm facts.

## What our Orb integration cost {#what-orb-cost}

[Orb](https://orb.net/) is an optional connection-monitoring agent that the image can run. WLAN Commander wraps it in its own Python code, `wc_orb`, which pauses Orb during speed tests and keeps its settings in step with the console. **The first three costs below were in that wrapper, not in Orb**, which does not use Python. An earlier version of this page said otherwise; a reader caught it.

- **CPU, even with Orb off.** Every 15 seconds our sampler started a second Python process to reconcile the console's Orb settings: about 300 ms of CPU, plus about 70 ms for three `systemctl` calls that stopped and disabled an Orb that was already stopped. That was roughly 30% of each sampler run. The reconcile now runs inside the sampler's own process and acts only when the state actually differs.
- **Writes.** Speed tests and iPerf listeners pause Orb so it does not skew their results. Our wrapper stored each pause in its own settings file on Data, `/data/settings/orb/settings.json`, and refreshed it every 15 seconds: 239 rewrites an hour. Pauses now live in RAM; the file changes only when you change a setting.
- **A torn settings file.** On a real NanoPi card, that same wrapper file held 50 zero bytes, a write cut short by a power loss. On that card, a pre-release of the unified image failed a boot step on every start. The released image sets a torn file aside and falls back to defaults, without deleting anything.
- **Orb's own data, which we chose to keep on the card.** Two other cards, from earlier builds, had ext4 errors in Orb's history files on their old root partitions. Orb stores its measurement history on disk by default. It has an `ORB_EPHEMERAL_MODE` setting that keeps that data in RAM to protect flash, at the cost of losing it when Orb restarts. Orb's own OpenWrt sensor images, built with [sensorbox](https://github.com/orbforge/sensorbox), set it on every device. Our image does not. A power cut during one of those writes fits the damage, but we did not prove the cause.

Orb is now **off by default** on fresh installs, its home is on the persistent Data partition, and turning it on is a choice in **Network tests → Orb**. We have not yet measured how much Orb writes once it is running, or how much `ORB_EPHEMERAL_MODE` saves.

## Capturing straight to the SD card

A separate finding from the [thermal tests](/field-notes/nanopi-thermal-testing/): writing a full-rate capture to the NanoPi's own SD card stalled. With about 450 Mbps of traffic on the air, I/O pressure reached 98%, up to 141 MiB waited unwritten, and dumpcap reported 5,358 drops in 62 seconds. A short 32 MiB check had measured the card at 14.7 MB/s for writes, and 450 Mbps of full frames is about 56 MB/s. That gap fits the stall, but the cause was not isolated to the card, the controller or heat. For heavy captures, stream to the phone or a laptop rather than the sensor's SD card.

## What we have not measured

- The NanoPi Zero2's SD card on the current unified image, at idle or under load. The runtime fixes above are in that image, but the huge-page and access-time fixes are specific to the Slate, and we have not checked whether the NanoPi kernel needs either.
- Orb's own write rate while it is enabled and measuring, with and without `ORB_EPHEMERAL_MODE`.
- Any SD card's real wear. Like the Slate's stick, consumer cards report no wear or endurance figures, which is why the Storage health card shows writes per day for them instead.

Evidence: the Slate 7 Pro qualification log for candidates 22–35, the per-partition and per-process samples behind each table, and the write-budget harness in the sensor source, all recorded 19–21 October 2026.
