Initialising STORE_FILE stopped the oops and exposed what it had been hiding: twelve
concurrent puts all reported `ok` and one write survived. The lock guarded the file
handle, not the store — it is taken, the handle is fetched or cached, and released
before the caller does anything with it. Sharing a struct file * is safe, so that is
all the handle needs, and it is not enough for the store.
Widening it inside append is also not enough, and this commit exists because the first
attempt did exactly that and changed nothing. append receives the layout as an argument,
and the caller read it from device_and_layout() *before* calling: twelve writers each
read log_used = 0, then queued on a lock inside append, then each appended at the same
offset against the layout they had already read. Measured with the lock in append: one
68-byte entry, twelve `ok`s. The lock has to be held from before the read.
So STORE_OP is taken by the four write ops — put, del, sync and the device write — and
everything under them runs with it held: ensure_boot_record, device_and_layout, append,
fold_now. None of those may take it again; a kernel mutex is not reentrant, and append
folds and then calls itself, while fold_now is reachable both from inside append and on
its own. That is why the lock sits at the ops rather than in the writers.
verify-file-store.sh MODE=race, twelve concurrent puts: 816 bytes of log = 12 x 68, and
control generation 13 = 1 + 12, where the previous kernel produced 68 and 2.
Regression: MODE=seq still exact; verify-boot-record passes (it is the path this
changes most, since ensure_boot_record now runs under the lock); verify-kernel-append
passes, four acknowledged writes surviving a SIGKILL with no shutdown.
CUBELinux
e53ae04033