meta: stop mutating the published format, publish it only once stored

[solracsf]

Follow-up to #7378, which stored baseMeta.fmt as an atomic.Pointer. That guards the pointer, not the struct it points at, and several call sites still wrote into the format every reader shares.

The reload path published too early: Load published the format before refresh handed it to the reload callbacks, and the mount's callback writes the command line overrides (Bucket, Storage, Tiers, UploadLimit) into it on every heartbeat. The parsing is split out of Load as loadFormat, and reloadFormat runs the callbacks while the format is still private, publishing it once they are done. The window where readers saw the stored values before the overrides were applied is gone too.

Status (pkg/meta/status.go) called Load, which publishes the format, and then RemoveSecret on that same pointer, replacing SecretKey, SessionToken and EncryptKey with removed for every reader of the volume. From a long lived client (the Java SDK calls Status on a mounted session) a later quota set would store the placeholders in the volume setting for good. Publishing there also dropped the patches the reload callbacks had applied, and because the next reload then finds the stored format unchanged and skips the callbacks, the client kept running without its own overrides until the stored format changed. It now reads through loadFormat, which does not publish, and scrubs a copy, so only the reported Sections carries the placeholders.

handleQuotaSet (pkg/meta/quota.go) turned DirStats or UserGroupQuota on in the format it had just read from getFormat(), then passed that same pointer on to be stored. On a mount or an SDK client that format is the one the reload callbacks have patched, so the bucket and the limits of a single client went into the volume setting for every other client. Two such calls also raced: each started from the same format, so whichever stored last dropped the flag the other had set, and both reported success. It now loads the stored format, sets the flag on that, and holds a lock across the load and the store.

GetFormat returns the format by value, which reads as a private copy, but Tiers is a map so the copy keeps pointing at the live one. Reading .config on a mount takes such a copy and hands it to the mount's patcher, which writes Tiers[0] straight back into the map the published format owns, while the heartbeat walks it in reflect.DeepEqual. That is a concurrent map read and write, which takes the process down rather than just racing. Format.Clone copies the map, and GetFormat hands that out.

Three related changes fell out of this.

handleQuotaSet only warned when the doInit enabling those flags failed, and created the quota anyway. Since the flag is no longer forced on locally, updateDirQuota sees it disabled and returns immediately, so the quota exists but is never accounted or enforced while the command reports success. It returns the error now, carrying EIO, and fs.FileSystem.HandleQuota passes that errno through instead of turning everything it does not recognise into EINVAL, so a metadata store that was briefly unreachable no longer reaches a Java caller as an invalid request.

Publishing moved out of the engines into Init. Each engine picked its own moment: redis between storing the setting and creating the root inode of a fresh volume, the SQL and KV engines before the transaction that stores it. A caller that got an error back could therefore be left running on a format the volume never received, and whether that happened depended on the engine. Init publishes once doInit reports success, so the engines only store and a fourth one cannot get the ordering wrong.

ErrNotFormatted replaces the message prefix that three places matched on with a string compare. Rewording that message would have quietly turned the reload's os.Exit(UmountCode) into a warning, leaving a mount running against a volume that no longer exists, with nothing failing to build.

Tests

TestPublishedFormatIsNotMutated covers the invariant in seven subtests (quota set, quota set keeps the local overrides local, tiers are copied, failed init, failed init (sql), status, reload callbacks), and TestQuotaSetFailsWhenFormatUpdateFails covers the error propagation. Each case was run against the code without its fix first and fails there, for example:

base_test.go:4533: GetFormat shares Tiers with the published format: Sc="patched"
base_test.go:4517: handleQuotaSet stored this client's override in the volume
base_test.go:4612: Status republished the stored format over the live one: Bucket=""
base_test.go:4661: handleQuotaSet did not report the failed store as EIO

The KV failed init case injects the failure through a tkvClient wrapper that fails txn but not simpleTxn, so doInit gets past its initial read and fails exactly where the setting would be stored. It changes Capacity rather than a flag, because enabling DirStats or UserGroupQuota runs a cleanup transaction that would fail earlier and hide the case under test. The SQL case gets the same reads pass, writes fail split from a read only client, since dbMeta.txn refuses to run while the initial read still goes through simpleTxn. That case fails with the setFormat call sql.go used to have before its transaction.