Skip to content

Feature proposal: codecarbon doctor to report measurement quality #1350

Description

@davidberenstein1957

Feature proposal.

The problem

CodeCarbon always produces a number, but it never tells the user how that number was obtained. On a Linux box where /sys/class/powercap/intel-rapl is root-only, the CPU silently falls back to a TDP-times-load model. On an unrecognised CPU model the TDP itself is a generic constant. On macOS without sudo for powermetrics it falls back again. RAM is always modelled, never measured. In every one of those cases the tracker keeps going and writes a confident float to emissions.csv.

The information exists — ResourceTracker picks a backend and logs it, CPU._mode records the mode, CPU._is_generic_tdp records whether the TDP was a real match — but it is only visible at INFO/DEBUG log level, mixed into startup noise, and codecarbon detect (which is where users look) prints a correct hardware list and says nothing about quality. Users find out their measurement was mostly an estimate later, usually from a reviewer.

This is a disclosure problem, not a measurement problem: the fallbacks are reasonable, they are just invisible.

Proposed design

A new CLI command:

codecarbon doctor [--json] [--strict]

It runs the existing hardware detection (EmissionsTracker(save_to_file=False).get_detected_hardware() path) and, for each power component, prints a status, the method, why a better method was not used, and the concrete fix:

CPU     12th Gen Intel(R) Core(TM) i7-1260P
        ESTIMATED - CPU load model over a 28 W TDP
        Why: /sys/class/powercap/intel-rapl exists but its energy counters are
             not readable by this user (permission denied)
        Fix: sudo chmod -R a+r /sys/class/powercap/intel-rapl
             https://docs.codecarbon.io/how-to/enable-rapl/

--json gives a machine-readable version for CI gates and bug reports, --strict exits non-zero when any component is estimated.

The public shape is one dataclass and one function in codecarbon/diagnostics.py:

@dataclass
class ComponentDiagnostic:
    component: str   # "CPU" | "RAM" | "GPU"
    detail: str
    status: str      # "measured" | "estimated" | "unavailable"
    method: str
    reason: str | None
    fix: str | None

def diagnose(hardware) -> list[ComponentDiagnostic]: ...

Why it fits existing extension points

No new measurement code and no new dependency. ResourceTracker already centralises the "which backend won" decision, CPU._mode and CPU._is_generic_tdp already encode the answer, and cpu.is_rapl_available() / powermetrics.is_powermetrics_available() / windows_emi.is_emi_available() already run the checks whose failure reason we want to surface. diagnose() reads the hardware objects the tracker builds; it does not duplicate detection. typer and rich are already required.

Scope boundary

First PR: diagnostics.py, the doctor command, docs. Deliberately out of scope, because they change behaviour for every existing user and deserve their own review:

  • a start-of-run warning from EmissionsTracker.__init__ when a component is estimated;
  • a measurement_quality provenance field on EmissionsData / the CSV;
  • per-backend why_unavailable() methods on each of rapl.py, powermetrics.py, windows_emi.py, gpu_*.py — the first version derives the reason from the existing availability checks instead, which covers the common Linux/macOS cases without touching six modules.

Also explicitly not in scope: doctor running privileged fix commands itself. It prints them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions