Skip to main content

Scorecards

A scorecard rolls up the quality and reliability signals for a Vibe (or a catalog/component) into a single 0–100 score. Open a Vibe and select the Scorecard tab to see it.

Each scorecard is a set of metrics. Every metric has a value, a target threshold, and a direction — and from those, its status (passing / warning / failing) is worked out automatically. You never have to hand-set a status to a number that doesn't match it.

What you'll learn​

  • The three ways to add metrics
  • How a metric is measured and scored
  • How to read the overall score and trends
  • How to push metrics automatically from CI

Three ways to populate a scorecard​

When a scorecard is empty, the tab offers three options:

  1. Apply a template — start from a standard set of checks defined for your workspace (for example Service Production-Readiness or AI Agent Quality). Applying a template creates its metrics on the scorecard in one click. Templates only define the targets — you (or CI) fill in the measured values.
  2. Add a metric — define a single metric by hand.
  3. Push from CI — keep metrics fresh automatically from your pipeline.

Adding a metric by hand​

Select Add metric and fill in:

FieldMeaning
NameWhat you're measuring, e.g. Test coverage. Unique within the scorecard.
TypeA classification (Test Coverage, SLO Compliance, Incident Count, …) or Custom.
DirectionWhether higher is better (coverage, SLO) or lower is better (incidents, latency).
Value / Threshold / UnitThe current measurement, its target, and the unit (%, ms, count).
WeightHow much this metric counts toward the overall score (default 1). Give critical metrics a higher weight.
StatusLeave on Auto to derive it from the numbers, or pin it manually.
RemediationGuidance shown to viewers when the metric is warning or failing.
Embed URLOptional dashboard (e.g. Grafana) embedded when the metric card is expanded.

The dialog shows a live Derived / Pinned status preview as you type.

How metrics are measured​

Score (0–100) — each metric is scored against its threshold:

  • Higher is better: value / threshold × 100 (e.g. coverage 86 vs target 80 → 100, capped).
  • Lower is better: threshold / value × 100 (e.g. 1 incident vs target 0–2 → high score; 10 incidents → low score).

Both are capped at 100, so meeting or beating the threshold is a perfect 100.

Status — derived from the score:

  • Passing — meets or beats the threshold.
  • Warning — within 10% of the threshold.
  • Failing — below that.

You can override a status manually, but the default (Auto) keeps it honest.

Overall score — the weighted average of every metric's score. Raising a metric's weight makes it count for more.

Reading the scorecard​

  • The overall score sits at the top, color-coded (green ≥ 80, amber ≥ 50, red below). Hover the ℹ️ for the formula.
  • Each metric card shows its value vs threshold, a progress bar, the direction (↑/↓), its weight, and a status icon. Failing/warning metrics show their remediation guidance inline.
  • Expand a card to see its score trend over recent measurements and any embedded dashboard.
  • Compare across the workspace at the bottom lists other scorecards, lowest score first, so the entities that need attention surface to the top.

Pushing metrics from CI​

Keep metrics current without anyone touching the UI. In the empty state (or your workspace settings) generate a push token — it's shown once, so copy it into your CI secrets. Then POST your measurements from the pipeline:

curl -X POST "$WORKSPACES_API_BASE/api/scorecard/push" \
  -H "Authorization: Bearer $SCORECARD_PUSH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "workspaceId": "<WORKSPACE_ID>",
    "ownerType": "VIBE",
    "ownerId": "<VIBE_ID>",
    "metrics": [
      { "name": "Test coverage", "type": "TEST_COVERAGE", "value": 86, "threshold": 80, "unit": "%", "direction": "HIGHER_IS_BETTER" },
      { "name": "Open incidents", "type": "INCIDENT_COUNT", "value": 0, "threshold": 0, "unit": "count", "direction": "LOWER_IS_BETTER" }
    ]
  }'

Notes:

  • The token is per workspace and validated against a stored hash — the raw value is never retrievable again, so regenerate it if it leaks.
  • Status is derived from value vs threshold; only send an explicit status to pin one.
  • Every pushed value adds a history point, which is what powers the trends.