Skip to content
mofcloud

Monitoring

Last updated:

MOF exposes Prometheus metrics: processes, whether bills and resources are syncing, background jobs, the license, HTTP and MCP traffic. Add Prometheus and Grafana next to it, or point your existing ones at it.

Requires
MOF v2.0.14+
Metrics endpoint
http://mof:8080/internal/v1/mof/metrics
Exposure
Internal only: Dash does not forward this path.

Docker Compose

Adds Prometheus and Grafana next to MOF. Prometheus comes with MOF’s alert rules, Grafana with Prometheus as its data source.

Download

Put both next to your compose.yaml.

Start

Terminal
docker compose -f compose.yaml -f compose.monitoring.yaml up -d

Import the dashboard

  1. Open http://SERVER_IP:3000, sign in as admin / admin and set a new password.
  2. Dashboards → New → Import → Upload dashboard JSON file, and choose mof-grafana-dashboard.json.
  • Allow port 3000 only from your own network.
  • About 300 MB more memory; Prometheus keeps 15 days of data. Images: prom/prometheus:v2.53.1, grafana/grafana:11.1.0; Docker Compose v2.23+.
  • Pass both files on later commands too (pull, up -d), or the monitoring services are left out.

Existing Prometheus and Grafana

FileUse
prometheus.yml ↓Scrape job for MOF. Prometheus must reach MOF’s port 8080 over your private network.
mof-alerts.yml ↓Alert rules; add it under rule_files.
mof-grafana-dashboard.json ↓Import in Grafana and choose your Prometheus data source.

With several web processes, each reports the deployment-wide values (processes, syncs, jobs); the dashboard and rules take max(), so scraping all of them is fine. Workers serve no HTTP; their work shows up through the web processes.

Kubernetes

The MOF Pods carry the usual prometheus.io annotations (scrape, port 8080, path), so a Prometheus that discovers Pods by annotation picks them up. With the Prometheus Operator, add a PodMonitor:

podmonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: mof
  namespace: mof
spec:
  selector:
    matchLabels:
      app: mof
  podMetricsEndpoints:
    - port: api
      path: /internal/v1/mof/metrics
      interval: 30s

Metrics

MetricLabelsMeaning
mof_processesmodeProcesses seen in the last 90 s, by mode (all-in-one, web, worker).
mof_account_sync_last_success_timestamp_secondsaccount, provider, dataWhen bills / resources last synced successfully. A SyncAll counts for both.
mof_account_sync_failedaccount, provider, data1 when the latest sync of that data failed.
mof_jobskind, stateBackground jobs by kind and state (available, running, retryable, discarded…).
mof_license_expiry_timestamp_seconds / mof_license_validWhen the license ends, and whether it is valid.
mof_mcp_requests_totalmethod, resultCalls to the remote MCP gateway.
mof_metrics_database_up0 when MOF could not read its database.
mof_http_requests_totalmethod, route, codeAPI requests. route is the route template (/v1/mof/prov/account/:accountId/bills), so IDs and probing scanners add no series; unknown paths count as unmatched.
mof_http_request_duration_secondsmethod, routeAPI latency (histogram, adds up across processes).
mof_users / mof_accountsstate / providerUsers (active, disabled) and cloud accounts by provider.
mof_database_size_bytesdatabaseSize of each database MOF manages: Mof and the providers’ bill databases, which grow with accounts and months.
go_sql_*, go_memstats_*, go_goroutinesDatabase connection pool and Go runtime.

Alerts

In mof-alerts.yml. Prometheus evaluates them; route them with your Alertmanager, or view them in Grafana → Alerting.

AlertWhen
MofDownMOF stopped answering scrapes for 2 minutes.
MofDatabaseUnreachableMOF cannot read its database.
MofAccountSyncFailingAn account’s bills or resources keep failing to sync (30 minutes).
MofAccountSyncStaleNo successful sync of an account’s bills or resources for 36 hours.
MofJobsDiscardedBackground jobs failed for good in the last hour.
MofJobBacklogMore than 50 jobs waiting for 30 minutes: add workers or look for a stuck one.
MofHighErrorRateMore than 5% of API requests fail with 5xx for 10 minutes.
MofLicenseExpiringThe license ends within 14 days.