📎Model Metrics

The following model metrics are computed during the inference time. These metrics can be monitored at the sensor level and on aggregation level.

• Class Wise Probability Distribution: This metric tracks the probability assigned to each class during inference. Monitoring this at the sensor level ensures that class probabilities align with expected behavior, while aggregated monitoring can detect shifts in overall model confidence over time.

• Embeddings Distribution: Embeddings represent the high-dimensional feature space used by the model. Tracking the distribution of embeddings at both the sensor and aggregate level helps monitor changes in how features are represented, potentially signaling data or model drift.

• Class Wise Score Distribution: This metric monitors the confidence scores for each class across predictions. Monitoring it per sensor ensures individual predictions align with expected outcomes, while aggregate monitoring helps detect changes in the model's overall confidence across the entire dataset.

• Class Frequency Distribution: This tracks how frequently each class appears in predictions. At the sensor level, it can detect anomalies or bias toward certain classes. At the aggregate level, monitoring this helps maintain a balanced dataset and track concept drift.