Model Production Monitoring Dashboard¶
The MPM Dashboard allows you to create your own visualizations using Python panels.
From here, you can control:
- View - Create a customized view of collection of panels
- Filters - Add any number of filters to add a column in MPM SDK methods
- Selections - Select Version, Daily or Hourly, and date range like on the Model performance tab
By default, every Python panel will use the values of Filters and Selections if specific values are not set in code.
Here is a sample Python panel script using the comet_mpm Python package:
from comet_mpm import API, ui
import plotly.graph_objects as go
import pandas as pd
# Create API to query MPM data
api = API()
model = api.get_model()
# Get number of predictions
df = model.get_nb_predictions()
#print(df)
# Plot number of predictions - Filters and date ranges will
# be automatically updated
fig = go.Figure()
for column in df.columns: # filters
fig.add_trace(
go.Scatter(
x=pd.to_datetime(df.index), # timestamp
y=df[column],
mode="lines",
name=column,
)
)
ui.display(fig)
Model methods¶
To get the model for the current view, use:
from comet_mpm import API
api = API()
model = api.get_model()
With a model instance, you can use the following methods. Each returns a Pandas DataFrame, with each filter representing a column:
- Model.get_nb_predictions() - Get the number of predictions
- Model.get_custom_metric(sql=...) - Get a custom metric using SQL
- Model.get_feature_drift(feature_name=...) - Get a features drift
- Model.get_feature_category_distribution(feature_name=...) - Get feature category distribution
- Model.get_feature_density(feature_name=...) - Get the probability density function (PDF) of a numeric feature
- Model.get_feature_percentiles(feature_name=...) - Get the specified percentiles for a numeric feature
In addition, you can also get a list of the numerical and categorical features:
- Model.get_numerical_features() - Get the list of numerical features available for this model
- Model.get_categorical_features() - Get the list of categorical features available for this model
Note that when you log a feature named, say, "age", then the feature_name will be "feature_age". In other words, the input name is prefixed with "feature_". This is true for all of the above methods using feature_name. Likewise, the output names are prefixed with "prediction_". You can use these prefixed names when writing the SQL of custom metrics. See below for more information on custom metric SQL.