com.datarobot.mlops_spark_utils

Class MLOpsReporting

java.lang.Object

com.datarobot.mlops_spark_utils.MLOpsReporting

public class MLOpsReporting
extends Object

Constructor Summary

Constructors

Constructor and Description
MLOpsReporting()

Method Summary

Modifier and Type	Method and Description
static void	reportActuals(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, String deploymentID, String modelId, String channelConfig) Report actuals/outcomes of predictions.
static void	reportAggregatedStats(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, Double scoringTime, List<String> targetColumnNames, Integer predictionsServiced, String deploymentID, String modelId, String channelConfig, Map<String,com.datarobot.mlops.stats.FeatureType> featureTypes, Integer distinctCategoryCount, Integer histogramBinCount, Integer aggregationMaxRecords, Integer groupSize) Report predictions statistics utility function for a Spark environment.
static void	reportPredictions(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, String deploymentID, String modelId, String channelConfig, Double scoringTime, List<String> targetColumnNames, String assocIdColName, Integer predictionsServiced, int groupSize) Report predictions utility function for a Spark environment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MLOpsReporting

public MLOpsReporting()

Method Detail

reportPredictions

public static void reportPredictions(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
                                     String deploymentID,
                                     String modelId,
                                     String channelConfig,
                                     Double scoringTime,
                                     List<String> targetColumnNames,
                                     String assocIdColName,
                                     Integer predictionsServiced,
                                     int groupSize)
                              throws com.datarobot.mlops.common.exceptions.DRCommonException

Report predictions utility function for a Spark environment. This function receives a dataframe (among other parameters) and will report the features, predictions and association ids from this single dataframe. This function supports reporting predictions for both Regression models and classifications models.

Parameters:

dataFrame - The Dataframe containing the data to report

deploymentID - Deployment id to report data to

modelId - Model this reporting is associated with

channelConfig - Channel configuration string, to configure how to send the prediction information to the MLOps Agent.

scoringTime - Time in milliseconds it took to perform scoring

targetColumnNames - A list of column names containing the prediction value. If only one column is provided then the model is assumed to be a regression model. If the number of columns is greater than 1 the model is assumed to be a classification model.

assocIdColName - Optional column name containing the association ids to report with the predictions.

Throws:

com.datarobot.mlops.common.exceptions.DRCommonException

reportAggregatedStats

public static void reportAggregatedStats(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
                                         Double scoringTime,
                                         List<String> targetColumnNames,
                                         Integer predictionsServiced,
                                         String deploymentID,
                                         String modelId,
                                         String channelConfig,
                                         Map<String,com.datarobot.mlops.stats.FeatureType> featureTypes,
                                         Integer distinctCategoryCount,
                                         Integer histogramBinCount,
                                         Integer aggregationMaxRecords,
                                         Integer groupSize)
                                  throws com.datarobot.mlops.common.exceptions.DRCommonException

Report predictions statistics utility function for a Spark environment. This function receives a dataframe (among other parameters) and will report the aggregated statistics about features and predictions from this single dataframe. This function supports reporting prediction statistics for both Regression models and classifications models.

Parameters:

dataFrame - The Dataframe containing the data to calculate and report statistics on

deploymentID - Deployment id to report statistics to

modelId - Model this reporting is associated with

channelConfig - Channel configuration string, to configure how to send the prediction information to the MLOps Agent.

scoringTime - Time in milliseconds it took to perform scoring

targetColumnNames - A list of column names containing the prediction value. If only one column is provided then the model is assumed to be a regression model. If the number of columns is greater than 1 the model is assumed to be a classification model.

Throws:

com.datarobot.mlops.common.exceptions.DRCommonException

reportActuals

public static void reportActuals(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
                                 String deploymentID,
                                 String modelId,
                                 String channelConfig)
                          throws com.datarobot.mlops.common.exceptions.DRCommonException

Report actuals/outcomes of predictions.

Parameters:

dataFrame - The dataframe containing all necessary data to report actuals

deploymentID - Deployment id to report statistics to

modelId - Model this reporting is associated with

channelConfig - Channel configuration string, to configure how to send the prediction information to the MLOps Agent.

Throws:

com.datarobot.mlops.common.exceptions.DRCommonException