SageMaker ModelExplainabilityMonitor 基线作业给出错误 ValueError: Expected 2D array, got 1D array instead:答案

【问题标题】：SageMaker ModelExplainabilityMonitor baseline job gives error ValueError: Expected 2D array, got 1D array instead:SageMaker ModelExplainabilityMonitor 基线作业给出错误 ValueError: Expected 2D array, got 1D array instead:
【发布时间】：2022-11-29 22:47:48
【问题描述】：

我正在尝试为我的一个 ML 模型创建一个 SageMaker ModelExplainabilityMonitor。为了运行 ModelExplaniabilityMonitor 的基线，suggest_baseline() 方法需要 DataConfig、ModelConfig 和 ShapConfig。在 ShapConfig 中，我需要提供我正在计算的 SHAP 基线，方法是按照here 建议的特征进行计算。问题是当我运行方法 suggest_baseline() 时，它会启动 SageMaker 处理作业来创建影子端点，但它会给出端点重试错误，如下所示：

ClientError: An error occurred (ModelError) when calling the InvokeEndpoint operation (reached max retries: 0): Received server error (500) from primary with message "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500 Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p> ". See https://eu-west-2.console.aws.amazon.com/cloudwatch/home?region=eu-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/sm-clarify-pipelines-xqkqe9pekm5f-MACEModel-2Al-1669644628-2315 in account 450538937006 for more information.

当我检查基线作业创建的影子端点的云监视日志时，它显示了服务器超时的原因，如下所示：

ERROR - random_forest_training - Exception on /invocations [POST]

Traceback (most recent call last):
  File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper
    return fn(*args, **kwargs)
  File "/opt/ml/code/random_forest_training.py", line 38, in predict_fn
    prediction = model[0].predict_proba(input_data)
  File "/miniconda3/lib/python3.7/site-packages/sklearn/ensemble/_forest.py", line 673, in predict_proba
    X = self._validate_X_predict(X)
  File "/miniconda3/lib/python3.7/site-packages/sklearn/ensemble/_forest.py", line 421, in _validate_X_predict
    return self.estimators_[0]._validate_X_predict(X, check_input=True)
  File "/miniconda3/lib/python3.7/site-packages/sklearn/tree/_classes.py", line 388, in _validate_X_predict
    X = check_array(X, dtype=DTYPE, accept_sparse="csr")
  File "/miniconda3/lib/python3.7/site-packages/sklearn/utils/validation.py", line 72, in inner_f
    return f(**kwargs)
  File "/miniconda3/lib/python3.7/site-packages/sklearn/utils/validation.py", line 623, in check_array
    "if it contains a single sample.".format(array))

ValueError: Expected 2D array, got 1D array instead: array=[-0.07272727 -0.538843    0.21109799 -0.11960932  0.23030303 -0.09173553
 -0.17808585 -0.19966942 -0.06921487  0.01707989  0.          0.
 -0.02214876 -0.17888805  0.00661157 -0.04977043  0.01818182  0.15619835
  0.39504132 -0.05785124  0.01157025].

Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

它在 2D 中期望的数组是我的 shap 基线，我通过取特征的平均值来计算它。问题是我已经在发送 2D 基线，即 List[List[float]]。但是当我尝试将它包装在另一个列表中时，即 List[List[List[float]]] 基线作业的模式验证失败，因为它期望基线采用以下格式：

str（S3 对象的 URI，即具有形状值的 CSV 文件）。
List[List[float | int]]
List[Dict[name_of_column: shap_value_for_column]]
我已经尝试了所有这三种方法，但每种方法都会产生相同的错误。除此之外，我无法找到一种方法来转换这些形状基线。

任何帮助表示赞赏。

【问题讨论】：

标签： amazon-web-services machine-learning amazon-sagemaker shap amazon-sagemaker-clarify

【解决方案1】：

我假设你发送的是功能，所以你可以在发送之前申请 array.reshape(-1, 1)

也许这个例子可以让你深思:=)

x = np.array([2.0 , 2.4, 1.5])
y = np.array([196, 221, 136])

lr = LinearRegression()
lr.fit(x.reshape(-1, 1), y)

print(lr.predict([[2.4]]))

【讨论】：

与错误无关