java.lang.IllegalStateException：未找到集群。检查您的 -c 路径答案

【问题标题】：java.lang.IllegalStateException: No clusters found. Check your -c pathjava.lang.IllegalStateException：未找到集群。检查您的 -c 路径
【发布时间】：2013-02-22 15:07:36
【问题描述】：

我在命令行中使用以下命令使用 Mahout kmeans 算法对数据进行聚类

mahout kmeans -i /vect_out/tfidf-vectors/ -c /out_canopy -o /out_kmeans -dm   
org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure -cd 1.0 -x 20 -cl

其中 /out_canopy 是包含使用 Mahout canopy 集群创建的集群的目录，其中包含一个 clusters-0 目录，该目录本身包含一个名为 _logs 的目录和一个名为 part-r-00000 的文件

但是一直报如下错误

java.lang.IllegalStateException: No clusters found. Check your -c path.
at org.apache.mahout.clustering.kmeans.KMeansMapper.setup

【问题讨论】：

标签： hadoop machine-learning cluster-computing cluster-analysis mahout

【解决方案1】：

你确定/out_canopy 是目录吗？你试过了吗：

file /out_canopy

似乎有一个错字，你只想写 out_canopy 或类似的东西......

【讨论】：

out_canopy 确实似乎是一个目录。这就是hadoop fs -ls / 给drwxr-xr-x - rupinder supergroup 0 2013-03-07 17:11 /out_canopy

【解决方案2】：

这是一个特别令人头疼的问题。

1. Swallow IllegalStateExceptions thrown by removeShutdownHook in FileSystem. The javadoc states:

    public boolean removeShutdownHook(Thread hook)
    Throws:
    IllegalStateException - If the virtual machine is already in the process of shutting down 

So if we are getting this exception, it MEANS we are already in the process of shutdown, so we CANNOT, try what we may, removeShutdownHook. If Runtime had a method Runtime.isShutdownInProgress(), we could have checked for it before the removeShutdownHook call. As it stands, there is no such method. In my opinion, this would be a good patch regardless of the needs for this JIRA.

2. Not send SIGTERMs from the NM to the MR-AM in the first place. Rather we should expose a mechanism for the NM to politely tell the AM its no longer needed and should shutdown asap. Even after this, if an admin were to kill the MRAppMaster with a SIGTERM, the JobHistory would be lost defeating the purpose of 3614

【讨论】：