Spark 版本 2.0.0 的既定目标是在 ml 和现已弃用的 mllib 软件包之间实现功能对等。
不过，目前ml 包提供 ElasticNet 支持，但仅支持二元回归。要获得多项式显然我们必须接受使用已弃用的 mllib？
使用 mllib 的缺点：
ml 工作流，因此它不能干净地集成有没有一种方法可以使用ml 包实现一对多多项式？
【问题讨论】：
标签： apache-spark machine-learning
这是一个正在进行的答案。在spark.ml 中有一个OneVsRest 分类器。
显然，该方法是将LogisticRegressionClassifier 作为二进制分类器提供给它 - 这将在所有类中运行二进制版本并返回得分最高的类。
更新以响应@zero323。以下是来自 Xiangrui Meng 的关于 mllib 弃用的信息：
在 Spark 2.0 中将基于 RDD 的 MLlib API 切换到维护模式
Hi all,
More than a year ago, in Spark 1.2 we introduced the ML pipeline API built on top of Spark SQL’s DataFrames. Since then the new DataFrame-based API has been developed under the spark.ml package, while the old RDD-based API has been developed in parallel under the spark.mllib package. While it was easier to implement and experiment with new APIs under a new package, it became harder and harder to maintain as both packages grew bigger and bigger. And new users are often confused by having two sets of APIs with overlapped functions.
We started to recommend the DataFrame-based API over the RDD-based API in Spark 1.5 for its versatility and flexibility, and we saw the development and the usage gradually shifting to the DataFrame-based API. Just counting the lines of Scala code, from 1.5 to the current master we added ~10000 lines to the DataFrame-based API while ~700 to the RDD-based API. So, to gather more resources on the development of the DataFrame-based API and to help users migrate over sooner, I want to propose switching RDD-based MLlib APIs to maintenance mode in Spark 2.0. What does it mean exactly?
* We do not accept new features in the RDD-based spark.mllib package, unless they block implementing new features in the DataFrame-based spark.ml package.
* We still accept bug fixes in the RDD-based API.
* We will add more features to the DataFrame-based API in the 2.x series to reach feature parity with the RDD-based API.
* Once we reach feature parity (possibly in Spark 2.2), we will deprecate the RDD-based API.
* We will remove the RDD-based API from the main Spark repo in Spark 3.0.
Though the RDD-based API is already in de facto maintenance mode, this announcement will make it clear and hence important to both MLlib developers and users. So we’d greatly appreciate your feedback!
(As a side note, people sometimes use “Spark ML” to refer to the DataFrame-based API or even the entire MLlib component. This also causes confusion. To be clear, “Spark ML” is not an official name and there are no plans to rename MLlib to “Spark ML” at this time.)
Best,
Xiangrui
另一个更新为此有一个 JIRA，截至 2016 年 5 月，该工作已接近完成Support multiclass logistic regression in spark.ml
【讨论】：
ml.LogisticRegression，因为我们有一个自定义优化器。 ml 逻辑以breeze.linalg.LBFGS 或OWLQN 形式传入（即private） - 鉴别器为(elasticnet==0 or regparam ==0)。
mllib 被弃用的信息来源是什么？我已经看到一些单独的类（不同的回归变体）被标记为已弃用，但仅此而已。
mllib 的弃用发生在一两个版本之前。 2.0.0 的既定目标是实现功能对等 - 以便弃用不再是 catch-22。就信息的“来源”而言，它已经出现在开发邮件列表中，我相信博客也是如此。我可以查一下。
mllib deprecation 的信息添加到 WIP 答案中。
DataFrame 以支持 DataSet。我想这让我们保持警惕。