【发布时间】:2018-06-01 04:54:37
【问题描述】:
我在 PySpark 中有一个如下所示的数据框。
import pyspark.sql.functions as func
df = sqlContext.createDataFrame(
[(0.0, 0.2, 3.45631),
(0.4, 1.4, 2.82945),
(0.5, 1.9, 7.76261),
(0.6, 0.9, 2.76790),
(1.2, 1.0, 9.87984)],
["col1", "col2", "col3"])
df.show()
+----+----+-------+
|col1|col2| col3|
+----+----+-------+
| 0.0| 0.2|3.45631|
| 0.4| 1.4|2.82945|
| 0.5| 1.9|7.76261|
| 0.6| 0.9| 2.7679|
| 1.2| 1.0|9.87984|
+----+----+-------+
# round 'col3' in a new column:
df2 = df.withColumn("col4", func.round(df["col3"], 2))
df2.show()
+----+----+-------+----+
|col1|col2| col3|col4|
+----+----+-------+----+
| 0.0| 0.2|3.45631|3.46|
| 0.4| 1.4|2.82945|2.83|
| 0.5| 1.9|7.76261|7.76|
| 0.6| 0.9| 2.7679|2.77|
| 1.2| 1.0|9.87984|9.88|
+----+----+-------+----+
在上面的数据框中col4是double。现在我想将col4 转换为Integer
df2 = df.withColumn("col4", func.round(df["col3"], 2).cast('integer'))
+----+----+-------+----+
|col1|col2| col3|col4|
+----+----+-------+----+
| 0.0| 0.2|3.45631| 3|
| 0.4| 1.4|2.82945| 2|
| 0.5| 1.9|7.76261| 7|
| 0.6| 0.9| 2.7679| 2|
| 1.2| 1.0|9.87984| 9|
+----+----+-------+----+
但我想将 col4 值四舍五入到最接近的值
expected result
+----+----+-------+----+
|col1|col2| col3|col4|
+----+----+-------+----+
| 0.0| 0.2|3.45631| 3|
| 0.4| 1.4|2.82945| 3|
| 0.5| 1.9|7.76261| 8|
| 0.6| 0.9| 2.7679| 3|
| 1.2| 1.0|9.87984| 10|
+----+----+-------+----+
我该怎么做?
【问题讨论】:
标签: python apache-spark pyspark apache-spark-sql rounding