计算具有多列的行的等级 - scala

Question

我想根据金额字段对一组列进行排名。对于组合字段（Nbr，TypeCd）的相同值，rank应该相同。这需要在 scala 中，spark 和 scala 版本分别为 1.3 和 2.10。需要的代码是scala中需要的。

示例输入数据：

Id  Nbr    TypeCd   Amt
a1  2010    online  25
a2  2010    online  25
a3  2010    online  15
a4  2011    store   100
a5  2011    store   50
a4  2011    store   90
a6  2011    store   90
a7  2011    store   90

样本输出数据

Id      Nbr    TypeCd   Amt Rank
a1  2010       online   25  1
a2  2010       online   25  1
a3  2010       online   15  3
a4  2011       store    100 1
a5  2011       store    50  5
a4  2011       store    90  2
a6  2011       store    90  2
a7  2011       store    90  2

Answer 1

这不是最佳答案，但考虑到您的限制，我会这样做：

// Group by Nbr & TypeCd
val groupedData = input.map(x => ((x._2, x._3), x)).groupByKey().map(x => x._2.toList)

// Within each subgroup sort based on the Amount in descending order
val sortedSubGroups = groupedData.map(x => x.sortWith(_._4 > _._4))

// Now assign ranks in each subgroups
val rankAssignedList = sortedSubGroups.map(x => assignRank(x)).flatMap(x => x)

其中assignRank，只是遍历子组中所有元素的列表，并按如下方式分配排名：

def assignRank(inputList : List[(String, Int, String, Int)]) : List[(String, Int, String, Int, Int)] = 
{
    var rank = 1;
    var prevAmt = 0;
    var elementPosition = 1;
    var outputList = List[(String, Int, String, Int, Int)]();
    for(element <-  inputList)
    {
        // Check if the Current Amount is lesser than Previous one
        if(element._4 != prevAmt)
        {
            rank = elementPosition
        }

        outputList = outputList ::: List((element._1, element._2, element._3, element._4 , rank))
        prevAmt = element._4
        elementPosition = elementPosition + 1
    }

    return outputList
}