将 SQL Case 语句转换为 Spark

Question

如何将此 SQL 案例语句转换为 Spark SQL？

 replace_old_engagements_sql = """ UPDATE """ + my_table_name + """
                            SET Engagement = CASE Engagement
                                                WHEN '800000026680' THEN '800000032764'
                                                WHEN '807000000041' THEN '808000000000'
                                                WHEN '870000012569' THEN '807000000412'
                                                WHEN '807000000279' THEN '808000000223'
                                                WHEN '807000000282' THEN '808000000223'
                                                WHEN '870000000403' THEN '808000000223'
                                            END
                            WHERE LinkedAccountId in ('123456789101','109876543212') AND Engagement IN ('800000026680', '807000000041', '870000012569', '807000000279', '807000000282', '870000000403'); """

Answer 1

我猜你的 spark sql 会接近这个。

    spark.sql("""
INSERT OVERWRITE TABLE db.my_table_name
SELECT
CASE 
  WHEN LinkedAccountId in ('123456789101','109876543212') THEN
    CASE
    WHEN Engagement = '800000026680' THEN '800000032764'
    WHEN Engagement = '807000000041' THEN '808000000000'
    WHEN Engagement = '870000012569' THEN '807000000412'
    WHEN Engagement = '807000000279' THEN '808000000223'
    WHEN Engagement = '807000000282' THEN '808000000223'
    WHEN Engagement = '870000000403' THEN '808000000223'
    ELSE Engagement
    END
  ELSE Engagement
END as Engagement
from db.my_table_name
""")

Answer 2

# I hope this could be answer for your query in dataframe style which is creating new dataframe with the modified values as per the conditions specified

val out_df = input_df.filter($"LinkedAccountId".isin('123456789101','109876543212') and $"Engagement".isin('800000026680', '807000000041', '870000012569', '807000000279', '807000000282', '870000000403')).withColumn("Engagement",when($"Engagement" === '800000026680', '800000032764').when($"Engagement" === '807000000041','808000000000').when($"Engagement" === '870000012569', '807000000412').when($"Engagement" === '807000000279', '808000000223').when($"Engagement" === '807000000282', '808000000223').when($"Engagement" === '870000000403', '808000000223').otherwise('0'))