Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
367 views
in Technique[技术] by (71.8m points)

scala - Which one yields better performance in spark data frame?

Which one should be faster here ? Or both have same performance level?

-) Normal Function

  private def fun(lc: Column): Column = lc -  lag(lc, offset = 10, defaultValue = 0).over(Window.partitionBy("colN").orderBy())

-) UDF

val funUdf = udf (fun_)

1) df.withColumn("abc", fun("col1"))

2) df.withColumn("abc", funUdf ("col1"))

Which one will perform faster ?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...