Okay, so operations on RDDs are performed in parallel by different workers (usually on different machines in the cluster) and therefore you cannot pass in this type of "global" object arr
to be updated. You see, each worker will receive their own copy of arr
which they will update, but the driver will never know.
I'm guessing that what you want to do here is to collect all the arrays from the RDD, which you can do with a simple collect
action:
val dataRDD = sc.textFile(args(0)).map(line => line.split(" ")).map(x => Array(x(0).toInt, x(1).toInt, x(2).toInt))
val arr = dataRDD.collect()
Where arr
has type Array[Array[Int]]
. You can then run through arr
with normal array operations.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…