Recent questions tagged scala

0 votes

557 views

1 answer

scala - Optimize Spark job that has to calculate each to each entry similarity and output top N similar items for each

I have a Spark job that needs to compute movie content-based similarities. There are 46k movies. Each movie ... SparseVector, Rating: Double ) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

612 views

1 answer

scala - Reading Avro messages from Kafka with Spark 2.0.2 (structured streaming)

I have a spark 2.0 application that reads messages from kafka using spark streaming (with spark-streaming-kafka- ... I deserialize the value? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

541 views

1 answer

scala - How to set up implicit conversion to allow arithmetic between numeric types?

I'd like to implement a class C to store values of various numeric types, as well as boolean. Furthermore, ... m overlooking. Thanks in advance! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

728 views

1 answer

scala - Usage of spark DataFrame "as" method

I am looking at spark.sql.DataFrame documentation. There is def as(alias: String): DataFrame Returns a new ... of alias using this method. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

948 views

1 answer

scala - Spark: coalesce very slow even the output data is very small

I have the following code in Spark: myData.filter(t => t.getMyEnum() == null) .map(t => t.toString) . ... a better way to solve this problem? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

692 views

1 answer

scala - What does the <> operator do in Slick?

I was walking through the documentation of Slick to setup a quick working prototype. In the Mapped Tables ... help in understanding this. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

699 views

1 answer

scala - Convert a List of Options to an Option of List using Scalaz

I want to transform a List[Option[T]] into a Option[List[T]]. The signature type of the function is def lo2ol[T](lo: ... xs)) => x :: xs } }} See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - How to find duplicates in a list?

I have a list of unsorted integers and I want to find those elements which have duplicates. val dup = List(1,1 ... an easier way to solve this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

638 views

1 answer

scala - Safely copying fields between case classes of different types

Assuming you have case classes like the following case class Test1(a:String,b:Int,c:Char) case class Test2(a:String ... 2 at some point in time) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - SBT is unable to find credentials when attempting to download from an Artifactory virtual repo

I'm trying to run SBT behind a corporate firewall. Another team has configured an Artifactory proxy. This proxy ... .com/sbt/sbt/issues/2817 See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

496 views

1 answer

scala - Apache Spark: network errors between executors

I'm running Apache Spark 1.3.1 on Scala 2.11.2, and when running on an HPC cluster with large ... io.netty.buffer.PooledHeapByteBuf.setBytes(PooledHeapByteBuf.java:234) at io.n...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

630 views

1 answer

scala - "using" function

I've defined 'using' function as following: def using[A, B <: {def close(): Unit}] (closeable: B) (f: B ... out) => out.println(in.readLIne) } See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

633 views

1 answer

scala - Difference between trait inheritance and self type annotation

In Scala, I've seen the constructs trait T extends S and trait T { this: S => used to achieve similar ... would you use one over the other? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

732 views

1 answer

scala - What's the point of declaring an object as "final"?

I just noticed that it's possible to declare objects as final in Scala: final object O What's the point of doing ... A // not found: type A See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

738 views

1 answer

scala - Spark Structured Streaming with Hbase integration

We are doing streaming on kafka data which being collected from MySQL. Now once all the analytics has been ... DF_Hbase dataframe in hbase. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

640 views

1 answer

scala - How to implement actor model without Akka?

How to implement simple actors without Akka? I don't need high-performance for many (non-fixed count) actor instances ... up-to-date with Akka. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Pattern matching with shapeless coproduct

Can I use pattern matching with shapeless coproducts? import shapeless.{CNil, :+:} type ListOrString = List[Int ... ability to pattern match? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

687 views

1 answer

scala - Overriding multiple config values in Typesafe config when using an uberjar to deploy

I've an Akka application which uses multiple configuration values (IP address, port numbers) defined in resource/ ... conf file are used) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

549 views

1 answer

scala - Difference between map and mapAsync

Can anyone please explain me difference between map and mapAsync w.r.t AKKA stream? In the documentation it is said ... in the nature of these ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

796 views

1 answer

scala - Rolling your own reduceByKey in Spark Dataset

I'm trying to learn to use DataFrames and DataSets more in addition to RDDs. For an RDD, I know I can do ... doesn't have encoders for Map yet See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

625 views

1 answer

scala - Why does Spark application fail with “ClassNotFoundException: Failed to find data source: kafka” as uber-jar with sbt assembly?

I'm trying to run a sample like StructuredKafkaWordCount. I started with the Spark ... streams/tree/ad0d0134ddb7664d359c8dca40f1d16ddd94053f See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

689 views

1 answer

scala - How to change a column position in a spark dataframe?

I was wondering if it is possible to change the position of a column in a dataframe, actually to change the ... : how to change the schema? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

809 views

1 answer

scala - Defining sbt task that invokes method from project code?

I'm using SBT to build a scala project. I want to define a very simple task, that when I input generate in sbt ... to define such a task in SBT? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

630 views

1 answer

scala - How to carry on executing Future sequence despite failure?

The traverse method from Future object stops at first failure. I want a tolerant/forgiving version of this method ... better way to do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

922 views

1 answer

scala - Filename too long sbt

I am getting an error saying I have a file that is too long in sbt. [info] Compiling 29 Scala sources to / ... classfile-name and set it to 254. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

820 views

1 answer

scala - Re-download a SNAPSHOT version of a dependency using SBT

I have the following lines in my build.sbt file. resolvers += "specs2 snapshot repo" at "http://scala- ... version? Using update does nothing. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Play Framework - add a field to JSON object

I have a problem with adding a field to Json object in Play Framework using Scala: I have a case class ... c:3}) Any help appreciated! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

747 views

1 answer

scala - covariant type A occurs in contravariant position in type A of value a

I have following class: case class Box[+A](value: A) { def set(a: A): Box[A] = Box(a) } And ... someone please explain, why the error occurs? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags