Recent questions tagged scala

0 votes

578 views

1 answer

scala - How to pre-package external libraries when using Spark on a Mesos cluster

According to the Spark on Mesos docs one needs to set the spark.executor.uri pointing to a Spark distribution: ... backend part can be achieved. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

719 views

1 answer

scala - How to change the column type from String to Date in DataFrames?

I have a dataframe that have two columns (C, D) are defined as string column type, but the data in the ... didn't have info relate to date See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

757 views

1 answer

scala - Logging options for Slick

I'm createing a Play 2.1 app, in which I have decided to use Slick for database interaction. However I can ... for Slick. Anyone knows this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

807 views

1 answer

scala - covariant type T occurs in contravariant position

I know this question has been asked before, but either the answers don't apply to this case, or I don't ... make Test co-variant in T. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

770 views

1 answer

scala - What is the difference between Try and Either?

According to the documentation: The Try type represents a computation that may either result in an exception, or return ... one over the other? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

957 views

1 answer

scala - abstract type pattern is unchecked since it is eliminated by erasure

Could someone tell me how can I avoid the warning in the code block below: abstract class Foo[T <: Bar]{ case class ... t: T) => println(t) ^ See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Re-run with -feature for details

When I compile scala in play console, I got this: [warn] there were 1 feature warning(s); re-run with -feature ... . So how should I do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

766 views

1 answer

scala - Add new column in DataFrame base on existing column

I have a csv file with datetime column: "2011-05-02T04:52:09+00:00". I am using scala, the file is ... base on value of existing column? Thanks See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - Slick: query multiple tables/databases with getting column names

I have methods in my Play app that query database tables with over hundred columns. I can't define case class ... do I achieve this with Slick? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

848 views

1 answer

scala - List files on S3

I'm getting frustrated by not finding any good explanation on how to list all files in a S3 bucket. I ... S3 bucket? Answers much appreciated! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

618 views

1 answer

scala - Setting up sbt environment to hack on multiple libraries at once

Is there an equivalent to Leiningen's "checkouts" feature in sbt? Here is what I want to accomplish: I have ... something similar in sbt...) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

719 views

1 answer

scala - converting Akka's Future[A] to Future[Either[Exception,A]]

Is there a method in Akka (or in the standard library in Scala 2.10) to convert a Future[A] which might fail into ... 2.9/Akka and Scala 2.10. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

710 views

1 answer

scala - Mock Objects in Play[2.0]

I want to test my Play application by providing mock objects during a test. Off the top of my head, ... insight would be greatly appreciated. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

678 views

1 answer

scala - Spark runs out of memory when grouping by key

I am attempting to perform a simple transformation of common crawl data using Spark host on an EC2 using this ... without running out of memory? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.8k views

1 answer

scala - How do I convert a WrappedArray column in spark dataframe to Strings?

I am trying to convert a column which contains Array[String] to String, but I consistently get this error org. ... ", mkString($"arrayCol")) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - how to use Regexp_replace in spark

I am pretty new to spark and would like to perform an operation on a column of a dataframe so as to ... suitable way would be much appreciated See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

765 views

1 answer

scala - How to uncache RDD?

I used cache() to cache the data in memory but I realized to see the performance without cached data I need to ... how to do the uncache then! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

756 views

1 answer

scala - GSON JsonObject "Unsupported Operation Exception: null" getAsString

Running a Play! app with Scala. I'm doing a request where the response is expected to be a JSON string. ... s failing on all methods? Thanks. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

795 views

1 answer

scala - RestAPI service call from Spark Streaming

I have a use case where I need to call RESTAPI from spark streaming after messages are read from Kafka to ... timeout with streaming batch time. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

969 views

1 answer

scala - Spark - Sum of row values

I have the following DataFrame: January | February | March ----------------------------- 10 | 10 ... .select(colsToSum.map(col):_*).foreach ... See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

889 views

1 answer

scala - How to profile the time spent for a task in SBT

I'd like to know why sometimes my build is much slower. So I've decided to measure the time of ... ? Related questions: Profiling sbt builds See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

707 views

1 answer

scala - How to make tree mapping tail-recursive?

Suppose I have a tree data structure like this: trait Node { val name: String } case class BranchNode(name: ... to make it tail-recursive? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

893 views

1 answer

scala - How to catch slick postgres exceptions for duplicate key value violations

My table has a unique index on a pair of columns in my postgresql database. I want to know how I can catch ... much in scala either yet also. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

752 views

1 answer

scala - Why does Seq.contains accept type Any rather than the type parameter A?

For example: scala> val l:List[String] = List("one", "two") l: List[String] = List(one, two) scala> l. ... Seq short of cloning it into a Set? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

601 views

1 answer

scala - Spark Streaming mapWithState seems to rebuild complete state periodically

I am working on a Scala (2.11) / Spark (1.6.1) streaming project and using mapWithState() to keep track of ... to be a flaw in my application? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

715 views

1 answer

scala - Is it possible to override a type field?

scala> class C defined class C scala> class subC extends C defined class subC scala> class A { type T = C} ... is wrong with the example above ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

617 views

1 answer

scala - When to use Option

I learn Scala for some time and can't clearly understand the usage of Option. It helps me to avoid null checks when ... about when, not why. :) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

770 views

1 answer

scala - Difference between sc.broadcast and broadcast function in spark sql

I have used sc.broadcast for lookup files to improve the performance. I also came to know there is a function ... the reference/look up tables? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags