Recent questions tagged scala

0 votes

919 views

1 answer

scala - Spark - How to create a sparse matrix from item ratings

My question is equivalent to R-related post Create Sparse Matrix from a data frame, except that I would like to perform ... rate.toInt) }) ...? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

939 views

1 answer

scala - How to use spark to generate huge amount of random integers?

I need lots of random numbers, one per line. The result should be something like this: 24324 24324 4234234 4234234 ... piece of code? Thank you. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - How to print elements of particular RDD partition in Spark?

How to print the elements of a particular partition, say 5th, alone? val distData = sc.parallelize(1 to 50, 10) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

993 views

1 answer

scala - How to use from_json with Kafka connect 0.10 and Spark Structured Streaming?

I was trying to reproduce the example from [Databricks][1] and apply it to the new connector to Kafka and spark ... /main/scala-2.11/Main.scala See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.7k views

1 answer

scala - How to merge two columns of a `Dataframe` in Spark into one 2-Tuple?

I have a Spark DataFrame df with five columns. I want to add another column with its values being the tuple of ... df.schema.fieldNames(2)) ) ) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

871 views

1 answer

scala - Why is this cyclic reference with a type projection illegal?

The following pseudo-Scala yields an "illegal cyclic reference" error: trait GenT[A] trait T extends GenT[T#A ... a cyclic type definition? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

scala - Upload zip file using --archives option of spark-submit on yarn

I have a directory with some model files and my application has to access these models files in local file ... extracted or misused this option? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

813 views

1 answer

scala - Sequencing an HList

Given a Shapeless HList where every list element shares the same type constructor, how can the HList be ... for an arbitrary Applicative? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.3k views

1 answer

scala - Why is my object not a member of package <root> if it's in a separate source file?

I'm having a problem accessing an object defined in the root package. If I have all my code in one file, it ... package, where on earth is it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

912 views

1 answer

scala - Remove Temporary Tables from Apache SQL Spark

I have registertemptable in Apache Spark using Zeppelin below: val hvacText = sc.textFile("...") case class ... getting nowhere. Any guidance ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

922 views

1 answer

scala - How to add elements to Source dynamically?

I have example code to generate an unbound source and working with it: object Main { def main(args : Array[ ... } } How to implement MySources? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Adaptation of argument list by inserting () has been deprecated

I'm just in the process of upgrading from Scala 2.10.x to 2.11.2 and I'm receiving the following ... thinks it needs to causes other problems See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

865 views

1 answer

scala - Multiple packages definition

While browsing the source code of the Scala API, I met this package definition in scala/tags/R_2_8_0_final/src/ ... in more than one package? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

811 views

1 answer

scala - What's the difference between `::` and `+:` for prepending to a list)?

List has 2 methods that are specified to prepend an element to an (immutable) list: +: (implementing Seq.+:), ... list concatenation, ::: vs ++ See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

809 views

1 answer

scala - How to map variable names to features after pipeline

I have modified the OneHotEncoder example to actually train a LogisticRegression. My question is how to map ... ] Intercept: 2.6679020381781235 See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

822 views

1 answer

scala - Combining Futures, Eithers and Options in for comprehensions

I have a collection of methods that return different types: Either[ErrorResponse, X] Future[Either[ErrorResponse, X ... (client, responseType) } See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

749 views

1 answer

scala - How to force interpreter show complete stack trace?

Is there any way to force Scala interpreter (started through SBT) to print complete stack trace. By default, less ... but that really ugly... See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.3k views

1 answer

scala - SBT: Exclude class from Jar

I am converting a legacy jar project to SBT and for strange reasons that are not easily solved, this project comes ... I may be mistaken here). See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

736 views

1 answer

scala - How are co- and contra-variance used in designing business applications?

I know about using co- and contravariance in the standard library (e.g. collections and trait Function) I ... real world" business applications. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

973 views

1 answer

scala - Is there a way to add extra metadata for Spark dataframes?

Is it possible to add extra meta data to DataFrames? Reason I have Spark DataFrames for which I need ... such extra information on DataFrames? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

821 views

1 answer

scala - Ambiguous Reference to overloaded definition - One vs Two Parameters

Given the following companion object with overloaded versions of apply: object List { def apply[T](): List[T] ... single argument method isn't? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

941 views

1 answer

scala - Add a compile time only dependency in sbt

I would like to add a dependency to an sbt project which is only used for compilation. Neither should it be on the runtime ... "1.3.0" % ??? ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

930 views

1 answer

scala - Why do I get a "missing parameter for expanded function" in one case and not the other?

Case this works: Seq(fromDir, toDir) find (!_.isDirectory) foreach (println(_)) Whereas this doesn't: Seq ... is a reasonable explanation ;) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

904 views

1 answer

scala - Multiple executable jar files with different external dependencies from a single project with sbt-assembly

I have a single scala project built with sbt using a build.scala file. I can use sbt-assembly to generate ... to generate multiple jar files. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

914 views

1 answer

scala - Why doesn't Option have a fold method?

I wonder why scala.Option doesn't have a method fold like this defined: fold(ifSome: A => B , ifNone: = ... better than using map + getOrElse? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - View SQL query in Slick

Is there a way to observe an SQL statement that will be generated by Query? For example, I have this: val q ... I view its underlying raw SQL? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

967 views

1 answer

scala - Get the size/length of an array column

I'm new in Scala programming and this is my question: How to count the number of string for each row? My ... = [friends: array<string>] See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

870 views

1 answer

scala - How to create hive table from Spark data frame, using its schema?

I want to create a hive table using my Spark dataframe's schema. How can I do that? For fixed columns ... to automatically generate such query? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags