scala - Spark SQL filter multiple fields

Question

What is the corrent syntax for filtering on multiple columns in the Scala API? If I want to do something like this:

dataFrame.filter($"col01" === "something" && $"col02" === "something else")

or

dataFrame.filter($"col01" === "something" || $"col02" === "something else") 

EDIT:

This is what my original code looks like. Everything comes in as a string.

df.select($"userID" as "user", $"itemID" as "item", $"quantity" cast("int"), $"price" cast("float"), $"discount" cast ("float"), sqlf.substring($"datetime", 0, 10) as "date", $"group")
  .filter($"item" !== "" && $"group" !== "-1")

Answer 1

I think i see what the issue is. For some reason, spark does not allow two !='s in the same filter. Need to look at how filter is defined in Spark source code.

Now for your code to work, you can use this to do the filter

df.filter(col("item").notEqual("") && col("group").notEqual("-1"))

or use two filters in same statement

df.filter($"item" !== "").filter($"group" !== "-1").select(....)

This link here can help with different spark methods.