Thursday, 4 April 2019

Pivot a DataFrame

›
How to pivot the data to create multiple columns out of 1 column with multiple rows. spark-shell --queue= *; To adjust logging level use ...
26 comments:
Sunday, 3 March 2019

How to select multiple columns from a spark data frame using List[Column]

›
Let us create Example DataFrame to explain how to select List of columns of type "Column" from a dataframe spark-shell --qu...
9 comments:
Tuesday, 12 February 2019

How to select multiple columns from a spark data frame using List[String]

›
Lets see how to select multiple columns from a spark data frame. Create Example DataFrame spark-shell --queue= *; To adjust logging...
29 comments:
Wednesday, 2 January 2019

How to add Scalastyle plugin in eclipse

›
Scalastyle is a style checker for Scala. It checks your Scala code against a number of configurable rules, and marks the code which viola...
19 comments:
Thursday, 13 December 2018

error: not found: type Column in Spark Scala

›
How to get rid of below error spark-shell --queue= *; To adjust logging level use sc.setLogLevel(newLevel). Welcome to ____ ...
5 comments:
Thursday, 8 November 2018

How to do an aggregate function on a Spark Dataframe using collect_set

›
In order to explain usage of collect_set, Lets create a Dataframe with 3 columns. spark-shell --queue= *; To adjust logging leve...
3 comments:
Friday, 26 October 2018

How to add multiple withColumn to Spark Dataframe

›
In order to explain, Lets create a dataframe with 3 columns spark-shell --queue= *; To adjust logging level use sc.setLogL...
7 comments: