Thursday 13 December 2018

error: not found: type Column in Spark Scala


How to get rid of below error
spark-shell --queue= *;

To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.6.0
Spark context available as sc 
SQL context available as sqlContext.

scala>  val sqlcontext = new org.apache.spark.sql.SQLContext(sc)
sqlcontext: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@4f9a8d71  

scala> var selectExpr : List[Column] = List("Type","Item","Price")
<console>:25: error: not found: type Column
         var selectExpr : List[Column] = List("Type","Item","Price")
                               ^

You need to just import org.apache.spark.sql.Column
scala> import org.apache.spark.sql.Column

import org.apache.spark.sql.Column

scala> var selectExpr : List[Column] = List(col("Type"),col("Item"),col("Price"))
selectExpr: List[org.apache.spark.sql.Column] = List(Type, Item, Price)


4 comments:

  1. This design is incredible! You definitely know how to keep a reader amused. I was almost moved to start my own blog
    메이저사이트
    경마

    ReplyDelete
  2. Fantastic job. I really enjoyed
    what you had to say, and more than that, how you presented it.
    Too cool!
    카지노사이트
    토토사이트

    ReplyDelete
  3. I really enjoy your web’s topic. Very creative and friendly for users. Definitely bookmark this and follow it everyday.

    슬롯머신사이트

    ReplyDelete
  4. These are in fact impressive ideas in on the topic of blogging.
    You have touched some nice things here. 토토

    ReplyDelete