Recently when I was using spark-shell I couldn’t use JBLAS library which has DoubleMatrix classes. After a quick go through I could use it using –package option.
Run spark-shell using following command
./spark-shell --packages org.jblas:jblas:1.2.4-SNAPSHOT --driver-memory 4G
If the package lookup can’t find it in the local maven repositories try build JBLAS once and the library will be available
git clone https://github.com/mikiobraun/jblas.git cd jblas mvn install