I want to use spark via both python and scala with Jupyter notebook without having to import all those modules associated with pyspark. I’m using Windows.
Preparation
This post assumes configurations here and here.
Method
I tried a number of kernels in this list, only spylon-spark seems to work.
Follow the instructions here.
Result
