src/main/scala/com/github/mrpowers/spark/daria/sql/DariaWriters.scala
Method writeThenMerge
has 6 arguments (exceeds 4 allowed). Consider refactoring. Open
Open
df: DataFrame,
format: String = "csv", // csv, parquet
sc: SparkContext, // pass in spark.sparkContext
tmpFolder: String, // will be deleted, so make sure it doesn't already exist
filename: String, // the full filename you want outputted
Method writeSingleFile
has 6 arguments (exceeds 4 allowed). Consider refactoring. Open
Open
df: DataFrame, // must be small
format: String = "csv", // csv, parquet
sc: SparkContext, // pass in spark.sparkContext
tmpFolder: String, // will be deleted, so make sure it doesn't already exist
filename: String, // the full filename you want outputted