Dstream 是一個 rdd的隊列。
當(dāng)spark stream 窗口函數(shù)的間隔不是batchDuration的倍數(shù)時會報錯。
Exception in thread "main" java.lang.Exception: The window duration of windowed DStream (10000 ms) must be a multiple of the slide duration of parent DStream (3000 ms)
at org.apache.spark.streaming.dstream.WindowedDStream.<init>(WindowedDStream.scala:35)
at org.apache.spark.streaming.dstream.DStream$$anonfun$window$1.apply(DStream.scala:766)
at org.apache.spark.streaming.dstream.DStream$$anonfun$window$1.apply(DStream.scala:766)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:679)
at org.apache.spark.streaming.StreamingContext.withScope(StreamingContext.scala:264)
at org.apache.spark.streaming.dstream.DStream.window(DStream.scala:765)