spark sql實(shí)現(xiàn)月銷售額占比以及月環(huán)比和月同比(大廠數(shù)倉(cāng)sql面試題)

1.首先創(chuàng)造數(shù)據(jù),數(shù)據(jù)有4列分別是店鋪id、訂單id、訂單金額、訂單日期。

                spark.createDataFrame(Seq(
                    ("1","11",10,"2023-01-01"),
                    ("1","22",20,"2023-01-02"),
                    ("1","33",10,"2023-02-28"),
                    ("1","44",30,"2023-03-02"),
                    ("1","55",10,"2023-05-02"),
                    ("1","55",20,"2023-06-02"),
                    ("1","11",10,"2022-01-01"),
                    ("1","22",20,"2022-01-02"),
                    ("1","33",10,"2022-02-28"),
                    ("1","44",30,"2022-03-02"),
                    ("1","55",10,"2022-05-02"),
                    ("1","55",20,"2022-06-02"),
                    ("11","11",10,"2023-01-01"),
                    ("11","22",30,"2023-01-02"),
                    ("11","33",10,"2023-02-28"),
                    ("11","44",20,"2023-03-02"),
                    ("11","55",10,"2023-05-02"),
                    ("11","55",30,"2023-06-02"),
                    ("11","11",10,"2022-01-01"),
                    ("11","22",20,"2022-01-02"),
                    ("11","33",10,"2022-02-28"),
                    ("11","44",30,"2022-03-02"),
                    ("11","55",20,"2022-05-02"),
                    ("11","55",30,"2022-06-02")
                )).toDF("shop_id","order_id","amount","event_day").createOrReplaceTempView("t1")

數(shù)據(jù)如下:

+-------+--------+------+----------+
|shop_id|order_id|amount| event_day|
+-------+--------+------+----------+
|      1|      11|    10|2023-01-01|
|      1|      22|    20|2023-01-02|
|      1|      33|    10|2023-02-28|
|      1|      44|    30|2023-03-02|
|      1|      55|    10|2023-05-02|
|      1|      55|    20|2023-06-02|
|      1|      11|    10|2022-01-01|
|      1|      22|    20|2022-01-02|
|      1|      33|    10|2022-02-28|
|      1|      44|    30|2022-03-02|
|      1|      55|    10|2022-05-02|
|      1|      55|    20|2022-06-02|
|     11|      11|    10|2023-01-01|
|     11|      22|    30|2023-01-02|
|     11|      33|    10|2023-02-28|
|     11|      44|    20|2023-03-02|
|     11|      55|    10|2023-05-02|
|     11|      55|    30|2023-06-02|
|     11|      11|    10|2022-01-01|
|     11|      22|    20|2022-01-02|
+-------+--------+------+----------+

2.計(jì)算月銷售額占比
通過(guò)窗口函數(shù)實(shí)現(xiàn),首先聚合月銷售額,之后再根據(jù)月銷售額集合為年銷售額,最后計(jì)算占比即可。

        spark.sql(
            s"""
               |select
               |shop_id,
               |month,
               |year,
               |m_amount,
               |y_amount,
               |round(m_amount/y_amount,4) ratio
               |from
               |(
               |select
               |shop_id,
               |month,
               |m_amount,
               |date_format(month,'yyyy') year,
               |sum(m_amount) over(partition by date_format(month,'yyyy')) y_amount
               |from
               |(
               |select
               |shop_id,
               |date_format(event_day,'yyyy-MM') month,
               |sum(amount) m_amount
               |from t1 group by shop_id,date_format(event_day,'yyyy-MM')
               |) a) aa order by shop_id,month
               |""".stripMargin).show()

結(jié)果


+-------+-------+----+--------+--------+------+
|shop_id|  month|year|m_amount|y_amount| ratio|
+-------+-------+----+--------+--------+------+
|      1|2022-01|2022|      30|     220|0.1364|
|      1|2022-02|2022|      10|     220|0.0455|
|      1|2022-03|2022|      30|     220|0.1364|
|      1|2022-05|2022|      10|     220|0.0455|
|      1|2022-06|2022|      20|     220|0.0909|
|      1|2023-01|2023|      30|     210|0.1429|
|      1|2023-02|2023|      10|     210|0.0476|
|      1|2023-03|2023|      30|     210|0.1429|
|      1|2023-05|2023|      10|     210|0.0476|
|      1|2023-06|2023|      20|     210|0.0952|
|     11|2022-01|2022|      30|     220|0.1364|
|     11|2022-02|2022|      10|     220|0.0455|
|     11|2022-03|2022|      30|     220|0.1364|
|     11|2022-05|2022|      20|     220|0.0909|
|     11|2022-06|2022|      30|     220|0.1364|
|     11|2023-01|2023|      40|     210|0.1905|
|     11|2023-02|2023|      10|     210|0.0476|
|     11|2023-03|2023|      20|     210|0.0952|
|     11|2023-05|2023|      10|     210|0.0476|
|     11|2023-06|2023|      30|     210|0.1429|
+-------+-------+----+--------+--------+------+

3.計(jì)算月環(huán)比
月環(huán)比計(jì)算公式=(本月-上月)/上月
為了方便說(shuō)明分幾步進(jìn)行:
3.1 聚合本月數(shù)據(jù)(為了方便查看進(jìn)行了order by正常不用的)

        spark.sql(
            s"""
               |select
               |shop_id,
               |date_format(event_day,'yyyy-MM') event_day,
               |sum(amount) amount
               |from t1 group by shop_id,date_format(event_day,'yyyy-MM') order by date_format(event_day,'yyyy-MM')
               |""".stripMargin).createOrReplaceTempView("t2")

輸出結(jié)果

+-------+---------+------+
|shop_id|event_day|amount|
+-------+---------+------+
|      1|  2022-01|    30|
|      1|  2022-02|    10|
|      1|  2022-03|    30|
|      1|  2022-05|    10|
|      1|  2022-06|    20|
|      1|  2023-01|    30|
|      1|  2023-02|    10|
|      1|  2023-03|    30|
|      1|  2023-05|    10|
|      1|  2023-06|    20|
|     11|  2022-01|    30|
|     11|  2022-02|    10|
|     11|  2022-03|    30|
|     11|  2022-05|    20|
|     11|  2022-06|    30|
|     11|  2023-01|    40|
|     11|  2023-02|    10|
|     11|  2023-03|    20|
|     11|  2023-05|    10|
|     11|  2023-06|    30|
+-------+---------+------+

3.2 聚合上個(gè)月的數(shù)據(jù)
因?yàn)槲覀冇?jì)算月環(huán)比需要上個(gè)月的銷售額,例如,2022-02需要2022-01月的銷售額才能計(jì)算2022-02的月環(huán)比,即(2022-02銷售額-2022-01銷售額)/2022-01銷售額,那么怎么獲取2022-01銷售額呢?這里采用的方式是join,所以就需要讓上一個(gè)月和本月的日期一樣,就是讓2022-01變成2022-02,然后進(jìn)行join即可。

        spark.sql(
            s"""
               |select
               |shop_id,
               |date_format(add_months(event_day,1),'yyyy-MM') event_day,
               |sum(amount) amount
               |from t1 group by shop_id,date_format(add_months(event_day,1),'yyyy-MM')
               |order by shop_id,date_format(add_months(event_day,1),'yyyy-MM')
               |""".stripMargin).createOrReplaceTempView("t3")

輸出

+-------+---------+------+
|shop_id|event_day|amount|
+-------+---------+------+
|      1|  2022-02|    30|
|      1|  2022-03|    10|
|      1|  2022-04|    30|
|      1|  2022-06|    10|
|      1|  2022-07|    20|
|      1|  2023-02|    30|
|      1|  2023-03|    10|
|      1|  2023-04|    30|
|      1|  2023-06|    10|
|      1|  2023-07|    20|
|     11|  2022-02|    30|
|     11|  2022-03|    10|
|     11|  2022-04|    30|
|     11|  2022-06|    20|
|     11|  2022-07|    30|
|     11|  2023-02|    40|
|     11|  2023-03|    10|
|     11|  2023-04|    20|
|     11|  2023-06|    10|
|     11|  2023-07|    30|
+-------+---------+------+

3.3將本月和上個(gè)月的進(jìn)行join
對(duì)于沒有上個(gè)月的情況,輸出為null,代表無(wú)意義,也可以將null轉(zhuǎn)為其他值,依據(jù)具體需求而定。

        spark.sql(
            s"""
               |select
               |shop_id,
               |a_event_day,
               |round((a_amount-b_amount)/b_amount,4) huanbi
               |from
               |(
               |select
               |t2.shop_id,
               |t2.event_day a_event_day,
               |t3.event_day b_event_day,
               |t2.amount a_amount,
               |t3.amount b_amount
               |from t2 left join t3
               |on t2.shop_id=t3.shop_id and t2.event_day=t3.event_day
               |)  order by shop_id,a_event_day
               |""".stripMargin).show()

輸出

+-------+-----------+-------+
|shop_id|a_event_day| huanbi|
+-------+-----------+-------+
|      1|    2022-01|   null|
|      1|    2022-02|-0.6667|
|      1|    2022-03|    2.0|
|      1|    2022-05|   null|
|      1|    2022-06|    1.0|
|      1|    2023-01|   null|
|      1|    2023-02|-0.6667|
|      1|    2023-03|    2.0|
|      1|    2023-05|   null|
|      1|    2023-06|    1.0|
|     11|    2022-01|   null|
|     11|    2022-02|-0.6667|
|     11|    2022-03|    2.0|
|     11|    2022-05|   null|
|     11|    2022-06|    0.5|
|     11|    2023-01|   null|
|     11|    2023-02|  -0.75|
|     11|    2023-03|    1.0|
|     11|    2023-05|   null|
|     11|    2023-06|    2.0|
+-------+-----------+-------+

有些人會(huì)想到用快窗函數(shù),然后使用lead或者lag獲取上一行或者下一行,但是這總方式是有問(wèn)題的,如果月份不連續(xù),那么計(jì)算的就是錯(cuò)誤的。

4 月同比
今年1月和去年1月進(jìn)行比較。邏輯和環(huán)比一樣只是12個(gè)月,不是1個(gè)月。

        spark.sql(
            s"""
               |select
               |shop_id,
               |date_format(add_months(event_day,12),'yyyy-MM') event_day,
               |sum(amount) amount
               |from t1 group by shop_id,date_format(add_months(event_day,12),'yyyy-MM')
               |order by shop_id,date_format(add_months(event_day,1),'yyyy-MM')
               |""".stripMargin).createOrReplaceTempView("t3")


        spark.sql(
            s"""
               |select
               |shop_id,
               |a_event_day,
               |round((a_amount-b_amount)/b_amount,4) tongbi
               |from
               |(
               |select
               |t2.shop_id,
               |t2.event_day a_event_day,
               |t3.event_day b_event_day,
               |t2.amount a_amount,
               |t3.amount b_amount
               |from t2 left join t3
               |on t2.shop_id=t3.shop_id and t2.event_day=t3.event_day
               |)  order by shop_id,a_event_day
               |""".stripMargin).show()

輸出

+-------+-----------+-------+
|shop_id|a_event_day| tongbi|
+-------+-----------+-------+
|      1|    2022-01|   null|
|      1|    2022-02|   null|
|      1|    2022-03|   null|
|      1|    2022-05|   null|
|      1|    2022-06|   null|
|      1|    2023-01|    0.0|
|      1|    2023-02|    0.0|
|      1|    2023-03|    0.0|
|      1|    2023-05|    0.0|
|      1|    2023-06|    0.0|
|     11|    2022-01|   null|
|     11|    2022-02|   null|
|     11|    2022-03|   null|
|     11|    2022-05|   null|
|     11|    2022-06|   null|
|     11|    2023-01| 0.3333|
|     11|    2023-02|    0.0|
|     11|    2023-03|-0.3333|
|     11|    2023-05|   -0.5|
|     11|    2023-06|    0.0|
+-------+-----------+-------+
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容