前言
学习Spark SQL解析sql生成DataFrame的源码，进行记录
基于Spark3.0.1
代码
以下面的代码作为入口，进行跟踪调试1
2val df = spark.sql("select * from test_parquet")
df.show
主要的类、方法
spark.sql即SparkSession.sql1
2
3
4
5
6
7
8
9
10
11
12
13/**
* Executes a SQL query using Spark, returning the result as a `DataFrame`.
* The dialect that is used for SQL parsing can be configured with 'spark.sql.dialect'.
*
* @since 2.0.0
*/
def sql(sqlText: String): DataFrame = withActive {
val tracker = new QueryPlanningTracker
val plan = tracker.measurePhase(QueryPlanningTracker.PARSING) {
sessionState.sqlParser.parsePlan(sqlText)
}
Dataset.ofRows(self, plan, tracker)
}