FailedChanges

Summary

  1. [SPARK-32906][SQL] Struct field names should not change after (details)
Commit b49aaa33e13814a448be51a7e65a29cb515b8248 by viirya
[SPARK-32906][SQL] Struct field names should not change after
normalizing floats
### What changes were proposed in this pull request?
This PR intends to fix a minor bug when normalizing floats for struct
types;
``` scala> import
org.apache.spark.sql.execution.aggregate.HashAggregateExec scala> val df
= Seq(Tuple1(Tuple1(-0.0d)), Tuple1(Tuple1(0.0d))).toDF("k") scala> val
agg = df.distinct() scala> agg.explain()
== Physical Plan ==
*(2) HashAggregate(keys=[k#40], functions=[])
+- Exchange hashpartitioning(k#40, 200), true, [id=#62]
  +- *(1) HashAggregate(keys=[knownfloatingpointnormalized(if
(isnull(k#40)) null else named_struct(col1,
knownfloatingpointnormalized(normalizenanandzero(k#40._1)))) AS k#40],
functions=[])
     +- *(1) LocalTableScan [k#40]
scala> val aggOutput = agg.queryExecution.sparkPlan.collect { case a:
HashAggregateExec => a.output.head } scala> aggOutput.foreach { attr =>
println(attr.prettyJson) }
### Final Aggregate ###
[ {
"class" :
"org.apache.spark.sql.catalyst.expressions.AttributeReference",
"num-children" : 0,
"name" : "k",
"dataType" : {
   "type" : "struct",
   "fields" : [ {
     "name" : "_1",
               ^^^
     "type" : "double",
     "nullable" : false,
     "metadata" : { }
   } ]
},
"nullable" : true,
"metadata" : { },
"exprId" : {
   "product-class" : "org.apache.spark.sql.catalyst.expressions.ExprId",
   "id" : 40,
   "jvmId" : "a824e83f-933e-4b85-a1ff-577b5a0e2366"
},
"qualifier" : [ ]
} ]
### Partial Aggregate ###
[ {
"class" :
"org.apache.spark.sql.catalyst.expressions.AttributeReference",
"num-children" : 0,
"name" : "k",
"dataType" : {
   "type" : "struct",
   "fields" : [ {
     "name" : "col1",
               ^^^^
     "type" : "double",
     "nullable" : true,
     "metadata" : { }
   } ]
},
"nullable" : true,
"metadata" : { },
"exprId" : {
   "product-class" : "org.apache.spark.sql.catalyst.expressions.ExprId",
   "id" : 40,
   "jvmId" : "a824e83f-933e-4b85-a1ff-577b5a0e2366"
},
"qualifier" : [ ]
} ]
```
### Why are the changes needed?
bugfix.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added tests.
Closes #29780 from maropu/FixBugInNormalizedFloatingNumbers.
Authored-by: Takeshi Yamamuro <yamamuro@apache.org> Signed-off-by:
Liang-Chi Hsieh <viirya@gmail.com>
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NormalizeFloatingNumbers.scala (diff)