Changes

Summary

  1. [SPARK-36242][CORE][3.0] Ensure spill file closed before set success = (details)
Commit 1061c9f020182b17a00926bdf0af811c53b8d4cc by dhyun
[SPARK-36242][CORE][3.0] Ensure spill file closed before set success = true in ExternalSorter.spillMemoryIteratorToDisk method

### What changes were proposed in this pull request?
The main change of this pr is move `writer.close()` before `success = true` to ensure spill file closed before set `success = true` in `ExternalSorter.spillMemoryIteratorToDisk` method.

### Why are the changes needed?
Avoid setting `success = true` first and then failure of close spill file

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

- Pass the Jenkins or GitHub Action
- Add a new Test case to check `The spill file should not exists if writer close fails`

Closes #33514 from LuciferYang/SPARK-36242-30.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
The file was modifiedcore/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala (diff)
The file was addedcore/src/test/scala/org/apache/spark/util/collection/ExternalSorterSpillSuite.scala