Changes

Summary

  1. [SPARK-37008][SQL][TEST] Replace `UseCompressedOops` with (details)
  2. [SPARK-36992][SQL] Improve byte array sort perf by unify getPrefix (details)
Commit ee2647e52f84320b74b946533037aca41320c04e by srowen
[SPARK-37008][SQL][TEST] Replace `UseCompressedOops` with `UseCompressedClassPointers` to pass `WholeStageCodegenSparkSubmitSuite` with Java 17

### What changes were proposed in this pull request?
`UseCompressedOops` Option has some different behaviors between Java 8, Java 11 and Java 17, if we execute command

```
java -XX:+PrintFlagsFinal -XX:-UseCompressedOops -version | grep Compressed
```

The result of Java 8 is

```
bool UseCompressedClassPointers                = false                               {lp64_product}
bool UseCompressedOops                             := false                               {lp64_product}
```

The result of Java 11 is

```
bool UseCompressedClassPointers               = false                                {lp64_product} {default}
bool UseCompressedOops                             = false                                {lp64_product} {command line}
```

The result of Java 17 is
```
bool UseCompressedClassPointers               = true                           {product lp64_product} {ergonomic}
bool UseCompressedOops                             = false                          {product lp64_product} {command line}
```

We found that `-XX:-UseCompressedOops` and `-XX:-UseCompressedClassPointers` are no longer  react in chain when use Java 17, if we execute

```
mvn clean install -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.execution.WholeStageCodegenSparkSubmitSuite
```

with Java 17, the test failed due to `Exception in thread "main" org.scalatest.exceptions.TestFailedException: 16 was not greater than 16`, so the change of this pr is  replace `UseCompressedOops` with `UseCompressedClassPointers` to pass `WholeStageCodegenSparkSubmitSuite` with Java 17 .

### Why are the changes needed?
Pass UT with JDK 17

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

- Pass the Jenkins or GitHub Action
- Manual test with Java 11 passed

```
Run completed in 15 seconds, 546 milliseconds.
Total number of tests run: 1
Suites: completed 2, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

```

- Manual test with Java 17

```
mvn clean install -pl sql/core -am -Dtest=none -DwildcardSuites=org.apache.spark.sql.execution.WholeStageCodegenSparkSubmitSuite
```

**Before**

```
2021-10-14 04:32:38.038 - stderr> Exception in thread "main" org.scalatest.exceptions.TestFailedException: 16 was not greater than 16
  2021-10-14 04:32:38.038 - stderr> at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
  2021-10-14 04:32:38.038 - stderr> at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
  2021-10-14 04:32:38.038 - stderr> at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)
  2021-10-14 04:32:38.038 - stderr> at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.sql.execution.WholeStageCodegenSparkSubmitSuite$.main(WholeStageCodegenSparkSubmitSuite.scala:82)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.sql.execution.WholeStageCodegenSparkSubmitSuite.main(WholeStageCodegenSparkSubmitSuite.scala)
  2021-10-14 04:32:38.038 - stderr> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2021-10-14 04:32:38.038 - stderr> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  2021-10-14 04:32:38.038 - stderr> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2021-10-14 04:32:38.038 - stderr> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
  2021-10-14 04:32:38.038 - stderr> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
```

**After**

```
Run completed in 11 seconds, 15 milliseconds.
Total number of tests run: 1
Suites: completed 2, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```

Closes #34286 from LuciferYang/SPARK-37008.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSparkSubmitSuite.scala (diff)
Commit f9cc7fbbad5fcef908f6e3e460c3770db6856a5c by srowen
[SPARK-36992][SQL] Improve byte array sort perf by unify getPrefix function of UTF8String and ByteArray

### What changes were proposed in this pull request?

Unify the getPrefix function of `UTF8String` and `ByteArray`.

### Why are the changes needed?

When execute sort operator, we first compare the prefix. However the getPrefix function of byte array is slow. We use first 8 bytes as the prefix, so at most we will call 8 times with `Platform.getByte` which is slower than call once with `Platform.getInt` or `Platform.getLong`.

### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

pass `org.apache.spark.util.collection.unsafe.sort.PrefixComparatorsSuite`

Closes #34267 from ulysses-you/binary-prefix.

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
The file was modifiedcommon/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java (diff)
The file was modifiedcommon/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java (diff)
The file was addedcommon/unsafe/src/test/java/org/apache/spark/unsafe/array/ByteArraySuite.java