Changes

Summary

  1. [SPARK-37399][SPARK-37403][PYSPARK][ML] Merge {ml, mllib}/common.pyi (details)
  2. [MINOR][DOCS] Update scaladoc of KnownSizeEstimation (details)
  3. [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty (details)
Commit 3565c3aff9be6cc4e9d6faf9625c0c40ea5fa8d4 by mszymkiewicz
[SPARK-37399][SPARK-37403][PYSPARK][ML] Merge {ml, mllib}/common.pyi into common.py

### What changes were proposed in this pull request?

This PR inlines the type annotations for `{ml, mllib}/common.py`.

### Why are the changes needed?

This allows us to run type checks against the code within both versions of `common.py`.

This would help contributors catch some issues more easily, like this one: https://github.com/apache/spark/pull/34606#discussion_r749783512

### Does this PR introduce _any_ user-facing change?

Potentially. The `C` TypeVar is now public.

### How was this patch tested?

Existing tests.

Closes #34671 from nchammas/SPARK-37393-inline-ml-common-type-annotations.

Authored-by: Nicholas Chammas <nicholas.chammas@gmail.com>
Signed-off-by: zero323 <mszymkiewicz@gmail.com>
The file was modifiedpython/pyspark/ml/common.py (diff)
The file was removedpython/pyspark/mllib/common.pyi
The file was modifiedpython/pyspark/ml/_typing.pyi (diff)
The file was modifiedpython/pyspark/mllib/_typing.pyi (diff)
The file was modifiedpython/pyspark/mllib/common.py (diff)
The file was removedpython/pyspark/ml/common.pyi
Commit 6c73cee8108731414bb56be5f52bd2dffd1eb6d5 by srowen
[MINOR][DOCS] Update scaladoc of KnownSizeEstimation

### What changes were proposed in this pull request?
Followup #9813

### Why are the changes needed?
Fix scaladoc.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?

Existing tests.

Closes #34692 from pan3793/SPARK-11792.

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Sean Owen <srowen@gmail.com>
The file was modifiedcore/src/main/scala/org/apache/spark/util/SizeEstimator.scala (diff)
Commit 24561caad4685b17e70596a4586773cff345bc40 by wenchen
[SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

### What changes were proposed in this pull request?

Currently, there is an inconsistency when handling an empty location for `ALTER NAMESPACE .. SET LOCATION` between v1 and v2 command. In v1 command, an empty string location will result in the `IllegalArgumentException` exception thrown whereas v2 uses the empty string as it is.

This PR proposes to make the behavior consistent by following the v1 command behavior.

### Why are the changes needed?

To make the behavior consistent and the reason for following v1 behavior is that "Spark should be responsible to qualify the user-specified path using its spark/hadoop configs, before passing the path to v2 sources": https://github.com/apache/spark/pull/34610#discussion_r754028045

### Does this PR introduce _any_ user-facing change?

Yes, now the empty string location will result in the `IllegalArgumentException` exception thrown even for v2 catalogs.

### How was this patch tested?

Added a new test

Closes #34686 from imback82/empty_location_fix.

Authored-by: Terry Kim <yuminkim@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala (diff)