Changes

Summary

  1. [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more (commit: f893a19) (details)
  2. [SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName (commit: 7fb9f68) (details)
  3. [SPARK-32189][DOCS][PYTHON] Development - Setting up IDEs (commit: 9c653c9) (details)
Commit f893a19c4cf62dd13bf179de75af6feb677c4154 by gurwls223
[SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more
information in installation guide
### What changes were proposed in this pull request?
This PR:
- rephrases some wordings in installation guide to avoid using the terms
that can be potentially ambiguous such as "different favors"
- documents extra dependency installation `pip install pyspark[sql]`
- uses the link that corresponds to the released version. e.g.)
https://spark.apache.org/docs/latest/building-spark.html vs
https://spark.apache.org/docs/3.0.0/building-spark.html
- adds some more details
I built it on Read the Docs to make it easier to review:
https://hyukjin-spark.readthedocs.io/en/stable/getting_started/install.html
### Why are the changes needed?
To improve installation guide.
### Does this PR introduce _any_ user-facing change?
Yes, it updates the user-facing installation guide.
### How was this patch tested?
Manually built the doc and tested.
Closes #29779 from HyukjinKwon/SPARK-32180.
Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by:
HyukjinKwon <gurwls223@apache.org>
(commit: f893a19)
The file was removedpython/docs/source/getting_started/installation.rst
The file was modifiedpython/docs/source/getting_started/index.rst (diff)
The file was addedpython/docs/source/getting_started/install.rst
The file was modifiedpython/setup.py (diff)
The file was modifiedpython/docs/source/conf.py (diff)
Commit 7fb9f6884f5e085e97b60fe45055247c2d17245c by gurwls223
[SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName
### What changes were proposed in this pull request?
Add optional `allowMissingColumns` argument to SparkR `unionByName`.
### Why are the changes needed?
Feature parity.
### Does this PR introduce _any_ user-facing change?
`unionByName` supports `allowMissingColumns`.
### How was this patch tested?
Existing unit tests. New unit tests targeting this feature.
Closes #29813 from zero323/SPARK-32799.
Authored-by: zero323 <mszymkiewicz@gmail.com> Signed-off-by: HyukjinKwon
<gurwls223@apache.org>
(commit: 7fb9f68)
The file was modifiedR/pkg/R/DataFrame.R (diff)
The file was modifiedR/pkg/R/generics.R (diff)
The file was modifiedpython/pyspark/sql/dataframe.py (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/Dataset.scala (diff)
The file was modifiedR/pkg/tests/fulltests/test_sparkSQL.R (diff)
Commit 9c653c957f5cd9237cc2ad0a5bc28ead2dab75cb by gurwls223
[SPARK-32189][DOCS][PYTHON] Development - Setting up IDEs
### What changes were proposed in this pull request?
This PR proposes to document the way of setting up IDEs
![스크린샷 2020-09-21 오전 10 43
12](https://user-images.githubusercontent.com/44108233/93727715-5c2a6e80-fbf7-11ea-821b-555723b00bc8.png)
![스크린샷 2020-09-21 오전 10 43
45](https://user-images.githubusercontent.com/44108233/93727716-5f255f00-fbf7-11ea-9c6c-7b8a973bc511.png)
### Why are the changes needed?
To let users know how to setup IDEs
### Does this PR introduce _any_ user-facing change?
Yes, it adds a new page in the documentation about setting IDEs.
### How was this patch tested?
Manually built the doc.
Closes #29781 from itholic/SPARK-32189.
Authored-by: itholic <haejoon309@naver.com> Signed-off-by: HyukjinKwon
<gurwls223@apache.org>
(commit: 9c653c9)
The file was addeddocs/img/pycharm-with-pyspark2.png
The file was addeddocs/img/pycharm-with-pyspark3.png
The file was addedpython/docs/source/development/setting_ide.rst
The file was addeddocs/img/pycharm-with-pyspark1.png
The file was modifiedpython/docs/source/development/index.rst (diff)
The file was modifiedpython/docs/source/development/debugging.rst (diff)