Console Output

Skipping 8,887 KB.. Full Log
[WARNING] hadoop-client-runtime-3.3.1.jar, protobuf-java-2.6.1.jar define 2 overlapping resources: 
[WARNING]   - META-INF/maven/com.google.protobuf/protobuf-java/pom.properties
[WARNING]   - META-INF/maven/com.google.protobuf/protobuf-java/pom.xml
[WARNING] hadoop-client-runtime-3.3.1.jar, htrace-core4-4.1.0-incubating.jar define 4 overlapping resources: 
[WARNING]   - META-INF/maven/com.fasterxml.jackson.core/jackson-annotations/pom.properties
[WARNING]   - META-INF/maven/com.fasterxml.jackson.core/jackson-annotations/pom.xml
[WARNING]   - META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.properties
[WARNING]   - META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml
[WARNING] commons-logging-1.1.3.jar, htrace-core4-4.1.0-incubating.jar define 2 overlapping resources: 
[WARNING]   - META-INF/maven/commons-logging/commons-logging/pom.properties
[WARNING]   - META-INF/maven/commons-logging/commons-logging/pom.xml
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.3.0-SNAPSHOT.jar with /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.3.0-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/dependency-reduced-pom.xml
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.3.0-SNAPSHOT-sources.jar
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:test-jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.3.0-SNAPSHOT-test-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  5.391 s]
[INFO] Spark Project Tags ................................. SUCCESS [  6.798 s]
[INFO] Spark Project Sketch ............................... SUCCESS [  7.849 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  1.695 s]
[INFO] Spark Project Networking ........................... SUCCESS [  3.764 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  2.029 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  9.617 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  5.023 s]
[INFO] Spark Project Core ................................. SUCCESS [03:30 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 37.447 s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 47.364 s]
[INFO] Spark Project Streaming ............................ SUCCESS [01:26 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [04:19 min]
[INFO] Spark Project SQL .................................. SUCCESS [07:25 min]
[INFO] Spark Project ML Library ........................... SUCCESS [04:59 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 14.135 s]
[INFO] Spark Project Hive ................................. SUCCESS [03:10 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 43.498 s]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 12.081 s]
[INFO] Spark Project YARN ................................. SUCCESS [01:45 min]
[INFO] Spark Project Mesos ................................ SUCCESS [01:26 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [01:37 min]
[INFO] Spark Project Assembly ............................. SUCCESS [ 11.874 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 46.303 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:02 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [01:57 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [01:08 min]
[INFO] Spark Project Examples ............................. SUCCESS [01:21 min]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [ 14.551 s]
[INFO] Spark Avro ......................................... SUCCESS [01:47 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 17.949 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  41:45 min
[INFO] Finished at: 2021-12-09T05:47:06+08:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "hadoop-2.7" could not be activated because it does not exist.
+ echo '=========== PySpark test ==========='
=========== PySpark test ===========
+ source /home/jenkins/venv-test/py36/bin/activate
++ '[' /home/jenkins/venv-test/py36/bin/activate = /tmp/jenkins2974116158935212815.sh ']'
++ deactivate nondestructive
++ unset -f pydoc
++ '[' -z '' ']'
++ '[' -z '' ']'
++ '[' -n /bin/bash ']'
++ hash -r
++ '[' -z '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/home/jenkins/venv/py36
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ PATH=/home/jenkins/venv/py36/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ export PATH
++ '[' -z '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /home/jenkins/venv/py36
++ PS1='(py36) '
++ export PS1
++ alias pydoc
++ true
++ '[' -n /bin/bash ']'
++ hash -r
+ ./python/run-tests --python-executables=python3.6
Running PySpark tests. Output is in /home/jenkins/workspace/spark-master-test-python-arm/python/unit-tests.log
Will test against the following Python executables: ['python3.6']
Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-pandas', 'pyspark-pandas-slow', 'pyspark-resource', 'pyspark-sql', 'pyspark-streaming']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.8
Starting test(python3.6): pyspark.ml.tests.test_base (temp output: /tmp/python3.6__pyspark.ml.tests.test_base__854asoq0.log)
Starting test(python3.6): pyspark.ml.tests.test_algorithms (temp output: /tmp/python3.6__pyspark.ml.tests.test_algorithms__67yuqseg.log)
Starting test(python3.6): pyspark.ml.tests.test_evaluation (temp output: /tmp/python3.6__pyspark.ml.tests.test_evaluation__uvucd9ea.log)
Starting test(python3.6): pyspark.ml.tests.test_feature (temp output: /tmp/python3.6__pyspark.ml.tests.test_feature__0tv7pmta.log)
Finished test(python3.6): pyspark.ml.tests.test_base (16s)
Starting test(python3.6): pyspark.ml.tests.test_image (temp output: /tmp/python3.6__pyspark.ml.tests.test_image__xt_9az6x.log)
Finished test(python3.6): pyspark.ml.tests.test_evaluation (21s)
Starting test(python3.6): pyspark.ml.tests.test_linalg (temp output: /tmp/python3.6__pyspark.ml.tests.test_linalg__g3x22g90.log)
Finished test(python3.6): pyspark.ml.tests.test_image (18s)
Starting test(python3.6): pyspark.ml.tests.test_param (temp output: /tmp/python3.6__pyspark.ml.tests.test_param__gqywmz19.log)
Finished test(python3.6): pyspark.ml.tests.test_feature (35s)
Starting test(python3.6): pyspark.ml.tests.test_persistence (temp output: /tmp/python3.6__pyspark.ml.tests.test_persistence__sb9g6fue.log)
Finished test(python3.6): pyspark.ml.tests.test_linalg (35s)
Starting test(python3.6): pyspark.ml.tests.test_pipeline (temp output: /tmp/python3.6__pyspark.ml.tests.test_pipeline__hel9rmw8.log)
Finished test(python3.6): pyspark.ml.tests.test_param (23s)
Starting test(python3.6): pyspark.ml.tests.test_stat (temp output: /tmp/python3.6__pyspark.ml.tests.test_stat__tuyz6fhp.log)
Finished test(python3.6): pyspark.ml.tests.test_pipeline (6s)
Starting test(python3.6): pyspark.ml.tests.test_training_summary (temp output: /tmp/python3.6__pyspark.ml.tests.test_training_summary__om_loumv.log)
Finished test(python3.6): pyspark.ml.tests.test_stat (17s)
Starting test(python3.6): pyspark.ml.tests.test_tuning (temp output: /tmp/python3.6__pyspark.ml.tests.test_tuning__1tgi_c5n.log)
Finished test(python3.6): pyspark.ml.tests.test_algorithms (98s)
Starting test(python3.6): pyspark.ml.tests.test_util (temp output: /tmp/python3.6__pyspark.ml.tests.test_util__g7dzzaaw.log)
Finished test(python3.6): pyspark.ml.tests.test_training_summary (47s)
Starting test(python3.6): pyspark.ml.tests.test_wrapper (temp output: /tmp/python3.6__pyspark.ml.tests.test_wrapper__r7hsi7b8.log)
Finished test(python3.6): pyspark.ml.tests.test_persistence (75s)
Starting test(python3.6): pyspark.mllib.tests.test_algorithms (temp output: /tmp/python3.6__pyspark.mllib.tests.test_algorithms__2m34p9ox.log)
Finished test(python3.6): pyspark.ml.tests.test_util (25s)
Starting test(python3.6): pyspark.mllib.tests.test_feature (temp output: /tmp/python3.6__pyspark.mllib.tests.test_feature__jr6r_kl3.log)
Finished test(python3.6): pyspark.ml.tests.test_wrapper (21s)
Starting test(python3.6): pyspark.mllib.tests.test_linalg (temp output: /tmp/python3.6__pyspark.mllib.tests.test_linalg__76s_l4ym.log)
Finished test(python3.6): pyspark.mllib.tests.test_feature (37s)
Starting test(python3.6): pyspark.mllib.tests.test_stat (temp output: /tmp/python3.6__pyspark.mllib.tests.test_stat__2diq0blx.log)
Finished test(python3.6): pyspark.mllib.tests.test_algorithms (79s)
Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (temp output: /tmp/python3.6__pyspark.mllib.tests.test_streaming_algorithms__uximz7w9.log)
Finished test(python3.6): pyspark.mllib.tests.test_stat (29s)
Starting test(python3.6): pyspark.mllib.tests.test_util (temp output: /tmp/python3.6__pyspark.mllib.tests.test_util__wi30lo4t.log)
Finished test(python3.6): pyspark.mllib.tests.test_util (13s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_base (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_base__asf7rgzy.log)
Finished test(python3.6): pyspark.mllib.tests.test_linalg (74s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_binary_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_binary_ops__cykpwnrg.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_base (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_boolean_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_boolean_ops__graspv_u.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_binary_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_categorical_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_categorical_ops__hxxncvqr.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_boolean_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_complex_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_complex_ops__np77yl03.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_categorical_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_date_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_date_ops__vycm9m_6.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_complex_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_datetime_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_datetime_ops__lh7_67ug.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_date_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_null_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_null_ops__8ai8oymc.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_datetime_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_num_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_num_ops__h96teve_.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_null_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_string_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_string_ops___zvbmkg_.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_num_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.data_type_ops.test_udt_ops (temp output: /tmp/python3.6__pyspark.pandas.tests.data_type_ops.test_udt_ops__9wltrqei.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_string_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.indexes.test_base (temp output: /tmp/python3.6__pyspark.pandas.tests.indexes.test_base__clf4xq1b.log)
Finished test(python3.6): pyspark.pandas.tests.data_type_ops.test_udt_ops (0s)
Starting test(python3.6): pyspark.pandas.tests.indexes.test_category (temp output: /tmp/python3.6__pyspark.pandas.tests.indexes.test_category__p8qq4rgn.log)
Finished test(python3.6): pyspark.pandas.tests.indexes.test_base (0s)
Starting test(python3.6): pyspark.pandas.tests.indexes.test_datetime (temp output: /tmp/python3.6__pyspark.pandas.tests.indexes.test_datetime__vov552ue.log)
Finished test(python3.6): pyspark.pandas.tests.indexes.test_category (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_frame_plot (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_frame_plot___j030nek.log)
Finished test(python3.6): pyspark.pandas.tests.indexes.test_datetime (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_frame_plot_matplotlib (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_frame_plot_matplotlib___p8n2cnb.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_frame_plot (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_frame_plot_plotly (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_frame_plot_plotly__ajt7oaze.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_frame_plot_matplotlib (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_series_plot (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_series_plot__xy364hnf.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_frame_plot_plotly (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_series_plot_matplotlib (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_series_plot_matplotlib__6vc7g3lc.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_series_plot (0s)
Starting test(python3.6): pyspark.pandas.tests.plot.test_series_plot_plotly (temp output: /tmp/python3.6__pyspark.pandas.tests.plot.test_series_plot_plotly__9gvp2_9_.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_series_plot_matplotlib (0s)
Starting test(python3.6): pyspark.pandas.tests.test_categorical (temp output: /tmp/python3.6__pyspark.pandas.tests.test_categorical__vnjnr6t6.log)
Finished test(python3.6): pyspark.pandas.tests.plot.test_series_plot_plotly (0s)
Starting test(python3.6): pyspark.pandas.tests.test_config (temp output: /tmp/python3.6__pyspark.pandas.tests.test_config__4rna8doa.log)
Finished test(python3.6): pyspark.pandas.tests.test_categorical (0s)
Starting test(python3.6): pyspark.pandas.tests.test_csv (temp output: /tmp/python3.6__pyspark.pandas.tests.test_csv__ma23wpni.log)
Finished test(python3.6): pyspark.pandas.tests.test_config (0s)
Starting test(python3.6): pyspark.pandas.tests.test_dataframe (temp output: /tmp/python3.6__pyspark.pandas.tests.test_dataframe__1ak9riag.log)
Finished test(python3.6): pyspark.pandas.tests.test_csv (0s)
Starting test(python3.6): pyspark.pandas.tests.test_dataframe_conversion (temp output: /tmp/python3.6__pyspark.pandas.tests.test_dataframe_conversion__qaceewy8.log)
Finished test(python3.6): pyspark.pandas.tests.test_dataframe (0s)
Starting test(python3.6): pyspark.pandas.tests.test_dataframe_spark_io (temp output: /tmp/python3.6__pyspark.pandas.tests.test_dataframe_spark_io__dhy_y_ol.log)
Finished test(python3.6): pyspark.pandas.tests.test_dataframe_conversion (0s)
Starting test(python3.6): pyspark.pandas.tests.test_default_index (temp output: /tmp/python3.6__pyspark.pandas.tests.test_default_index__553cn6hm.log)
Finished test(python3.6): pyspark.pandas.tests.test_dataframe_spark_io (0s)
Starting test(python3.6): pyspark.pandas.tests.test_expanding (temp output: /tmp/python3.6__pyspark.pandas.tests.test_expanding__0cvqq0o3.log)
Finished test(python3.6): pyspark.pandas.tests.test_default_index (0s)
Starting test(python3.6): pyspark.pandas.tests.test_extension (temp output: /tmp/python3.6__pyspark.pandas.tests.test_extension__g76k50qc.log)
Finished test(python3.6): pyspark.pandas.tests.test_expanding (0s)
Starting test(python3.6): pyspark.pandas.tests.test_frame_spark (temp output: /tmp/python3.6__pyspark.pandas.tests.test_frame_spark__buqyjeoq.log)
Finished test(python3.6): pyspark.pandas.tests.test_extension (0s)
Starting test(python3.6): pyspark.pandas.tests.test_groupby (temp output: /tmp/python3.6__pyspark.pandas.tests.test_groupby__zz9189wm.log)
Finished test(python3.6): pyspark.pandas.tests.test_frame_spark (0s)
Starting test(python3.6): pyspark.pandas.tests.test_indexing (temp output: /tmp/python3.6__pyspark.pandas.tests.test_indexing__0suwitqs.log)
Finished test(python3.6): pyspark.pandas.tests.test_groupby (0s)
Starting test(python3.6): pyspark.pandas.tests.test_indexops_spark (temp output: /tmp/python3.6__pyspark.pandas.tests.test_indexops_spark___xf1bk_q.log)
Finished test(python3.6): pyspark.pandas.tests.test_indexing (0s)
Starting test(python3.6): pyspark.pandas.tests.test_internal (temp output: /tmp/python3.6__pyspark.pandas.tests.test_internal__vfmexmxl.log)
Finished test(python3.6): pyspark.pandas.tests.test_indexops_spark (0s)
Starting test(python3.6): pyspark.pandas.tests.test_namespace (temp output: /tmp/python3.6__pyspark.pandas.tests.test_namespace__93q1jt4f.log)
Finished test(python3.6): pyspark.pandas.tests.test_internal (0s)
Starting test(python3.6): pyspark.pandas.tests.test_numpy_compat (temp output: /tmp/python3.6__pyspark.pandas.tests.test_numpy_compat__mybc9e38.log)
Finished test(python3.6): pyspark.pandas.tests.test_namespace (0s)
Starting test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames (temp output: /tmp/python3.6__pyspark.pandas.tests.test_ops_on_diff_frames__sng8zk3q.log)
Finished test(python3.6): pyspark.pandas.tests.test_numpy_compat (0s)
Starting test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby (temp output: /tmp/python3.6__pyspark.pandas.tests.test_ops_on_diff_frames_groupby__swr7vazq.log)
Finished test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames (0s)
Starting test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby_expanding (temp output: /tmp/python3.6__pyspark.pandas.tests.test_ops_on_diff_frames_groupby_expanding__d0agyt9e.log)
Finished test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby (0s)
Starting test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby_rolling (temp output: /tmp/python3.6__pyspark.pandas.tests.test_ops_on_diff_frames_groupby_rolling__lycqt5hk.log)
Finished test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby_expanding (0s)
Starting test(python3.6): pyspark.pandas.tests.test_repr (temp output: /tmp/python3.6__pyspark.pandas.tests.test_repr__ulzewxkb.log)
Finished test(python3.6): pyspark.pandas.tests.test_ops_on_diff_frames_groupby_rolling (0s)
Starting test(python3.6): pyspark.pandas.tests.test_reshape (temp output: /tmp/python3.6__pyspark.pandas.tests.test_reshape__cdmje4y1.log)
Finished test(python3.6): pyspark.pandas.tests.test_repr (0s)
Starting test(python3.6): pyspark.pandas.tests.test_rolling (temp output: /tmp/python3.6__pyspark.pandas.tests.test_rolling__izee66wx.log)
Finished test(python3.6): pyspark.pandas.tests.test_reshape (0s)
Starting test(python3.6): pyspark.pandas.tests.test_series (temp output: /tmp/python3.6__pyspark.pandas.tests.test_series__zx_npaso.log)
Finished test(python3.6): pyspark.pandas.tests.test_rolling (0s)
Starting test(python3.6): pyspark.pandas.tests.test_series_conversion (temp output: /tmp/python3.6__pyspark.pandas.tests.test_series_conversion__yfyzxa8d.log)
Finished test(python3.6): pyspark.pandas.tests.test_series (0s)
Starting test(python3.6): pyspark.pandas.tests.test_series_datetime (temp output: /tmp/python3.6__pyspark.pandas.tests.test_series_datetime__yo1bbgy6.log)
Finished test(python3.6): pyspark.pandas.tests.test_series_conversion (0s)
Starting test(python3.6): pyspark.pandas.tests.test_series_string (temp output: /tmp/python3.6__pyspark.pandas.tests.test_series_string__5x2q_37p.log)
Finished test(python3.6): pyspark.pandas.tests.test_series_datetime (0s)
Starting test(python3.6): pyspark.pandas.tests.test_spark_functions (temp output: /tmp/python3.6__pyspark.pandas.tests.test_spark_functions__osqwwtpp.log)
Finished test(python3.6): pyspark.pandas.tests.test_series_string (0s)
Starting test(python3.6): pyspark.pandas.tests.test_sql (temp output: /tmp/python3.6__pyspark.pandas.tests.test_sql__ddynplfy.log)
Finished test(python3.6): pyspark.pandas.tests.test_spark_functions (0s)
Starting test(python3.6): pyspark.pandas.tests.test_stats (temp output: /tmp/python3.6__pyspark.pandas.tests.test_stats__hisgthby.log)
Finished test(python3.6): pyspark.pandas.tests.test_sql (0s)
Starting test(python3.6): pyspark.pandas.tests.test_typedef (temp output: /tmp/python3.6__pyspark.pandas.tests.test_typedef__v_p0c1kq.log)
Finished test(python3.6): pyspark.pandas.tests.test_stats (0s)
Starting test(python3.6): pyspark.pandas.tests.test_utils (temp output: /tmp/python3.6__pyspark.pandas.tests.test_utils__r09d_svh.log)
Finished test(python3.6): pyspark.pandas.tests.test_typedef (0s)
Starting test(python3.6): pyspark.pandas.tests.test_window (temp output: /tmp/python3.6__pyspark.pandas.tests.test_window__y2n8v2bk.log)
Finished test(python3.6): pyspark.pandas.tests.test_utils (0s)
Starting test(python3.6): pyspark.sql.tests.test_arrow (temp output: /tmp/python3.6__pyspark.sql.tests.test_arrow__t2x48qfr.log)
Finished test(python3.6): pyspark.pandas.tests.test_window (0s)
Starting test(python3.6): pyspark.sql.tests.test_arrow_map (temp output: /tmp/python3.6__pyspark.sql.tests.test_arrow_map__dgbmgg0m.log)
Finished test(python3.6): pyspark.sql.tests.test_arrow (0s) ... 75 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_catalog (temp output: /tmp/python3.6__pyspark.sql.tests.test_catalog__yeoxtyqd.log)
Finished test(python3.6): pyspark.sql.tests.test_arrow_map (0s) ... 7 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_column (temp output: /tmp/python3.6__pyspark.sql.tests.test_column__guhul_w4.log)
Finished test(python3.6): pyspark.sql.tests.test_catalog (20s)
Starting test(python3.6): pyspark.sql.tests.test_conf (temp output: /tmp/python3.6__pyspark.sql.tests.test_conf__1i3hkmw0.log)
Finished test(python3.6): pyspark.sql.tests.test_column (23s)
Starting test(python3.6): pyspark.sql.tests.test_context (temp output: /tmp/python3.6__pyspark.sql.tests.test_context__3s5j_ww9.log)
Finished test(python3.6): pyspark.sql.tests.test_conf (11s)
Starting test(python3.6): pyspark.sql.tests.test_dataframe (temp output: /tmp/python3.6__pyspark.sql.tests.test_dataframe__v10dd9c3.log)
Finished test(python3.6): pyspark.sql.tests.test_context (38s)
Starting test(python3.6): pyspark.sql.tests.test_datasources (temp output: /tmp/python3.6__pyspark.sql.tests.test_datasources__m_vcx2sw.log)
Finished test(python3.6): pyspark.sql.tests.test_dataframe (52s) ... 12 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_functions (temp output: /tmp/python3.6__pyspark.sql.tests.test_functions__l4n6p0qy.log)
Finished test(python3.6): pyspark.sql.tests.test_datasources (22s)
Starting test(python3.6): pyspark.sql.tests.test_group (temp output: /tmp/python3.6__pyspark.sql.tests.test_group__qkvbd9_g.log)
Finished test(python3.6): pyspark.sql.tests.test_group (17s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_cogrouped_map__vcedyyzb.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (0s) ... 16 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_grouped_map__a1siresa.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (0s) ... 21 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_map (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_map__mhx9en_8.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_map (0s) ... 8 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_udf__1nphv1bx.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (0s) ... 8 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_udf_grouped_agg__zwa05dc1.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (0s) ... 16 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_udf_scalar__jeqyyavp.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (0s) ... 52 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_udf_typehints__mubw80in.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (0s) ... 10 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window (temp output: /tmp/python3.6__pyspark.sql.tests.test_pandas_udf_window__m962x2mx.log)
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (0s) ... 14 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_readwriter (temp output: /tmp/python3.6__pyspark.sql.tests.test_readwriter__t9zeezra.log)
Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (158s)
Starting test(python3.6): pyspark.sql.tests.test_serde (temp output: /tmp/python3.6__pyspark.sql.tests.test_serde__zueu1ip_.log)
Finished test(python3.6): pyspark.sql.tests.test_functions (42s)
Starting test(python3.6): pyspark.sql.tests.test_session (temp output: /tmp/python3.6__pyspark.sql.tests.test_session__78xpnpy1.log)
Finished test(python3.6): pyspark.sql.tests.test_readwriter (33s)
Starting test(python3.6): pyspark.sql.tests.test_streaming (temp output: /tmp/python3.6__pyspark.sql.tests.test_streaming__avei2y49.log)
Finished test(python3.6): pyspark.sql.tests.test_serde (26s)
Starting test(python3.6): pyspark.sql.tests.test_types (temp output: /tmp/python3.6__pyspark.sql.tests.test_types__2bfkhen2.log)
Finished test(python3.6): pyspark.sql.tests.test_session (49s)
Starting test(python3.6): pyspark.sql.tests.test_udf (temp output: /tmp/python3.6__pyspark.sql.tests.test_udf__tycpqpah.log)
Finished test(python3.6): pyspark.sql.tests.test_types (46s)
Starting test(python3.6): pyspark.sql.tests.test_udf_profiler (temp output: /tmp/python3.6__pyspark.sql.tests.test_udf_profiler__139i_91a.log)
Finished test(python3.6): pyspark.sql.tests.test_streaming (52s)
Starting test(python3.6): pyspark.sql.tests.test_utils (temp output: /tmp/python3.6__pyspark.sql.tests.test_utils__ya07l70u.log)
Finished test(python3.6): pyspark.sql.tests.test_utils (13s)
Starting test(python3.6): pyspark.streaming.tests.test_context (temp output: /tmp/python3.6__pyspark.streaming.tests.test_context__c9izwe6g.log)
Finished test(python3.6): pyspark.sql.tests.test_udf_profiler (18s)
Starting test(python3.6): pyspark.streaming.tests.test_dstream (temp output: /tmp/python3.6__pyspark.streaming.tests.test_dstream__xmdexsg2.log)
Finished test(python3.6): pyspark.ml.tests.test_tuning (377s)
Starting test(python3.6): pyspark.streaming.tests.test_kinesis (temp output: /tmp/python3.6__pyspark.streaming.tests.test_kinesis__mg69fo8a.log)
Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.6): pyspark.streaming.tests.test_listener (temp output: /tmp/python3.6__pyspark.streaming.tests.test_listener__2_xsrn5o.log)
Finished test(python3.6): pyspark.streaming.tests.test_context (20s)
Starting test(python3.6): pyspark.tests.test_appsubmit (temp output: /tmp/python3.6__pyspark.tests.test_appsubmit__w8930dbn.log)
Finished test(python3.6): pyspark.sql.tests.test_udf (58s)
Starting test(python3.6): pyspark.tests.test_broadcast (temp output: /tmp/python3.6__pyspark.tests.test_broadcast__1oqe34jz.log)
Finished test(python3.6): pyspark.streaming.tests.test_listener (9s)
Starting test(python3.6): pyspark.tests.test_conf (temp output: /tmp/python3.6__pyspark.tests.test_conf__tvxk6ia9.log)
Finished test(python3.6): pyspark.tests.test_conf (18s)
Starting test(python3.6): pyspark.tests.test_context (temp output: /tmp/python3.6__pyspark.tests.test_context__c_b0wgv0.log)
Finished test(python3.6): pyspark.tests.test_broadcast (36s)
Starting test(python3.6): pyspark.tests.test_daemon (temp output: /tmp/python3.6__pyspark.tests.test_daemon__e36cvz2u.log)
Finished test(python3.6): pyspark.tests.test_daemon (5s)
Starting test(python3.6): pyspark.tests.test_install_spark (temp output: /tmp/python3.6__pyspark.tests.test_install_spark__3b53ie9w.log)
Finished test(python3.6): pyspark.tests.test_install_spark (21s)
Starting test(python3.6): pyspark.tests.test_join (temp output: /tmp/python3.6__pyspark.tests.test_join__0zpt8g7m.log)
Finished test(python3.6): pyspark.tests.test_context (54s)
Starting test(python3.6): pyspark.tests.test_profiler (temp output: /tmp/python3.6__pyspark.tests.test_profiler__1qqf6l9m.log)
Finished test(python3.6): pyspark.tests.test_join (10s)
Starting test(python3.6): pyspark.tests.test_rdd (temp output: /tmp/python3.6__pyspark.tests.test_rdd__209npu1a.log)
Finished test(python3.6): pyspark.tests.test_appsubmit (84s)
Starting test(python3.6): pyspark.tests.test_rddbarrier (temp output: /tmp/python3.6__pyspark.tests.test_rddbarrier__eggdt1l4.log)
Finished test(python3.6): pyspark.tests.test_profiler (11s)
Starting test(python3.6): pyspark.tests.test_readwrite (temp output: /tmp/python3.6__pyspark.tests.test_readwrite__y7prot4d.log)
Finished test(python3.6): pyspark.tests.test_rddbarrier (8s)
Starting test(python3.6): pyspark.tests.test_serializers (temp output: /tmp/python3.6__pyspark.tests.test_serializers__2tvgo43o.log)
Finished test(python3.6): pyspark.streaming.tests.test_dstream (113s)
Starting test(python3.6): pyspark.tests.test_shuffle (temp output: /tmp/python3.6__pyspark.tests.test_shuffle__xnlvpdp6.log)
Finished test(python3.6): pyspark.tests.test_serializers (12s)
Starting test(python3.6): pyspark.tests.test_statcounter (temp output: /tmp/python3.6__pyspark.tests.test_statcounter__gnh5g2co.log)
Finished test(python3.6): pyspark.tests.test_shuffle (12s)
Starting test(python3.6): pyspark.tests.test_taskcontext (temp output: /tmp/python3.6__pyspark.tests.test_taskcontext__y4xiwvn9.log)
Finished test(python3.6): pyspark.tests.test_readwrite (22s)
Starting test(python3.6): pyspark.tests.test_util (temp output: /tmp/python3.6__pyspark.tests.test_util__k_gfajls.log)
Finished test(python3.6): pyspark.tests.test_statcounter (9s)
Starting test(python3.6): pyspark.tests.test_worker (temp output: /tmp/python3.6__pyspark.tests.test_worker__bh66f9b_.log)
Finished test(python3.6): pyspark.tests.test_util (5s)
Starting test(python3.6): pyspark.accumulators (temp output: /tmp/python3.6__pyspark.accumulators__ttalfr77.log)
Finished test(python3.6): pyspark.accumulators (8s)
Starting test(python3.6): pyspark.broadcast (temp output: /tmp/python3.6__pyspark.broadcast__tuo2z_mk.log)
Finished test(python3.6): pyspark.broadcast (6s)
Starting test(python3.6): pyspark.conf (temp output: /tmp/python3.6__pyspark.conf__3b04vs4b.log)
Finished test(python3.6): pyspark.conf (4s)
Starting test(python3.6): pyspark.context (temp output: /tmp/python3.6__pyspark.context__6bitvf1m.log)
Finished test(python3.6): pyspark.tests.test_worker (24s)
Starting test(python3.6): pyspark.ml.classification (temp output: /tmp/python3.6__pyspark.ml.classification__ms_5ito0.log)
Finished test(python3.6): pyspark.context (23s)
Starting test(python3.6): pyspark.ml.clustering (temp output: /tmp/python3.6__pyspark.ml.clustering__xbj0t8p9.log)
Finished test(python3.6): pyspark.tests.test_taskcontext (80s)
Starting test(python3.6): pyspark.ml.evaluation (temp output: /tmp/python3.6__pyspark.ml.evaluation__sjrle6so.log)
Finished test(python3.6): pyspark.tests.test_rdd (128s)
Starting test(python3.6): pyspark.ml.feature (temp output: /tmp/python3.6__pyspark.ml.feature__6bi4tj09.log)
Finished test(python3.6): pyspark.ml.clustering (48s)
Starting test(python3.6): pyspark.ml.fpm (temp output: /tmp/python3.6__pyspark.ml.fpm__9viw0yg7.log)
Finished test(python3.6): pyspark.ml.evaluation (27s)
Starting test(python3.6): pyspark.ml.functions (temp output: /tmp/python3.6__pyspark.ml.functions__l7f4157b.log)
Finished test(python3.6): pyspark.ml.classification (88s)
Starting test(python3.6): pyspark.ml.image (temp output: /tmp/python3.6__pyspark.ml.image__2gdo1bz5.log)
Finished test(python3.6): pyspark.ml.image (7s)
Starting test(python3.6): pyspark.ml.linalg.__init__ (temp output: /tmp/python3.6__pyspark.ml.linalg.__init____1t6kvhu9.log)
Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.6): pyspark.ml.recommendation (temp output: /tmp/python3.6__pyspark.ml.recommendation__us3s1sxz.log)
Finished test(python3.6): pyspark.ml.fpm (25s)
Starting test(python3.6): pyspark.ml.regression (temp output: /tmp/python3.6__pyspark.ml.regression__8lwztmwj.log)
Finished test(python3.6): pyspark.ml.functions (19s)
Starting test(python3.6): pyspark.ml.stat (temp output: /tmp/python3.6__pyspark.ml.stat__d7kz6lwl.log)
Finished test(python3.6): pyspark.ml.stat (29s)
Starting test(python3.6): pyspark.ml.tuning (temp output: /tmp/python3.6__pyspark.ml.tuning__gklgljcp.log)
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

[Stage 0:>                                                          (0 + 1) / 1]

                                                                                

[Stage 140:>                                                      (0 + 2) / 100]

[Stage 140:===>                                                   (7 + 3) / 100]

[Stage 140:========>                                             (16 + 2) / 100]

[Stage 140:============>                                         (24 + 3) / 100]

[Stage 140:==================>                                   (35 + 2) / 100]

[Stage 140:==========================>                           (49 + 2) / 100]

[Stage 140:==================================>                   (63 + 2) / 100]

[Stage 140:=======================================>              (74 + 3) / 100]

[Stage 140:============================================>         (82 + 2) / 100]

[Stage 140:==================================================>   (93 + 2) / 100]

                                                                                

[Stage 169:==========>                                           (20 + 2) / 100]

[Stage 169:===================>                                  (36 + 2) / 100]

[Stage 169:=============================>                        (54 + 2) / 100]

[Stage 169:=====================================>                (70 + 2) / 100]

[Stage 169:===============================================>      (88 + 2) / 100]

                                                                                

[Stage 317:>                                                       (0 + 2) / 10]

[Stage 317:============================>                           (5 + 2) / 10]

                                                                                
/usr/lib/python3.6/runpy.py:125: RuntimeWarning: 'pyspark.ml.recommendation' found in sys.modules after import of package 'pyspark.ml', but prior to execution of 'pyspark.ml.recommendation'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
/home/jenkins/workspace/spark-master-test-python-arm/python/pyspark/context.py:280: FutureWarning: Python 3.6 support is deprecated in Spark 3.2.
  warnings.warn("Python 3.6 support is deprecated in Spark 3.2.", FutureWarning)
**********************************************************************
File "/home/jenkins/workspace/spark-master-test-python-arm/python/pyspark/ml/recommendation.py", line 322, in __main__.ALS
Failed example:
    predictions[0]
Expected:
    Row(user=0, item=2, newPrediction=0.69291...)
Got:
    Row(user=0, item=2, newPrediction=0.6929099559783936)
**********************************************************************
   1 of  41 in __main__.ALS
***Test Failed*** 1 failures.

Had test failures in pyspark.ml.recommendation with python3.6; see logs.
Process leaked file descriptors. See https://www.jenkins.io/redirect/troubleshooting/process-leaked-file-descriptors for more information
Build step 'Execute shell' marked build as failure
Archiving artifacts
Finished: FAILURE