Spark 1.1.1 Released

Spark Release 1.1.1

Spark 1.1.1 is a maintenance release with bug fixes. This release is based on the branch-1.1 maintenance branch of Spark. Recommending all 1.1.0 users to upgrade to this stable release.

To download Spark 1.1.1 visit the downloads page.

Fixes

Spark 1.1.1 contains bug fixes in several components. Some of the more important fixes are highlighted below. You can visit the Spark issue tracker for the full list of fixes.

Spark Core

• Avoid many small spills in external data structures (SPARK-4480)
• Memory leak in connection manager timeout thread (SPARK-4393)
• Incorrect of channel read return value may lead to data truncation (SPARK-4107)
• Stream corruption exceptions observed in sort-based shuffle (SPARK-3948)
• Integer overflow in sort-based shuffle key comparison (SPARK-3032)
• Lack of thread safety in Hadoop configuration usage in Spark (SPARK-2546)

SQL

• Wrong Parquet filters are created for all inequality predicates with literals on the left hand side (SPARK-4468)
• Support backticks in aliases (SPARK-3708 and SPARK-3834)
• ColumnValue types do not match in Spark rows vs Hive rows (SPARK-3704)

PySpark

• Fix sortByKey on empty RDD (SPARK-4304)
• Avoid using the same random seed for all partitions (SPARK-4148)
• Avoid OOMs when take() is run on empty partitions (SPARK-3211)

MLlib

• KryoException caused by ALS.trainImplicit in PySpark (SPARK-3990)

Streaming

• Block replication continuously fails if target is down (SPARK-3495)
• Block replication may choose driver as target (SPARK-3496)

GraphX

• Ensure VertexRDD.apply uses mergeFunc (SPARK-2062)