big data

Hi everyone, Jolan here, with my first actual blog post! It's been a couple of months since the last release of the Kotlin API for Apache Spark and we feel like all of the exciting changes since then are worth another announcement. Let me remind you what the Kotlin API for Apache Spark is and why it was created. Apache Spark is a framework for distributed computations, which data engineers usually use to solve different tasks, like in the ETL process. It supports multiple languages straight out of the box, including Java, Scala, Python, and R. We at JetBrains are committed to supporting one

The Kotlin API for Apache Spark is now widely available. This is the first stable release of the API that we consider to be feature-complete with respect to the user experience and compatibility with core Spark APIs. Get on Maven Central Let’s take a look at the new features this release brings to the API. Typed select and sortMore column functionsMore KeyValueGroupedDataset wrapper functionsSupport for Scala TupleN classesSupport for date and time typesSupport for maps encoded as tuplesConclusion Typed select and sort The Scala API has a typed select method that returns Dataset