Making Kotlin Ready for Data Science

This year at KotlinConf 2019, Roman Belov gave an overview on Kotlin’s approach to data science. Now that the talk is available for everyone to see, we decided to recap it and share a bit more on the current state of Kotlin tools and libraries for data science.

How does Kotlin fit data science? Following the need to analyze large amounts of data, the last few years has brought a true renaissance to the data science discipline. All this renaissance of data science couldn’t be possible without proper tools. Before, you needed a programming language designed specifically for data science, but today you can already do it with general-purpose languages. Of course this requires general-purpose languages to make the right design decisions, not to mention getting the community to help in. All this made certain languages, such as Python, more popular for data science than others.

With the concept of Kotlin Multiplatform, Kotlin aims to replicate its developer experience and extend its interoperability to other platforms as well. The major qualities of Kotlin by design include conciseness, safety, and interoperability. These fundamental language traits make it a great tool for a wide variety of tasks and platforms. Data science is certainly one of these tasks.

The great news is that the community has already begun adopting Kotlin for data science, and this adoption is happening at a fast pace. The brief report below outlines how ready Kotlin is for data science, including the Kotlin libraries and Kotlin tools for data science.

Jupyter

First and foremost, thanks to their interactivity, Jupyter notebooks are very convenient for transforming, visualizing, and presenting data. With the extensibility and the open-source nature of Jupyter, it has turned into a large ecosystem around data science and was integrated into tons of other solutions related to data. Among them is the Kotlin kernel for Jupyter notebooks. With this kernel, you can write and run Kotlin code in Jupyter notebooks and use third-party data science frameworks written in Java and Kotlin.

An example of a reproducible Kotlin Jupyter notebook can be found in this repo. To quickly play with a Kotlin notebook, you can launch it on Binder (please note the environment will normally take a minute to set up).

Apache Zeppelin

Due to the strong support for Spark and Scala, Apache Zeppelin is very popular among data engineers. Similar to Jupyter, Zeppelin has a plugin API (called Interpreters) to extend its core with support for other tools and languages. Currently, the latest release of Zeppelin (0.8.2) doesn’t come with a bundled Kotlin interpreter. But anyway, it is available in the master branch of Zeppelin. To learn how to deploy Zeppelin with Kotlin support in a Spark cluster, see these instructions.

Apache Spark

Since Spark has a robust Java API, you can already use Kotlin to work with the Spark Java API from both Jupyter and Zeppelin without any problems. However we’re working on improving this integration by adding full support for Kotlin classes with Spark’s Dataset API. Support for Kotlin with Spark’s shell is also in progress.

Libraries

Using Kotlin for data science alone, without libraries, makes little sense. Luckily, thanks to the recent efforts of the community, there’s already a number of nice Kotlin libraries that you can use right away.

Here are some of the most useful libraries:

  • kotlin-statistics is a library that provides a set of extension functions to perform exploratory and production statistics. It supports basic numeric list/sequence/array functions (from sum to skewness), slicing operators (e.g. countBy, simpleRegressionBy, etc), binning operations, discrete PDF sampling, naive bayes classifier, clustering, linear regression, and more.
  • kmath is a library inspired by numpy; this library supports algebraic structures and operations, array-like structures, math expressions, histograms, streaming operations, wrappers around commons-math and koma, and more.
  • krangl is a library inspired by R’s dplyr and Python’s pandas; this library provides functionality for data manipulation using a functional-style API; it allows you to filter, transform, aggregate, and reshape tabular data.
  • lets-plot is a library for declaratively creating plots based on tabular data. This library is inspired by R’s ggplot and The Grammar of Graphics, and is integrated tightly with the Kotlin kernel. It is multi-platform and can be used not just with JVM, but also from JS and Python.
  • kravis is another library inspired by R’s ggplot for visualizing tabular data.

For a more complete list of useful links, please refer to Kotlin data science resources by Thomas Nield.

Lets-Plot for Kotlin

Lets-Plot is an open-source plotting library for statistical data written entirely in Kotlin. Being a multiplatform library, it has an API designed specifically for Kotlin. You can familiarize yourself with how to use this API by reading its user guide.

For interactivity, Lets-Plot is tightly integrated with the Kotlin kernel for Jupyter notebooks. Once you have the Kotlin kernel installed and enabled, add the following line to a Jupyter notebook:

%use lets-plot

Then you will be able to call Lets-Plot API functions from your cells, and see the results immediately beneath the cells as you would normally have by using ggplot with R or Python:

Kotlin bindings for NumPy

NumPy is a popular package for scientific computing with Python. It provides powerful capabilities for multi-dimensional array processing, linear algebra, Fourier transform, random numbers, and other mathematical tasks. Kotlin Bindings for NumPy is a Kotlin library that enables calling NumPy functions from Kotlin code by providing statically typed wrappers for NumPy functions.

Contribution

The entire Kotlin ecosystem is based on the idea of open source and would not be possible without the help of many contributors. Kotlin for data science is only emerging and needs your help now as ever! Here’s how you can pitch in:

  • Talk about your pain points and share your ideas on how to make Kotlin even better-suited for data-science tasks – your tasks.
  • Contribute to the open source data-science-related libraries, and create your own libraries and tools – anything that you think can help Kotlin become a language of choice for data science.

The Kotlin community has a dedicated channel called #datascience in its Slack. We invite you to join this channel to ask questions, find out in what areas help is needed and how you can contribute, and of course share your feedback and your work with the community.

Keep in mind that Kotlin is still in the very early stages of becoming the tool of choice for data scientists. It’s going to be an exciting and challenging journey! It will require building a rich ecosystem of tools and libraries, as well as adjusting the language design to meet the needs of data-related tasks. If you see things not working as you would expect, please share your experience – or get involved and help fix them. Give them a try, especially the Jupyter kernel and libraries, and share your feedback with us.

Resources

Most of the information in this post, and much more, can be found on the official Kotlin website.

KotlinConf 2019 had more inspiring talks about data science, including a Kotlin for Science by Alexander Nozik and another one Gradient Descent with Kotlin by Erik Meijer.

We also recommend watching these talks from the past two KotlinConf conferences: a talk by Holger Brandl (the creator of krangl, Kotlin’s analog of Python’s pandas), and this talk by Thomas Nield (the creator of kotlin-statistics).

That’s it for today (and probably for this year). Wrapping it all up, the community is adopting Kotlin for data science at a good pace, so now it’s your turn.

Let’s Kotlin!

Posted in Data Science, Libraries, Tools | Tagged , , | 1 Comment

What to Expect in Kotlin 1.4 and Beyond

During the keynote at KotlinConf, Andrey highlighted our strategic view on the current areas of focus for the evolution of Kotlin, and our plans for Kotlin 1.4 which will be released some time next year.

Watch the entire keynote below

Our vision is for Kotlin to be a reliable companion for all your endeavors, a default language choice for your tasks. To accomplish this, we’re going to make it shine on all platforms. Multiple case studies from companies well-known in the industry show that we are making good progress in this direction.

Kotlin 1.4 that is going to arrive in spring 2020 will make another step forward for the Kotlin ecosystem.

Focusing on quality

Most of all, Kotlin 1.4 will focus on quality and performance. Kotlin is a modern language that already pioneers many ideas and approaches. We’re going to keep it modern and always evolving. At the moment, however, we believe that Kotlin has reached the stage where improving the overall experience is more important than adding big features. This is why Kotlin 1.4 will deliver only a few small language changes, which are explained in detail below.

Continue reading

Posted in Roadmap | Tagged | 18 Comments

Kotlin 1.3.60 Released

We’re happy to present the new release today, Kotlin 1.3.60. In addition to the quality improvements, this version focuses on:

  • Optimizing the comparison of inline classes.
  • Tooling improvements for debugging, J2K converter, and Gradle scripts written in Kotlin.
  • Support for more Kotlin/Native platforms/targets.
  • Improving the Kotlin/MPP IDE experience.
  • For Kotlin/JS, adding support for source maps and improving the platform test runner integration.
  • Preview for some already implemented features of Kotlin 1.4.

You can find the complete list of changes in the change log. As always, we’re really grateful to our external contributors.

Let’s dive into the details!

Language changes

An incremental release doesn’t bring any language changes other than minor improvements (like changing confusing error messages) or updates for experimental features (like inline classes). To take a sneak peek at what is coming in Kotlin 1.4, read the corresponding section below.

Continue reading

Posted in Releases | Tagged | 13 Comments

KotlinConf 2019 Live: Join the Broadcast, Attend the Q&A!

It’s turning into a nice tradition to broadcast KotlinConf. This year the community will have full access to the conference via the KotlinConf 2019 Live broadcast.

We will be covering four of the presentation tracks scheduled for the conference, and the icing on the cake is that there will be some special content that’s only available online through KotlinConf 2019 Live! Tune in to watch 12 insightful interviews with the speakers over both conference days, moderated by Huyen Tue Dao, also a speaker at KotlinConf and the host of “Android Dialogs”.

stream_banners_1200x628
We will begin our broadcast with the Keynote by Andrey Breslav, then continue with two full days of talks, and finish with a closing panel. During the panel, speakers and the Kotlin development team will answer questions raised from the community. Don’t miss the discussion – post your questions on Twitter with the #kc19ask hashtag, and then tune into the Closing Panel live stream to get the answers.

There is also another cool opportunity to watch KotlinConf 2019 together with your community. Host a KotlinConf 2019 Global meetup and get support from JetBrains! Learn more about these events in our blog post and submit your own event for support.

What, when, and how

The broadcast will begin on December 5, at 8 AM GMT, with the Keynote delivered by Kotlin development team lead Andrey Breslav. After the keynote, you can tune in to one of 4 streams of the talks scheduled at the conference website (the “Hands-on Labs” track will not be broadcast). The schedule of 12 interviews with speakers, hosted by Huyen Tue Dao, will be announced soon, so please follow us on Twitter @kotlinconf for the latest updates. The closing panel broadcast will begin on December 6, at 4:15 PM GMT.

Join the broadcast on the kotlinconf.com home page at any time. Sign up for reminders to make sure you don’t miss the keynote or the closing panel. We will remind you 24 hours prior to the keynote, and again once we are live.

If you cannot attend the broadcast of a talk or cannot choose among the four tracks, don’t worry – all the talks will be recorded and linked to from the KotlinConf website. We will also email you as soon as the videos are available.

Enjoy KotlinConf 2019!

Posted in community, Events | Tagged , | 2 Comments

KotlinConf 2019 Global: Join In!

2019 has been amazing for the Kotlin community with developments like the language of choice on Android, a wave of Kotlin/Everywhere events, Breakout Project of the Year at OSCON ’19… and the year is not over yet.

Today we reveal one more wonderful opportunity for Kotlin lovers: get together for KotlinConf 2019 Global with your local user group!

KC19_Global_badge_Blogpost

Organize a meetup for your community to watch the KotlinConf 2019 keynote and other sessions together, and JetBrains will support your community. If you’re not an organizer, keep an eye on future blog posts and follow @kotlinconf on Twitter, where we will share a global map of the upcoming events.

Submit your Event

To host KotlinConf 2019 Global, please complete and send us the form below. Every community, be it KUG, GDG, AWSUG, or any other, is welcome to join in. You can request support for the events happening between December 5, 2019 and March 5, 2020 the events should be submitted until January 5, 2020.

Before you submit an event, please make sure to announce it on your website or any other suitable platform.

Submit an event

KotlinConf 2019 Live Stream

We will kick off the KotlinConf live stream on December 5, at 8 am GMT, with an opening keynote by Andrey Breslav, and will continue streaming all tracks over the two conference days. Feel free to join the live stream at any point or access the video recordings later.

The keynote and session recordings should be available within the next two weeks following the conference. Follow @kotlinсonf for the latest information.

KotlinConf Global Branding

To announce your event, use the branding materials provided. Please do not use Kotlin or JetBrains branding. The name of the event should include “KotlinConf 2019 Global” and the name of the event location. You can edit the materials according to your event location and date.

Branding materials

Posted in community, Events | 13 Comments

Instil + Kotlin == Happiness

A Case Study of the JetBrains Training Partnership

Instil is a software development company based in Belfast, Northern Ireland. They specialize in mobile / cloud projects, consultancy, and training. Their training department is made up of 3 full-time and 4 associate trainers, all of whom have at least 15 years of experience in developing, coaching, and consultancy. 

At JetBrains,  we’re very excited to have partnered with Instil for our Kotlin Certified Training ProgramThe following post, which tells their Kotlin story, has been written by Garth Gilmour from Instil.

Instil first became involved in Kotlin in 2014. Back then Microservices, Single Page Applications and Cloud were very much in their infancy, and polyglot programming meant some adventurous soul had extended their Grade build file via Groovy. We were very much a traditional JVM shop focused around Java. Several of our developers had successfully completed the Coursera Scala certification and I had written a modestly popular Scala course, but there didn’t seem to be much appetite on either the development or the training side for pushing that option any harder.

It was my colleague Gareth Fleming who first spotted the merits of Kotlin and began to advocate for it within the company. This was no accident. Gareth has an extensive background in mobile development, but at the time was teaching Java courses in response to overwhelming demand on the services side of the business. So he was well placed to appreciate how Kotlin would be an advantage both to experienced software teams and to graduate developers trying to acclimatise to the JVM ecosystem. Gareth became such a fan he started speaking on Kotlin at conferences, managed to drag me away from Scala (not an easy task at the time) and persuaded our development teams to try Kotlin on new Android projects.

Continue reading

Posted in Uncategorized | 1 Comment

Kotlin Heroes programming contest, round #2

Kotlin Heroes

We are excited to announce the second round of our Kotlin Heroes programming contest taking place on the Codeforces platform. JetBrains is working once again with Codeforces to promote Kotlin in the competitive programming community, as well as to give the Kotlin community a platform to compare and hone their algorithmic programming skills. We hope that any of you who entered our first round on the 22nd of May this year will join us again now you know what to expect and try to improve (or at least hold) your standing on the leaderboard.

You can see the standings from the first Kotlin heroes here, it was a closely fought contest. We would like to thank everyone for entering and submitting their answers and congratulate the top 3 winners Petr, ecnerwala, and abacabadabacaba on their incredible achievement, especially considering they were up against 4,500 other registrants from over 63 countries. We’d also like to give a shout out to tourist, for being the only other person who managed to solve every problem set.

Episode 2 of the Kotlin Heroes competition will be hosted again on Codeforces, this time on the 7th of September, 2019, at 14:35 UTC (17:35 MSK, 07:35 PDT, 22:35 CST). The contest will last 2,5 hours and will feature a set of problems designed for both beginners and seasoned competitive programmers alike. The top three winners will receive prizes of $512, $256, and $128, respectively. The top 50 contestants will win a Kotlin Heroes t-shirt and an exclusive Kotlin badge. Finally, every competitor who solves at least one problem will enter a draw for one of the 50 exclusive Kotlin Heroes t-shirts.

Join Kotlin Heroes!

Continue reading

Posted in community | 1 Comment

Kotlin 1.3.50 released

We’re happy to announce the release of Kotlin 1.3.50 today. In addition to the quality and tooling improvements, the main focus for this version has been on:

  • Designing a new Duration and Time Measurement API (available for preview).
  • Working on an improved Java-to-Kotlin converter.
  • Experimental generation of external declarations for npm dependencies in Gradle Kotlin/JS projects (using Dukat).
  • A separate plugin for debugging Kotlin/Native code in IntelliJ IDEA Ultimate.
  • Java compilation support in multiplatform projects.

You can find the complete list of changes in the change log. As always, we’d like to thank our external contributors. Now let’s dive into the details!

Null-check optimizations planned for Kotlin 1.4

As you probably know, Kotlin decreases the possibility of NullPointerExceptions by providing support for nullable types. However, because of interoperability with Java code, it’s impossible to avoid NPEs completely. To help developers better understand the source of a nullability problem if it occurs, Kotlin compiler throws different types of runtime exceptions with clear error messages instead of pure NPEs. It turned out that this approach has its disadvantages: it reduces possible null check optimizations that can be performed either by the Kotlin compiler or by various kinds of bytecode processing tools, such as the Android R8 optimizer.

Continue reading

Posted in Releases | Tagged | 27 Comments

Kotlin wins Breakout Project of the Year award at OSCON ’19

Today at the O’Reilly Open Source Awards 2019, we were honored to hear that Kotlin has won the prestigious Breakout Project of the Year award, this award recognizes a project that has started “breaking out in a big way” over the past year, which we are really happy about. We want to say a massive thank you to O’Reilly OSCON for this recognition, but more importantly, we want to take a minute to thank the incredible Kotlin community who have brought the project to where it is today.

Since our first commit to the GitHub repository on the 7th of November, 2010, we have watched in awe as the open-source community has embraced the language and driven it forward with their contribution, support, and ideas which have gone into the language we are all very proud of.

There is still work to do, but as long as we have such an incredible community behind us we know we can make this language what it needs to be for developers. If you haven’t used Kotlin yet, we would love to have you try. Your contributions and feedback are what help us improve. Get involved in the evolution of the language and let’s see how far we can go together.

Again, to everyone who has helped make Kotlin the success it is today, thank you and have a nice Kotlin! :)

Posted in Uncategorized | 4 Comments

Kotlin 1.3.40 released

We’re happy to present the new release today, Kotlin 1.3.40. In addition to the quality and tooling improvements the main focus for this version has been on:

  • Gradle support for NPM, Yarn, and Webpack for Kotlin/JS
  • Test runner improvements for multiplatform projects
  • New type inference
  • Performance and interoperability improvements for Kotlin/Native

Also, new functions were added to the standard library in an experimental state.

You can find the complete list of changes in the change log. As always, we’re really grateful to our external contributors.

Let’s dive into the details!

Continue reading

Posted in Releases | Tagged | 15 Comments