Kotlin
A concise multiplatform language developed by JetBrains
State of Kotlin Scripting 2024
Update: we made some updates to the original post to accurately reflect the state of Custom Scripting and to avoid misinterpretations.
TL;DR: Kotlin scripting remains an essential part of the Kotlin infrastructure. We continue to support it in the experimental state, and we are concluding certain experiments and reducing the number of scripting-related technologies that we provide and actively develop.
Kotlin scripting is the technology that enables executing Kotlin code as scripts without prior compilation or packaging into executables. In addition, several extension mechanisms exist to facilitate the usage of Kotlin for specific applications, such as configuring build tools. On top of that, REPL functionality, which is closely related to scripting, is available for Kotlin in various forms (including, for example, Kotlin Notebook).
All these technologies can be found in Kotlin and other JetBrains projects. Some of them are well known, others are barely used, but all require a noticeable amount of attention from the team.
The evolution of scripting in Kotlin
Scripting was introduced into Kotlin long ago as an experiment to investigate new ways of using the language. Since then, the development has been driven by the demand of external and internal adopters, as well as some experiments born inside our team. Over time, we accumulated a lot of functionality related to scripting in our codebase, and the effort required to support it grew quite significantly. Therefore, at some point, we started to review the functionality we have, how it is used inside and outside of JetBrains, and how we can distill it into a reasonable minimal subset that we are willing to support and develop without breaking too many existing use cases.
This blog post attempts to summarize our findings, explain our decisions, and outline the future directions in this area. We hope it will clarify the future of scripting in Kotlin and give the community a solid basis for making technical decisions.
While we already have a firm roadmap for the immediate future, we welcome and appreciate your feedback and ideas for further improvements. We are especially interested in hearing about any particular use cases for scripting that you may have. We encourage you to use comments here or create YouTrack issues describing your scenarios and how the announced changes may affect them. Your insights are invaluable to us and will play a crucial role in shaping our future plans for the language’s development.
Contribute Your Feedback and Ideas
Basic scripting
We conducted some research into scripting usage in the past and concluded that besides a few main adoptions, like Gradle build scripts, there is a relatively small number of known uses for scripting and REPL. This might be attributable to various factors, but the result is nonetheless clear: The functionality is not as popular as we had anticipated.
Some technological problems could (and will) be addressed, but there are some inherent issues with attempting to position Kotlin as a scripting language.
In particular, Kotlin is not an interpreted language, and it cannot achieve the user experience typical for dedicated scripting languages. To achieve the current script-like behavior, we compile the code under the hood. The compilation process is quite intensive because the Kotlin compiler was not designed for such scenarios.
That is why we made the following decision:
Although we will continue to provide generalized support for scripting in Kotlin, which includes compilation and evaluation of basic `.kts` files, as well as custom scripting (more about this below), we are not prepared to recommend Kotlin scripting as a generally applicable scripting solution, for example, as a replacement for Bash or Python.
On the other hand, we believe there are many new usage scenarios where, despite the known limitations, the use of Kotlin scripting could be beneficial. There are also several other potential use cases that we would like to explore further.
Custom script types
The most powerful scripting solutions require extension mechanisms that allow for the customization of script compilation and evaluation. One of the most prominent examples is Gradle Kotlin DSL, where the Custom Scripting API is used to bring Kotlin language with traditional Gradle DSL to build scripts. This brings us to the second decision:
We will continue to support the Custom Scripting API (an unofficial name for a set of APIs for customizing script compilation and evaluation, as well as APIs for embedding script hosts into third-party applications). Therefore:
- We are concentrating our efforts on improving the user experience with a few known use cases, in particular Gradle Kotlin DSL and select others.
- Outside these use cases, the API remains in the experimental state. Hence, things like documentation and IDE support for generic custom script support may continue to lag behind.
Main.kts
In addition to the basic and custom scripting, we would like to continue supporting `.main.kts` scripts. The `.main.kts` scripts were initially developed to demonstrate the utility of the Custom Scripting API. However, with its support of dependencies and other features, it was adopted by many users as a default Kotlin scripting solution. Therefore, the next decision is:
We will continue to develop the `.main.kts` script type, which is already helpful for simple automation tasks. We have plans to extend its functionality and streamline IDE support. Such scripts continue to be supported out of the box in the Kotlin compiler and the Kotlin plugin for IntelliJ IDEA.
Kotlin REPL
The default REPL implementation has been a part of the Kotlin compiler (`kotlinc`) and Kotlin plugin for IntelliJ IDEA since the first release. Still, the functionality is limited, and improving it was never a priority for the team. Therefore, the user experience is not on par with the general Kotlin user experience in IntelliJ IDEA. We made several attempts to spin off alternative REPL implementations (e.g. the Kotlin Interactive Shell), but unfortunately, these never gained enough traction.
Meanwhile, we are improving the Kotlin Notebook plugin for IntelliJ IDEA, which offers a smooth, extensive, and interactive experience working in Kotlin. With Kotlin Notebook, you can develop and experiment with Kotlin code, receive immediate outputs, and visualize data. We believe that it is an excellent replacement for all our current REPL solutions.
Besides that, IntelliJ IDEA provides Kotlin Scratch files, where you can quickly prototype your code. Therefore:
We plan to sunset the default REPL implementations in the Kotlin compiler and the IntelliJ IDEA plugin.
- CLI REPL (via `kotlinc`) will continue to function at least until the release of Kotlin 2.3, but its operation will be limited to compatibility mode, i.e. with the `-language-version 1.9` option set (and may require an opt-in flag starting from release 2.2).
- The default Kotlin REPL in the IntelliJ IDEA plugin will be removed in one of the next IntelliJ IDEA releases.
We will continue to promote the Kotlin Notebook plugin and IDE Scratch files as solutions for interactive Kotlin development.
This decision places some uncertainty on the external REPL implementations, like the Kotlin Interactive Shell. Although we plan to keep some REPL-related functionality in the compiler and Custom Scripting API (not least because solutions like Kotlin Notebook rely on them), with a final switch to the K2 compiler, a significant portion of this functionality will be changed or dropped, so it may require substantial effort to rewrite such implementations to the changed APIs.
Other technologies based on Kotlin scripting
Besides these main areas, there are a few other related technologies and APIs for scripting that we are currently supporting. We believe the cases explicitly mentioned above cover a majority of possible user needs. Therefore, we plan to drop most other scripting-related components and libraries from the compiler and IntelliJ IDEA. In particular:
- JSR-223 support – considering that the original JSR is in the withdrawn state, we do not believe supporting the de-facto obsolete API makes sense. The existing implementation will continue to function at least until the release of Kotlin 2.3 in the language version 1.9 compatibility mode (but we may consider renaming the artifact to raise awareness of the planned changes), and it will be dropped after that.
- `KotlinScriptMojo` – a Maven plugin that supports script execution during the Maven build. We did not find evidence of enough usages to keep maintaining it, so we plan to drop it in one of the next Kotlin releases.
- `kotlin-scripting-ide-services` – a library for implementing code completion functionality, mainly for REPL implementations. It is currently used in projects like Kotlin Interactive. It is heavily based on the infrastructure of the pre-K2 compiler and cannot be easily ported to the K2 version. Therefore, it will most likely stop working around the release of Kotlin 2.3 and will be removed from the codebase. We may consider reimplementing similar functionality on top of K2 in the future, but for now, this is not something we are actively pursuing.
Moving forward
We hope the focused approach will allow us to move forward and provide a better experience with widely used scripting technologies. At the same time, it will free some resources for exciting language features and applications. We encourage you to share your use cases for scripting and appreciate your feedback on these changes and how they may affect your work.
Share Your Feedback and Use Cases
What else to read
Subscribe to Kotlin Blog updates
