Big Data Tools 2022.3: Integration with AWS Glue Data Catalog, Code Completion for SQL Expressions in Zeppelin Notebooks
In this update we've added integrations with AWS Glue, Tencent Cloud Object Storage, enhanced Zeppelin notebooks support, and delivered important fixes. Read on to learn about the most important changes in the Big Data Tools plugin or try it right away by installing it to the 2022.3 of IntelliJ IDEA Ultimate, PyCharm Professional, DataSpell or DataGrip. Tencent Cloud Object Storage (COS) Support We've supported a new distributed storage service. Now you can work with Tencent COS right from the IDE. Supported features include file versioning with the ability to compare two ve
Big Data Tools 2022.2 is here!
The highlights of this release include integration with Hive Metastore and the ability to monitor Flink jobs right inside your IDE, as well as SSO authentication on Amazon S3. The new version provides many other noteworthy changes that are covered below. Get the latest version by installing it to the 2022.2 of your IDE. Hive Metastore Integration We’ve added the ability to create a Hive Metastore connection from the IDE and browse Hive catalogs, tables, and columns. Big Data Tools now also provides code completion for Spark SQL based on Hive Metastore data. Apa
Big Data Tools 2022.2 EAP: What’s New?
Big Data Tools 2022.2 EAP is now available. You can try the newly added features right away by installing the latest plugin version to the 2022.2 EAP of your IDE. Please note this is an Early Access Program build, meaning it’s not fully tested. Hive Metastore support Ability to create a Hive metastore connection from the EMR cluster window and browse Hive catalogs, tables, and columns. Apache Flink Monitoring You can now monitor Flink applications right in your IDE. Just like in the Flink Dashboard, you can launch and stop jobs, all without leaving your IDE. Th
Big Data Tools: Amazon EMR Support, Alibaba OSS Integration, and more
Explore the new features for data engineers in the JetBrains Big Data Tools plugin, including Amazon EMR support and streamlined work with remote file systems.
Big Data Tools 2021.3.1 Is Now Available
We’ve just released a new build of the Big Data Tools plugin that is compatible with the 2021.3 versions of IntelliJ IDEA Ultimate, PyCharm Professional Edition, and DataGrip. The plugin also supports our new data science IDE, DataSpell. In this release, we’ve introduced a number of changes to the user interface, added several features, and fixed a long list of bugs. Let’s take a closer look. Kafka support Producer and consumer management in the Kafka connection tool windowThis allows you to quickly test apps by providing test data or checking the contents of existing topics.You’ll no
Big Data Tools 2021.3 Is Now Available
We’ve just released a new build of the Big Data Tools plugin that is compatible with the 2021.3 version of IntelliJ IDEA Ultimate, PyCharm Professional, and DataGrip. The plugin also supports our new data science IDE, DataSpell. In this release, we’ve introduced a number of changes to the user interface, added a bunch of features, and fixed a long list of bugs. Let’s take a closer look. (more…)
Big Data Tools 2021.3 EAP
Recently we released a new build of the Big Data Tools plugin that is compatible with the 2021.3 EAP versions of IntelliJ IDEA Ultimate and PyCharm Professional. DataGrip 2021.3 EAP support will be available immediately after the release in October. The plugin also supports our new data science IDE – JetBrains DataSpell. If you still use previous versions, now is the perfect time to upgrade both your IDE and the plugin. This year, we introduced a number of new features as well as some features that have been there for a while, for example, running Spark Submit with a run configuration
Big Data Tools Compatible with 2020.3 Is Now Available
In recent Big Data Tools releases, we've added a lot of new features, like Kafka support, a New Project wizard, additional Zeppelin settings, improved Python support, and more. We’ve received a lot of requests for us to polish these new additions, and we’ve taken this feedback to heart. This Version is a massive bug-fix release that greatly improves a wide array of features, both new and old. Here’s the full list of all the bugs we’ve fixed and improvements we’ve made: Improved support for symlinked files and folders (BDIDE-1879, BDIDE-1923)Fixed the problem causing strings in Scala code
Big Data Tools Update
We released another update of the Big Data Tools plugin recently, and this article is an overview of its new features. The previous release was a momentous event for us as the plugin left the Early Access preview stage and became generally available. Since then, we've received a lot of feedback from you and have started developing features that the community wants. For example, Big Data Tools now has support for Kafka, which can be easily connected in just a few clicks. Let's take a closer look at what has changed in the new release. Kafka monitoring Throughout the year, we conduct
Big Data Tools 1.0 Generally Available
A new update of the Big Data Tools plugin has been released. This is our first version for general use, after a year and a half of the Early Access Preview program. Install the plugin from the JetBrains Plugin Repository or from inside your IDE to edit Zeppelin notebooks, upload files to cloud filesystems, and monitor Hadoop and Spark clusters. The following JetBrains IDEs support the plugin: IntelliJ IDEA Ultimate, PyCharm Professional Edition, and DataGrip. In this release, we’ve added many useful features and addressed a variety of bugs. Let's dive into the details. Zeppelin 0.9 Support
Big Data Tools Update Is Out: Experimental Python Support and Search Function in Zeppelin Notebooks
Big Data Tools plugin for version 2021.1 of IntelliJ IDEA Ultimate, PyCharm Professional, and DataGrip has been released. You can install it from the JetBrains Marketplace or from inside your IDE. The plugin allows you to edit Zeppelin notebooks, upload files to cloud filesystems, and monitor Hadoop and Spark clusters. In this release, we've added experimental Python support and global search inside Zeppelin notebooks. We’ve also addressed a variety of bugs. Let's talk about the details. Experimental and preliminary Python support Although PySpark in Zeppelin is getting a lot of
Big Data Tools Update Is Out: SSH Tunnels, Filters and Limits for Spark, User Defined Modules and More
I am pleased to announce that the EAP of Big Data Tools plugin compatible with IntelliJ IDEA Ultimate 2020.2 is available from today. Our goal was to fix as many problems as possible and improve the plugin from the inside out. In addition to some under-the-hood changes, two visible improvements have been made: With just a couple of mouse clicks, you can enable SSH tunnels to forward ports for Hadoop and Spark connections. Filters and limits in the Applications tab of the Spark Monitoring tool window give you control over how much data is loaded from servers. SSH tunneling The servers we