Big Data Tools 1.6 is here!
The highlights of this release include integration with Hive Metastore and the ability to monitor Flink jobs right inside your IDE, as well as SSO authentication on Amazon S3. The new version provides many other noteworthy changes that are covered below.
Get the latest version by installing it to the 2022.2 of your IDE.
Hive Metastore Integration
In v.1.6, we’ve added the ability to create a Hive Metastore connection from the IDE and browse Hive catalogs, tables, and columns.
Big Data Tools now also provides code completion for Spark SQL based on Hive Metastore data.
Apache Flink Monitoring in the IDE
You can now monitor Apache Flink applications right in your IDE with the ability to submit jobs.
Improved AWS EMR Integration
The Amazon EMR integration allows you to monitor clusters and nodes. As of v.1.6, you can view EMR clusters as a tree view and filter them by the following states: terminated last day/hour/week. We also added the ability to open logs as .txt files.
Enhanced RFS Storages Support
Amazon S3 now supports SSO authentication and can filter by region for S3 connections.
Microsoft Azure now supports multiple containers.
We also added the ability to create SFTP connection with project SSH configuration, and introduced multiple improvements to RFS file editor.
Kafka Monitoring Refinements
Monitoring your Kafka event streaming processes has become even more convenient. Now you can filter topics in the topics lists. Support for SASL_PLAINTEXT authentication has also been added.
Improved Usability of the Big Data Tools Panel
We’ve reworked keyboard interactions inside the Big Data Tools panel and added multiple shortcuts, such as opening Kafka, Hadoop, and Spark Monitoring from the panel by simply clicking Enter, and adding convenient drag-and-drop functionality.
For the full list of new features and enhancements see the changelog on the Big Data tools plugin page. Please share your feedback with us and report any issues you encounter to our issue tracker.
For more updates follow us on Twitter!
The Big Data Tools team