Big Data Tools: Amazon EMR Support, Alibaba OSS Integration, and more
In this new release of Big Data Tools, we’ve added support for more instruments for data engineers, including Amazon EMR data processing platform and Alibaba Cloud Object Storage Service. Now you can work with them directly from your IDE. Read on to learn about these and other enhancements introduced in this release.
Amazon EMR Support
Now you can monitor clusters and nodes in the Amazon EMR data processing platform.
User-friendly and simple UI interface to work with clusters.
With just one click, you can create a connection to:
- Hadoop and Spark Monitoring
- AWS S3
- SFTP main and data nodes
You can also open internal cluster service web pages (like Ganglia) through an automatically created SSH tunnel.
Run applications with Spark Submit on Amazon EMR
You can configure all Spark-Submit parameters inside the IDE without using the console, as well as upload jars from a local machine to S3 or a similar cluster.
Remote File Systems support improvements
Alibaba Cloud Object Storage Service support
Alibaba OSS has been added to an extensive list of remote and cloud file storages supported by the plugin, including AWS S3, Google Cloud Storage, Microsoft Azure, and more. Now you can connect your Alibaba OSS and access its data directly from the IDE.
New file viewer
With the new full-size viewer for remote file systems, you can find more metainfo about folders and files and view file information in a separate panel.
Those are the most notable highlights of our major release. For the full list of new features and enhancements see the What’s new on the JetBrains Big Data tools plugin page. Please share your feedback with us and report any issues you encounter to our issue tracker.
Follow us on Twitter!
The Big Data Tools team
Subscribe to Blog updates
Thanks, we've got you!
Big Data Tools 2023.1 Is Out!
In this release, we’ve added integration with Kafka Schema Registry, Kerberos authentication, and extended support for all cloud storages in Big Data Tools.
Big Data Tools 2022.3: Integration with AWS Glue Data Catalog, Code Completion for SQL Expressions in Zeppelin Notebooks
In this update we've added integrations with AWS Glue, Tencent Cloud Object Storage, enhanced Zeppelin notebooks support, and delivered important fixes. Read on to learn about the most important changes in the Big Data Tools plugin or try it right away by installing it to the 2022.3 of IntelliJ…
Big Data Tools 2022.2 is here!
The highlights of this release include integration with Hive Metastore and the ability to monitor Flink jobs right inside your IDE, as well as SSO authentication on Amazon S3. The new version provides many other noteworthy changes that are covered below. Get the latest version by installing it to…
Big Data Tools 2022.2 EAP: What’s New?
Big Data Tools 2022.2 EAP is now available. You can try the newly added features right away by installing the latest plugin version to the 2022.2 EAP of your IDE. Please note this is an Early Access Program build, meaning it’s not fully tested. Hive Metastore support Ability to creat…