Big Data Tools: Amazon EMR Support, Alibaba OSS Integration, and more

Anna Maltseva

Read this post in other languages:

• 简体中文

In this new release of Big Data Tools, we’ve added support for more instruments for data engineers, including Amazon EMR data processing platform and Alibaba Cloud Object Storage Service. Now you can work with them directly from your IDE. Read on to learn about these and other enhancements introduced in this release.

Amazon EMR Support

Now you can monitor clusters and nodes in the Amazon EMR data processing platform.

User-friendly and simple UI interface to work with clusters.

With just one click, you can create a connection to:

• HDFS
• Hadoop and Spark Monitoring
• Zeppelin
• AWS S3
• SFTP main and data nodes

You can also open internal cluster service web pages (like Ganglia) through an automatically created SSH tunnel.

Run applications with Spark Submit on Amazon EMR

You can configure all Spark-Submit parameters inside the IDE without using the console, as well as upload jars from a local machine to S3 or a similar cluster.

Remote File Systems support improvements

Alibaba Cloud Object Storage Service support

Alibaba OSS has been added to an extensive list of remote and cloud file storages supported by the plugin, including AWS S3, Google Cloud Storage, Microsoft Azure, and more. Now you can connect your Alibaba OSS and access its data directly from the IDE.

New file viewer

With the new full-size viewer for remote file systems, you can find more metainfo about folders and files and view file information in a separate panel.

Those are the most notable highlights of our major release. For the full list of new features and enhancements see the What’s new on the JetBrains Big Data tools plugin page. Please share your feedback with us and report any issues you encounter to our issue tracker.

Follow us on Twitter!

The Big Data Tools team

Subscribe to Big Data Tools Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!