Big Data Tools 2022.3: Integration with AWS Glue Data Catalog, Code Completion for SQL Expressions in Zeppelin Notebooks

Anna Maltceva

In this update we’ve added integrations with AWS Glue, Tencent Cloud Object Storage, enhanced Zeppelin notebooks support, and delivered important fixes. Read on to learn about the most important changes in the Big Data Tools plugin or try it right away by installing it to the 2022.3 of IntelliJ IDEA Ultimate, PyCharm Professional, DataSpell or DataGrip. 

Tencent Cloud Object Storage (COS) Support

We’ve supported a new distributed storage service. Now you can work with Tencent COS right from the IDE. Supported features include file versioning with the ability to compare two versions of the same file (Show Diff) and restore the desired version.

AWS Glue Monitoring

Integration with AWS Glue is now supported, letting you monitor your databases, view schemas and partitions, filter data, and customize database views. Learn how to configure a connection to your AWS Glue.

Zeppelin State Viewer Feature

Fine-tune variable introspection for your use case

The State Viewer feature which allows you to preview local variables and SQL schemas for the current Zeppelin session is now enabled by default and is available in local Zeppelin notebooks. It is more customizable, with many new configuration parameters:

Enjoy code assistance for SQL expressions

Code highlighting and completion are now supported for SQL expressions within Spark. We have also provided completion for tables based on data from Hive Metastore and AWS Glue.

Remote File Systems Updates

• Search bar for RFS connections to quickly navigate to a directory or a file within a selected storage.
• You can now provide the sudo password while connecting to an SFTP server to enable access to files with root permissions.
• In the connection settings, you can now disable SSL certificate validation if you trust the server.
• You can now connect to a custom Google Cloud Storage host.

Bulk actions for Kafka topics

You can now select multiple topics and delete or copy them at once. We have also improved the visibility of long values and implemented display of `null` values.

For the full list of new features and enhancements see the changelog on the Big Data tools plugin page. Please share your feedback with us and report any issues you encounter to our issue tracker.
For more updates follow us on Twitter!

The Big Data Tools team

Subscribe to Big Data Tools Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!