Big Data Tools 2021.3 Is Now Available

Oleg Chirukhin

We’ve just released a new build of the Big Data Tools plugin that is compatible with the 2021.3 version of IntelliJ IDEA Ultimate, PyCharm Professional, and DataGrip. The plugin also supports our new data science IDE, DataSpell.

In this release, we’ve introduced a number of changes to the user interface, added a bunch of features, and fixed a long list of bugs. Let’s take a closer look.

Major notebook UI overhaul

The most important thing you need to know about this release is that we totally re-designed all Zeppelin-related parts of the UI. It now looks much slicker and can do a lot more.

• The visual design of the whole interface has changed. Check out the screenshots below for examples.
• HTML rendering now works fine with any HiDPI display.
• We improved the stability of remote notes and added the synchronization status indicator to the UI. A blue strip on the left side of the editor identifies paragraphs that you haven’t made any changes to:
  
  
  Now let’s change something:
  
  
  As you can see, the strip turns yellow, and it stays that way while Zeppelin synchronizes. You’ll see the blue strip again when synchronization is complete:
  
  

• With the new Notebook Settings tab, you can toggle info about the notebook’s author and run dates, convert text output into a table, and toggle paragraph settings (folding, line numbering, wrapping, and confirmations).
  
  
  
• In Big Data Tools 1.3 you can create custom color schemes and preview them in Charts Settings.
  
  
  
• In the new UI, paragraphs and outputs are displayed as vertical blocks. It looks very similar to what you see in the Zeppelin web interface.
  
  Before the change:
  
  
  
  After the change:
  
  
  
• In previous versions, it was hard to figure out how to add new paragraphs, at least at first glance. The actions for doing so were hidden among other options inside the context menu.
  
  
  
• Now you can add a paragraph by hovering the mouse over any paragraph and using the large much bigger button that appears.
  
  
• You can still use the Ctrl+Alt+A and Ctrl+Alt+B hotkeys to add paragraphs, as well.
  
• The other most important paragraph actions are grouped under the smaller three dots button:
  
  
  

• Working with cell headers could sometimes be a bit awkward. The only way to change them was to interact with a separate dialog window hidden under the context menu. It felt as though the title was not an organic part of the paragraph.
  
  
  
  Now, paragraph titles have been given a new UI and they can be renamed right in the editor. Just click on the title and edit it as plain text.
  
  
  

• A new Clone Paragraph action is now available from the pop-up menu in the editor.
  
  
  

• All the rarely used actions are now grouped under the More Actions three dots button:
  
  
  
  For example, here you can find the option to export your code (without outputs) as an HTML file.
  
• Previously you were presented with a choice: you could display either tabular data or graphical data.
  
  
  
  With the new split mode, you can display tables and charts on the same screen.
  
  
  
• Additionally, you can use the folder icon button to open a table in a separate editor tab.
  
• On the same toolbar, you can click the DDL button to get the scheme in the SQL and JSON formats:
  
  
  
• In previous releases, excessively long lines of text may have occasionally caused headaches:
  
  
  
  Now you have an option to enable line wrapping in text outputs:
  
  

Big Data Tools tool window

The main tool window has also been improved.

• Amazon S3 connections now support SSH tunneling:
  
  
  
• You can now select multiple connections and remove them with a single click.
• The export and import actions for Zeppelin notes have been renamed to Save to Disk and Upload from Disk.
• Last but not least, you can now open the BDT Panel with a keyboard shortcut: 
• MacOS: Ctrl+Shift+Cmd+B
• Windows, Linux: Ctrl+Alt+Shift+B

Bug fixes and improvements

We’re actively developing and enhancing the Big Data Tools plugin and doing our best to take all your feedback into account so we can fix as many bugs as possible. Here’s a list of the most important fixes:

Zeppelin:

• Python: we’ve fixed the Install package button on MacOS.
• Notes can be opened from any file storage server as notes, rather than as JSON files.
• The problem with the notification about the interpreter’s restart status has been fixed.
• Shift+Shift (Search Everywhere) has been fixed
• We’ve fixed some table-specific bugs: 
  • Filter search works much better.
  • We’ve improved pagination support.
  • We’ve improved floating point value visualization.
  • Problems affecting search results have been resolved.

Remote file systems:

• ​​ORC, AVRO, and Parquet files without extensions are now reopened correctly when the IDE starts up.
• Several problems preventing Parquet files from opening have been fixed.
• The problem that caused the same content to be displayed for two different Parquet files with identical names has been resolved.
• We’ve added notifications that appear when opening files from the local driver and remote file storage in read-only mode.
• We’ve fixed the exception that would occur when showing file info on Parquet via SSH.

Big Data Tools tool window:

• Toolbar buttons work correctly when there are no connections.

Connection settings:

• GCS support for public buckets has been improved with more convenient error messages
• We’ve improved validation for file storage connections.
• We’ve improved the clarity of some of the settings error messages.

Spark and Hadoop monitoring:

• Spark and Hadoop monitoring filters can now be applied without refreshing the corresponding connections in the BDT tool window.

You can visit the What’s New section of the plugin page for an overview of the key recent improvements, or you can check out the full report on YouTrack, which includes just about all the release notes.

You can upgrade up to the latest version of the Big Data Tools plugin from your browser, from the plugin page, or right in the IDE. We are always eager to hear your feedback. Feel free to leave your thoughts and suggestions on the plugin page.

Documentation and social networks
We’re actively developing and enhancing the Big Data Tools plugin, and we’re doing our best to process your feedback and fix as many bugs as possible.

For an overview of the key recent improvements, please visit the What’s New section of the plugin page, where you’ll also find the release notes.

If you’re looking for more information about how to use any of the plugin’s features, check out the IntelliJ IDEA, PyCharm, DataGrip, or DataSpell documentation. Need more help? Leave us a message here in the comments or on Twitter.

We hope that these improvements will help you tackle bigger and better things – and have more fun in the process! Thank you for using the Big Data Tools plugin!

The Big Data Tools team

Subscribe to Big Data Tools Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!