Webinar Recording: “Live Development of a PyCharm Plugin” with Joachim Ansorg

This week we hosted a webinar with Joachim Ansorg presenting Live Development of a PyCharm Plugin. The webinar recording is now available, as well as a repo with the contents he showed in the webinar.

w

This webinar covered a tremendous amount of ground in just over an hour:

  • Background and architecture of IntelliJ plugins
  • Development using tests
  • Implementing interesting extension points
  • Viewing the plugin in the IDE (PyCharm)

The chosen topic was interesting in its own right: a plugin to let you suppress flake8 warnings using the standard # noqa comment syntax.

Thanks go to Joachim for spending his time with us and showing how even a small IDE plugin can easily justify the investment by saving developer time. He writes IntelliJ plugins professionally, and based on the feedback from companies we’ve referred to him, he does so quite well.

-PyCharm Team-
The Drive to Develop

Posted in Video, Webinar | Tagged | Leave a comment

Interview: Joachim Ansorg for next week’s PyCharm Plugins webinar

JetBrains IDEs have a lot that’s “integrated” into the “development environment.” At some times, it seems daunting: there’s an infinity of features, with useful new things to learn at every corner. At other times, though, there’s something missing, something unique to what you do or how you do it.

Fortunately the IntelliJ platform for IDEs has a powerful plugin model. In fact, most of PyCharm is done as plugins, either specific to PyCharm or reusable across all IntelliJ IDEs. Getting started can be daunting though, which is why we set up this webinar, to show writing a simple-but-useful PyCharm plugin.

r-2

Our presenter, Joachim Ansorg, does this professionally. To help set the stage for the webinar, we invited him to a brief interview. As a note, here is a longer interview, with video, done in November.

The previous interview goes into more detail, but briefly: tell us about your background, where you live, and how you’re employed?

I’m living in the south of Germany, near the border to Switzerland and France. My wife and I moved here 2.5 years ago. About two years ago, after 8 years as an employee in a small software company, I became self-employed and I’m still enjoying this :) Much of my business is around IntelliJ plugins.

I enjoy software development and most often use Java, Kotlin and Go nowadays. Sometimes a bit of Bash, Scala and tiny bits of C++. Some projects involve Python, but I’m usually reading Python code instead of writing it.

Do you enjoy writing IntelliJ plugins?

I really do! There’s always something new to learn, I’m feeling productive at the end of a day and most of all I’m building something useful and helpful for others.

How hard is it to write a quality, useful plugin?

That depends on the features you want to implement, on your Java/Kotlin skills and on your experience with the IntelliJ platform. Even a tiny plugin can be very useful! I hope that you’ll see this in the webinar.

In my experience the integration of a custom language is the most complex. This kind of plugin is useful, of course, but it’s harder to keep the quality up.

It’s a great help here to have a good test coverage. Luckily the IntelliJ platform comes with good support for this. Tests are run in a headless environment where almost all parts of the IDE are available. For example, you’re able to programmatically interact with an editor, call refactorings, validate parsing and a lot more.

My recommendation is to start small, always add tests for the features you’re working on and use CI to run your tests with all the major builds you’re supporting.

PyCharm plugins aren’t in Python, they’re in Java/Kotlin. How big of a barrier is that?

If you’ve not yet worked with Java or Kotlin before, then it’s probably difficult to get started. I’ve always profited from learning a new language, though. Knowing Java or Kotlin can be a great help sometimes. You’ll be able to use it to improve your personal Python development environment.

Both languages are not that difficult, the APIs of the IntelliJ SDK are probably more overwhelming at first.

What are some of the most interesting integration points?

Personally I like that most things are extensible and that there are only few restrictions. The IDE offers extension points to enable plugins to add new or to customize existing functionality.

Plugins can offer additional extension points to allow further customization by other plugins. For example, the Python support in IntelliJ is implemented as a plugin. This plugin offers some more, Python-only extension points.

The basics like inspections, quick fixes and code completion are very powerful. You can add a lot to the editor if you create a few of these extensions.

It’s possible to make a customized editor. For example, you could implement a graphical editor for your programming language, if you wanted to.

Tell us about some of the interesting customer projects you’ve done.

There’s the integration for kite.com. It integrates the Kite software, a smart and AI-powered tool for Python developers, into PyCharm and IntelliJ Ultimate. The plugin is not available in the repository but comes packaged with the Kite download.

Technically we had to dig deep into the IDE to offer better documentation rendering. For example, at one point we replaced the built-in quick-doc panel with our own in Python files. The panel override isn’t in place anymore but I really liked that this kind of change was possible in a plugin.

Personally I really liked to work on the Tezos plugin. Tezos is an interesting new blockchain. Here I was able to build a new plugin from scratch with good test coverage and a lot of best-practices. In the process I’ve also learned a lot of new things: about the blockchain, about Tezos bakeries, the unusual language Michelson and even a bit about OCaml. The plugin is open-source and available at https://www.plugin-dev.com/project/tezos-michelson/.

In the previous interview you mentioned the productivity gains possible from a highly-tailored plugin. Mind discussing this?

For me a plugin is a tool. A good tool helps you to get your work done. In my opinion a software tool should be out of your way, smart and a time-saver. That’s why I’m using IntelliJ. It helps me to get my tasks done more quickly and with less frustration overall.

If you’re doing something over and over again or if you’re slow at what you’re doing then a tool might help. Write a script or, if you want something more integrated, a plugin.

Custom file formats, custom programming languages or a yet unsupported technology stack might also be a good indicator that a plugin could support you. If there’s more than one developer, then this can easily add up to a larger amount of time.

Just imagine to save 10 minutes a day per developer in a team of 5 for a year. They will get more work done and will thank you that you care. At least I would :)

Let’s take Michelson as an example. It’s an awkward language, at least to an average person like me. In Michelson, there’s an instruction called PAPPAIIR. The instruction PAPPAAIIR is invalid, though. Can you spot the difference? :)

Now, let the Tezos plugin help you out! It’ll highlight the invalid instructions, it’ll show what PAPPAIIR is doing and saves your time by automating the manual tasks like formatting, validating and executing a Michelson file.

It’s an investment at first to build that plugin, but I’m sure that it’ll pay in the long run.

Is it reasonable for companies to learn enough to maintain what you created for them?

That’s a good question.

Maintenance is easier than the implementation. Tasks like the release of an update for the newest version of the IDE should be possible for most. There’s a build script and a good description how to release a new version. A good set of tests helps to make sure that everything is working with the latest major release.

Updates to the IntelliJ SDK may break plugins which were made for previous versions. This doesn’t happen very often, but cases like this usually require some more knowledge of the platform. I can either teach one or more developers of this company about plugin development and maintenance or could fix it when necessary.

What will you be showing in the webinar?

I’ll show how to suppress PyCharm’s warnings when a #noqa marker is present. #noqa is a marker used by flake8 to suppress warnings and errors. If you use #noqa then you usually don’t want to add additional PyCharm’s custom marker to keep it quiet.

This is a fix for https://youtrack.jetbrains.com/issue/PY-8656, which is one of the most-voted-on issues PyCharm’s YouTrack. It’s not that hard to implement, hopefully very helpful for many and a time-saver, too.

The initial version of the plugin will be published after the webinar so that all users of PyCharm will be able to use it soon afterwards.

Posted in Interview | Tagged | 1 Comment

PyCharm 2018.3.3

PyCharm 2018.3.3 is now available. It comes with several Python-related improvements as well as a bunch of platform enhancements.
Get it now from our website.

Improved in This Version

The debugger’s on-demand variable loading policy for NumPy arrays and Pandas DataFrames

on_demand

When the on-demand variable loading policy is enabled in the debug mode, PyCharm doesn’t load NumPy arrays and Pandas DataFrames values by default. Previously, loading all the values caused severe slowdowns in case of large datasets. You can load the values manually from the debugger’s variables view when needed by simply clicking “Show value”.

File encoding specification inspection for Python 3

encodinginspection

In PyCharm 2018.3.3, we’ve added the option to enable the “No encoding specified for a file” inspection for Python 3. This option is useful for those who work on Python 2/3 compatible code.

Further improvements

  • A fix for an extra __init__.py file created when moving a class into another module
  • Several Docker and Docker Compose support fixes
  • Fixes for embedded terminal
  • Many fixes coming from WebStorm and IntelliJ IDEA – see the release notes for details

Interested?

Get PyCharm 2018.3.3 now on our website

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm RC versions and stay up to date. Find the installation instructions on our website. Snap also works for various other Linux distros.

On Windows, macOS, and Linux, you can use our helpful Toolbox App to keep all of your JetBrains IDEs up to date. Read more about the app on our website.

Posted in Release Announcements | Tagged | 1 Comment

PyCharm 2018.3.3 RC and PyCharm 2018.2.7

A PyCharm 2018.3.3 Release Candidate is now available. It comes with several Python-related improvements and platform enhancements, which we want to double-check with you before finally releasing the update. Get it now from our Confluence page.

Apart from releasing this PyCharm 2018.3.3 RC, we’ve also rolled out an update for PyCharm 2018.2. It delivers a fix for lost Django support settings when adding an environment variable; for info on how to update it, skip to the end of this post.

Improved in PyCharm 2018.3.3 RC

The debugger’s on-demand variable loading policy for NumPy arrays and Pandas DataFrames

on_demand

When the on-demand variable loading policy is enabled in the debug mode, PyCharm doesn’t load NumPy arrays and Pandas DataFrames values by default. Previously, loading all the values caused severe slowdowns in case of large datasets. You can load the values manually from the debugger’s variables view when needed by simply clicking “Show value”.

File encoding specification inspection for Python 3

encodinginspection

In PyCharm 2018.3.3, we’ve added the option to enable the “No encoding specified for a file” inspection for Python 3. This option is useful for those who work on Python 2/3 compatible code.

Further improvements

  • A fix for an extra __init__.py file created when moving a class into another module
  • Several Docker and Docker Compose support fixes
  • Fixes for embedded terminal
  • Many fixes coming from WebStorm and IntelliJ IDEA – see the release notes for details

Interested?

Download the RC from our confluence page
If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm RC versions and stay up to date. Find the installation instructions on our website.

The release candidate (RC) is not an early access program (EAP) build and does not bundle an EAP license. To use PyCharm Professional Edition RC, you will need a currently active PyCharm subscription. If none is available, a free 30-day trial will start.

PyCharm 2018.2.7

We’ve fixed a bug with lost Django settings when an environment variable is added for PyCharm 2018.2 (PY-32767). If you’re using PyCharm 2018.2, please update to the new version.

To update: download the new version from our website, choose Help | Check for Updates in the IDE, or use JetBrains Toolbox App to keep all of your JetBrains IDEs updated.

Posted in Early Access Preview, Release Announcements | Leave a comment

Webinar: “Live Development of a PyCharm Plugin” with Joachim Ansorg

Update: Joachim has made a repo available with the contents he will show in the webinar.

PyCharm comes with a lot of functionality, yet perhaps something you’d like is missing. Luckily it’s made to be enhanced by plugins to be suitable for a much wider range of users. This webinar with noted IntelliJ/PyCharm plugin consultant Joachim Ansorg will show you how such a plugin is built for PyCharm.

  • Tuesday, January 15, 2019
  • 6:00 PM – 7:00 PM CET (12:00 PM – 1:00 PM EST)
  • Register here
  • Aimed at intermediate/advanced PyCharm users

r-2

Speaking to You

Joachim is a self-employed software developer, specializing in IntelliJ plugin development.

Born in 1982, he studied CS at a small university in Germany. After that he worked for a year as a software developer on the beautiful island of Crete in Greece. He failed to learn Greek there but is able to read and write the simpler alternatives like Java, Kotlin, and Go.

When he moved back to Germany he joined a smaller software company for about 8 years as a Java developer. He became self-employed in early 2017 to create plugins for IntelliJ and is enjoying this a lot.

Joachim is happily married to his wife Lisa and is living in southern Germany, about 15 minutes away from Basel/Switzerland.

Posted in Webinar | 1 Comment

PyCharm 2018.3.2

PyCharm 2018.3.2 is now available. This version comes with a couple of small improvements. Get it now from our website.

Improved in This Version

PyCharm splits F-strings for you

In PyCharm 2018.3, we improved a lot of things in F-strings, and we’ve just improved them a little further. When splitting an F-string in previous versions of PyCharm, this is the behavior:

F-strings 2018.3.1

Although this behavior is in line with what you’d expect from a text editor, wouldn’t it be nice if PyCharm could help you a little further? So we’ve made sure that breaking up an F-string in PyCharm 2018.3.2 leaves you with a valid Python file:

F-strings 2018.3.2

Further improvements

Interested?

Get PyCharm now on our website

If you’re on Ubuntu 16.04 or later, you can use snap to keep your PyCharm up to date. You can find the installation instructions on our website. Snap also works for various other Linux distros.

On Windows, macOS, and Linux, you can use our helpful Toolbox App to keep all of your JetBrains IDEs up to date. Read more about the app on our website

Posted in Release Announcements | Tagged | Leave a comment

Webinar Recording: “Automating Build, Test and Release Workflows with tox” with Oliver Bestwalter

Last Thursday we had a special webinar in our Munich offices with Oliver Bestwalter, talking about tox and release automation. The recording is now available.

w

Many thanks to Oliver for covering quite a lot of material with a good audience in just over an hour. I found the webinar very valuable: lots of things I need to know, that would be hard to figure out on my own, conveniently packaged and delivered by the maintainer of tox.

If you have any questions for us or for him, please feel free to leave comments below on this blog post.

-PyCharm Team-
The Drive to Develop

Posted in Video, Webinar | Tagged | Leave a comment

PyCharm 2018.3.2 RC

PyCharm 2018.3.2 Release Candidate is now available. This version comes with a couple of small improvements. Get it now from our Confluence page.

Improved in This Version

PyCharm splits F-strings for you

In PyCharm 2018.3, we improved a lot of things in F-strings, and we’ve just improved them a little further. When splitting an F-string in previous versions of PyCharm, this is the behavior:

F-strings 2018.3.1

Although this behavior is in line with what you’d expect from a text editor, wouldn’t it be nice if PyCharm could help you a little further? So we’ve made sure that breaking up an F-string in PyCharm 2018.3.2 leaves you with a valid Python file:

F-strings 2018.3.2

Further improvements

Interested?

Download the RC from our confluence page

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm RC versions and stay up to date. You can find the installation instructions on our website.

The release candidate (RC) is not an early access program (EAP) build, and does not bundle an EAP license. To use PyCharm Professional Edition RC, you will need a currently active PyCharm subscription. If none is available, a free 30-day trial will start.

Posted in Early Access Preview | Tagged | Leave a comment

Reminder: Webinar this Thursday, “Automating Build, Test and Release Workflows with tox” with Oliver Bestwalter

Interested in testing and release automation? We’re doing a webinar with Oliver Bestwalter of tox fame, this Thursday. Join us to learn more about test isolation with tox as well as how it fits in with a larger ecosystem of repeatable release processes.

  • Thursday, December 13
  • 4:00 PM – 5:00 PM CET (10:00 AM – 11:00 PM EST)
  • Register here
  • Aimed at intermediate Python developers

Webinar Bestwalter Register

Agenda

We will look at what is necessary to automate all important workflows involved in building, testing and releasing software using tox.

We’ll cover how to use tox to …

  • run static code analysis, automatic code formatting/fixing as a separate stage orchestrated by the pre-commit framework
  • run tests with pytest
  • measure and report test coverage
  • build and upload packages to pypi/devpi/artifactory

All this can be run and debugged locally from the command line or programmatically.

Posted in Webinar | Leave a comment

Interview: Oliver Bestwalter for tox webinar next week

Python has long distinguished itself with a culture of testing. In the last decade, two libraries have combined to give powerful testing in isolation — pytest and tox. The latter combines easily with pytest to give you a clean environment across test runs, including across multiple versions of Python.

tox certainly counts as one of those things lots of PyCharm customers know they should know, but don’t yet know. To make it easy to break the ice we’ve invited Oliver Bestwalter to introduce tox in a PyCharm webinar. Oliver is the maintainer of tox and advocate for release automation in projects.

Webinar Bestwalter Register

Here’s our interview with Oliver. As background, Oliver was previously interviewed on The Mouse vs. Python blog and the Test & Code podcast.

Give us a sneak peak on what you’re going to discuss in the webinar and what audience it is aimed at.

I want to do a time lapse journey through a hypothetical project that grows tests and automation over its lifetime. The focus will be on how to use tox to bundle all these development and automation tasks into a workflow that works well locally and on CI.

Let’s go back in time. Can you give us your Python origin story?

The short version is: I stumbled over the tutorial in 2006 and fell in love. The longer version is in the interview you linked :)

You are a core maintainer of tox. What’s it like running a large, popular open source project?

When I joined the project, Holger Krekel (the original author) and the first generation of contributors had started to move on to other projects. Work around tox had cooled down, so I thought I’d help out a bit. There were quite a few open pull requests and issues that needed addressing. So I did that and after a while Holger asked me if I want to handle releases as well. I also made contact with plugin authors to gather tox plugins into the tox-dev organization to raise their visibility. My main focus was on keeping the project alive, fixing annoying bugs and creating a welcoming atmosphere. That’s a lot of work that doesn’t result in very much code, but I still think it was the right thing to do rather than to just hack away at the code.

Over the past few months I wasn’t quite able to even follow what is going on in the issue tracker for a number of private and professional reasons. At the moment my activities are concentrated more on teaching Python with a focus on testing and automation (with pytest and tox obviously). Luckily Bernát Gábor joined as a maintainer this year and is currently very active in the project (thank you Bernát!).

So to answer the question: in the beginning it was a good feeling to keep an important project alive and improve its processes, but for quite a while now it has also meant feeling guilty a lot of the time, because I can’t do more “direct” work for tox a lot of the time.

Testing is one part of your release automation vision. Can you discuss how other parts — pre-commit hooks, black, CI — fit together?

I think of all test and automation tools as helpers to build lines of defense (against bugs) in a development process. The first line of defense is static code analysis. Ideally the most obvious problems are already pointed out directly in the editor when they happen. PyCharm inspections and quick fixes are great for that if you crank them all up to 11. The next line of defense are linters and automatic fixers (like flake8 and black). The pre-commit framework wraps that and more into a neat package that can prevent you from accidentally committing code that has obvious problems. If you don’t use it locally the first stage in CI will fail where the same checks are run. The next line of defense are automatic tests. Depending on the nature of the project there might be more (like running tests against the deployed environment on different stages).

The role that CI plays for me is quite simple: CI is just a task execution and report collection tool and should contain as little process knowledge as possible. So nothing fancy should happen there that isn’t easily reproduced on a developer box. CI simply runs everything that the developers run locally and makes artifacts available for release. This means that the process knowledge needs to be encoded in the tox configuration and the automation scripts that live alongside the production code. The real world is often a bit more complicated and this isn’t always completely possible (often due to security considerations) but that’s the aim.

What’s been your experience using an IDE like PyCharm to visualize testing and the other pieces?

I am with Brian Okken on this: PyCharm is a really good GUI for pytest (see or better listen: Episode 48 of Test & Code)

The Test runner tab provides a good overview of the tests that were run and I also appreciate the many ways how I can navigate from the results back in to the relevant code sections. What I appreciate even more is the convenience of running the tests right from the code rather than having to switch to the command line. Also: auto-test – very handy. It’s also great that PyCharm even supports code completion and navigation for pytest fixtures in recent releases.

Look ahead 3 years from now. What’s next for tox and the release automation field?

My crystal ball is at the repair shop at the moment, but I’ll try :)

In respect to tox that really depends who steps up to the plate and champions the development of new features and improvements. If I get the chance, I want to improve interpreter discovery and squash some bugs. I also want to improve the documentation and make tox more accessible to newcomers. Regarding the recent changes in packaging (PEP-517 and PEP-518) tox already has everything in place. Projects using flit and other tools that leverage the sudden freedom from setup.py and setuptools can use tox, but I am sure there will be some kinks to iron out along the way. tox also needs to grow native support for pyproject.toml (meaning that you won’t need a tox.ini anymore and configure it there alongside all the other tools). In three years tox will also be 100% bug-free and powered by renewable energy :)

Regarding the greater landscape I really hope that the development will go even further into the direction we are already heading: making build, test and deploy automation as easy and accessible to the masses as possible. I’ll do my best to help with that.

Posted in Interview | Leave a comment