Announcing PyCharm 2016.1.4

PyCharm 2016.1.4 has been uploaded and is now available from the download page. It also will be available soon as a patch update from within the IDE.

The Release Notes lists all fixes for this update. This build doesn’t have any additional fixes comparing to the PyCharm 2016.1.4 Release Candidate build, so here’s the recap:

  • numerous performance improvements along with fixes for various hangs
  • a number of fixes for Docker-based python interpreters and Docker Compose support
  • a fix for exception while renaming a string format parameter
  • and much more

Download PyСharm 2016.1.4 for your platform from our website and please report any problem you found in the Issue Tracker.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in comments to this post and on twitter.

The Drive to Develop
PyCharm team

Posted in Release Announcements | Leave a comment

Python Developers Survey 2016: Findings

Recently we conducted a survey among more than 1,000 Python developers to identify the latest trends and gain insight into how the Python development world looks today.
We’re excited to share some of these results with you.

PC_Developer_Survey2016

It turns out Python is becoming the language of choice for more and more developers. At the same time time Python 3 is gaining popularity and is expected to overtake Python 2 very soon. Another interesting finding is that a huge part of Python developers appear to be engaged in scientific development and data analysis, which correlates with results from other independent research showing that the number of scientific Python developers is growing steadily and is projected to catch up with web development in the foreseeable future. Discover many more interesting findings from the Python world in our detailed report.

As usual we’re anxious to hear your feedback and discuss these survey findings in the comments to this blog post and on twitter.
_
Your PyCharm Team
The Drive to Develop

Posted in Cool Feature, Survey | Leave a comment

PyCharm 2016.1.4 RC is Available

Today we announce the PyCharm 2016.1.4 Release Candidate build #145.1503. The list of bug fixes and improvements for this build can be found in the release notes.

Some highlights of the PyCharm 2016.1.4 RC are:

  • numerous performance improvements along with fixes for various hangs
  • a number of fixes for Docker-based python interpreters and Docker Compose support
  • a fix for exception while renaming a string format parameter
  • and much more

Patch updates from the previous PyCharm releases should be available shortly.

Please give PyCharm 2016.1.4 RC a try before its official release and please report any bugs and feature request to our issue tracker.

-PyCharm team

Posted in Early Access Preview | 2 Comments

Announcing PyСharm 2016.1.3 with important fixes

PyCharm 2016.1.3 is now available from the download page. It also will be available shortly as a patch update from within the IDE.

Some notable highlights of this update include:

  • A number of fixes for Docker-based interpreters support
  • A lot of fixes for Docker Compose support
  • Fixes for various code inspections
  • Fixes for Debugger
  • Various performance improvements for the code insight and navigation engines
  • And much more

The list of numerous bug fixes and improvements for this build can be found in the Release Notes.

In addition to the usual functionality fixes, this update contains important fixes for two security issues inside IntelliJ Platform. The vulnerabilities, in various forms, are also present in older versions of PyCharm; therefore, patches for those are also available.

Find more information about the issues, the affected versions and ways to update in this special blog post: http://blog.jetbrains.com/blog/2016/05/security-update-for-intellij-based-ides.

It is strongly recommended for all users to install the update as soon as possible.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in the comments to this blog post and on twitter.

Your PyCharm Team
The Drive to Develop

Posted in Release Announcements | 2 Comments

Debugger Interview with PyDev and PyCharm

PyCharm’s visual debugger is one its most powerful and useful features. The debugger got a big speedup in the recent PyCharm, and has an interesting backstory: JetBrains collaborated with PyDev, the popular Python plugin for Eclipse, and funded the work around performance improvements for the common debugger’s backend.

To tell us more about the improvements as well as cross-project cooperation, we interviewed the principles: Fabio Zadrozny, creator of PyDev, and Dmitry Trofimov, Team Lead for PyCharm.

Let’s jump right into it. Tell us about the speedups in the latest pydevd release.

FZ: The performance has always been a major focus of the debugger. I think that’s actually a requisite for a pure-python debugger.

To give an example here: Python debuggers work through the Python tracing facility (i.e.: sys.settrace), by handling tracing calls and deciding what to do at each call.

Usually a debugger would be called at each step to decide what to do, but pydevd is actually able to completely disable the tracing for most contexts (any context that doesn’t have a breakpoint inside it should run untraced) and re-evaluate its assumptions if a breakpoint is added.

Now, even having the performance as a major focus, the latest release was still able to give really nice speedups (the plain Python version had a 40% speed improvement overall while the Cython version had a 140% increase).

I must say that at that point, there weren’t any low-hanging fruits for speeding up the debugger, so, the improvement actually came from many small improvements and Cython has shown that it can give a pretty nice improvement given just a few hints to it.

DT: The performance of the debugger was one of the top voted requests in PyCharm tracker. The latest release addresses this by implementing some parts of the debugger in Cython, which leads to huge performance improvements on all type of projects.

Was the Cython decision an easy one?

FZ: Actually, yes, it was a pretty straightforward decision…

The main selling point is that the Cython version is very similar to the Python version, so, the same codebase is used for Cython and plain Python code — the Cython version is generated from the plain Python version by preprocessing it with a mechanism analogous to #IFDEF statements in C/C++.

Also, this means that with the same codebase it’s possible to support CPython (which can have the Cython speedups) while also supporting Jython, PyPy, IronPython, etc. I even saw someone post about the debugger being used in a javascript implementation of Python.

DT: The idea was to make the debugger faster by rewriting the bottlenecks in C, but at the same time optional to have any compiled binaries, so that pure Python version would still work. Also, it was desirable to have as little code duplication as possible. Cython let us do all that perfectly, so it was a natural decision.

Let’s take a step back and discuss the 2014 decision to merge efforts. How did this conversation get started?

FZ: I was doing a crowdfunding for PyDev which had a profiler as one of its main points, which was something that PyCharm wanted to add too. Although the initial proposal didn’t come through, we started talking about what we already had in common, which was the debugger backend and how each version had different features at that point. I think PyCharm had just backported some of the changes I had done in the latest PyDev version at that time to its fork, and we agreed it would be really nice if we could actually work in the same codebase.

DT: We have used the fork of Pydev debugger since the beginning of the PyCharm and occasionally I would check what was going in Pydev branch to backport features and fixes from there to PyCharm. Meanwhile, Fabio does the same, taking the latest fixes from PyCharm branch. As time passed and branches diverged, it was getting more and more difficult to compare the branches and backport fixes from one another.

After one of the tough merges, I thought, maybe we’d better create a common project that would be used in both IDEs. So I decided to contact Fabio and was very happy when he supported the idea.

Did the merging/un-forking go as you planned, or were there technical or project challenges?

FZ: The merging did go as planned…

The main challenge was the different feature set each version had back then. For instance, PyDev had some improvements on dealing with exceptions, finding referrers, stackless and debugger reload, whereas PyCharm had things such as the multiprocessing, gevent and Django templates (and the final version had to support everything from both sides).

The major pain point on the whole merging was actually on the gevent support, because the debugger really needs threads to work and gevent has an option for monkey-patching the threading library, which made the debugger go haywire.

DT: The main challenge was to test all the fixes done for the PyCharm fork of the debugger for the possible regressions in the merged version. We had a set of tests for debugger, but the coverage, of course, wasn’t 100%. So we made the list of all debugger issues fixed for the last 3 years (around 150 issues,) and just tested them. That helped us to ensure that we won’t have regressions in a release.

Fabio, how did it go on your end, having JetBrains sponsor some of your work? Any pushback in your community?

FZ: I must say I didn’t really have any pushback from the community. I’ve always been pretty open-minded about the code on PyDev (which was being used early on in PyCharm for the debugger) and I believe IDEs are a really personal choice. So I’m happy that the code I did can reach more people, even if not directly inside PyDev. Also, I think the community saw it as a nice thing as the improvements in the debugger made both, PyDev and PyCharm, better IDEs.

The Python-oriented IDEs likely have some other areas where they face common needs. What do you think are some top issues for Python IDEs in 2016 and beyond?

FZ: I agree that there are many common needs on IDEs — they do have the same target after all, although with wildly different implementations 😉

Python code in particular is pretty hard to analyze in real-time — which contrasts with being simple and straightforward to read — and that’s something all “smart” Python IDEs have to deal with, so, there’s a fine balance on performance vs. features there, and that’s probably always going to be a top issue in any Python IDE.

Unfortunately, this is probably also a place where it’s pretty difficult to collaborate as the type inference engine is the heart of a Python IDE (and it’s also what makes it unique in a sense as each implementation ends up favoring one side or the other).

DT: The dynamic nature of Python was always the main challenge for IDEs to provide an assistance to developers. A huge step forward was done with Python 3.5, by adding a type hinting notation and typeshed repository from which we will all benefit a lot. But still this thing is in its early stage and we need to define and learn effective ways to adopt type hinting.

Python performance is also a challenge. In the Python world, when you care about performance, you switch from using pure Python to libraries written in C, like numpy. Or you try pypy. But in both cases performance and memory profiling becomes hard or even impossible with current standard tools and libraries. I think that tool developers can collaborate on that to provide better instruments for measuring and improving the performance of Python apps.

What’s in the future for pydevd, performance or otherwise?

FZ: I must say that performance wise, I think it has reached a nice balance on ease of development and speed, so, right now, the plan is not having any regression 😉

Regarding new development, I don’t personally have any new features planned — the focus right now is on making it rock-solid!

DT: One of the additions to pydevd from the PyCharm side is the ability to capture the types of the function arguments in the running program. PyCharm tries to use this information for code completion, but this feature now is optional and off by default. With the new type hinting in Python 3.5 this idea gets a new spin and the types collected in run-time could be used to annotate functions with types or verify the existing annotations. We are currently experimenting only with types, but it could be taken further to analyse call hierarchy etc.

Posted in Uncategorized | 2 Comments

PyCharm 2016.1.3 RC is available

It’s been a while since we released PyCharm 2016.1.2 and today we announce the Release Candidate build of PyCharm 2016.1.3.

Some highlights of the PyCharm 2016.1.3 RC build 145.969 are:

  • A number of fixes for Docker-based interpreters support
  • Fixes for Docker Compose support
  • Fixes for false positive warnings in strings inspections
  • Various performance improvements for the code insight engine
  • And much more

The list of numerous bug fixes and improvements for this build can be found in the Release Notes.

We encourage you to give PyCharm 2016.1.3 RC a try before its official release. Please change the updates channel to “Early Access Program” in the updates settings (Help|Check for updates…) to get the patch update to the EAP build from the recently released PyCharm 2016.1.2 or download the full installer for your platform. Please report any bugs and feature request to our issue tracker.

-PyCharm team

Posted in Early Access Preview | 1 Comment

PyCharm Migration Tutorial for Text Editors

If you’re a Python developer who uses a text editor such as Vim, Emacs, or Sublime Text, you might wonder what it takes to switch to PyCharm as an IDE for your development. We’ve written a helpful Migrating from Text Editors tutorial for just this topic.

The tutorial starts with the basic question of “What is an IDE?” The line between text editor and IDE can be blurry. PyCharm views the distinction as: a project-level view of your code and coding activities, with project-wide features such as coding assistance and refactoring.

This document then goes over some of the important points when migrating: the project-oriented UI, working with projects instead of files, Vim/Emacs specifics, keyboard shortcuts, customizing, and a discussion of facilities important to text editor users (multiple cursors, split windows, etc.) It then closes by discussing areas the IDE can really help, for example, the managed running and debugging of your code.

Of course, this document is just an overview. Vim and Emacs in particularly have decades of development and features, and PyCharm itself is now very mature with many features itself, so a complete comparison would break the Internet. If you have a specific question, feel free to comment, and we hope you find the tutorial helpful.

Posted in Tutorial | Leave a comment

In-Depth Screencast on Testing

Earlier this year we rolled out a Getting Started Series of screencast videos on the basics of using PyCharm: setup, the UI, running Python code, debugging, etc. We knew at the time that some topics would need more treatment than a quick screencast, so we planned a follow-on series of “in-depth” screencasts, each on a single topic.

Here’s the first: In-Depth: Testing covers more topics than we went over in the Getting Started: Testing screencast. Which makes sense, as PyCharm itself has such a wealth of support in each of its features:

Here are the topics in this just-over-four-minute video:

We hope you enjoy this first in the series of In-Depth screencasts. We have more planned, such as version control. And please, if you have any topics that you’d like to see get expanded screencast attention, let us know.

Posted in Screencast, Tutorial, Video | 4 Comments

Announcing PyCharm 2016.1.2

PyCharm 2016.1.2 is now available from the download page.

Soon it will also be available as a patch update from within the IDE. A patch update from v2016.1.1, which exposed a severe performance regression, will be available shortly today. The patch update from v2016.1 is still under way and will be available tomorrow.

The Release Notes lists all fixes for this update.

We bring you this build shortly after releasing 2016.1.1 because of the important bug-fix, PyCharm freezes on large string literals [PY-19075].

Other fixes for various Docker Compose support problems and a fix for extra libraries indexing are currently in progress and planned for the next bug-fix update.

Download PyСharm 2016.1.2 for your platform from our website and please report any problem you found in the Issue Tracker.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in comments to this post and on twitter.

Your PyCharm Team
The Drive to Develop

Posted in Release Announcements | 16 Comments

Announcing PyCharm 2016.1.1

Today we are pleased to announce the PyCharm 2016.1.1 bug-fix update. It has been uploaded and is now available from the download page. It also will be available shortly as a patch update from within the IDE (from PyCharm 2016.1 only).

Some notable highlights of this update include:

  • Fixed wrong indentation after copy-paste
  • Fixed generation of reStructuredText doc strings for methods with parameters
  • Fixed folding of custom regions
  • Fixed numerous false positives in formatted strings
  • Fixed wrong highlighting of parameters in formatted strings
  • A fix for a problem with “Gevent compatible” debugger’s mode

Fixes for various Docker Compose support problems and a fix for extra libraries indexing are currently under way and will get to the next bug update.

For further details on the bug fixes and changes, please consult the Release Notes.
As usual, please report any problem you found in the issue tracker.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in the comments to this blog post and on twitter.

The Drive to Develop
PyCharm Team

Posted in Release Announcements | 16 Comments