PyCharm 2016.2 EAP 2 (build 162.917)

Two weeks have passed since the first EAP build of the PyCharm 2016.2. The PyCharm team has returned from PyCon 2016 in Portland, and today we are happy to present you the new PyCharm EAP build, with many new features and bug-fixes inside.

Here are some highlights.

vmprof Profiler Support 

For those who care about performance, the PyCharm Professional Edition provides profiling support. It allows running Python code with the cProfile or Yappi profiler enabled, viewing the profiling results in the execution statistics report, and seeing a graph visual representation.

While being very useful in many cases, cProfile and Yappi have their limitations. First, they are tracing profilers, which means that while measuring the duration of function calls, they add an overhead to every call making the final result sometimes inaccurate. Second, they collect the execution statistics for functions, not distinct calls of this functions. Thus, while it is possible to present results in a graph form, there is not enough data to view the call tree, which can be sometimes very informative and helpful.

This all made us support another Python profiler: vmprof.

vmprof is a sampling (statistical) Python profiler written in C (so it has minimal overhead), developed by members of the PyPy team. It works for Python versions 2.7 through 3.5, CPython as well as PyPy, and supports Linux, Mac OS X and Windows.

To launch the vmprof profiler from PyCharm, all you need is to have it installed on your selected Python interpreter and then profile your run configuration.

Screen Shot 2016-06-16 at 15.46.48

After that, you will see the normal profiling report but with an additional “Call Tree” tab.

Screen Shot 2016-06-16 at 14.16.22

The only limitation is, being a statistical profiler, vmprof-python needs some time to gather enough samples, so it may not work for very short (less than 10ms) scripts.

Configuration for Optimize Imports 

The new dedicated setting for “Optimize Imports” actions allows you to configure how imports are sorted and arranged in groups.

Screen Shot 2016-06-16 at 15.40.25

Thread Suspend Option

Previously with the PyCharm debugger, on reaching a breakpoint during execution, only the corresponding thread was suspended and others were still running. Now there is a suspend option with two values: All and Thread. ‘All’ makes all threads to stop at once whenever any breakpoint hit. With the ‘Thread’ option, only the corresponding thread stops.

Screen Shot 2016-06-16 at 15.22.47

Pandas Dataframes Viewer 

PyCharm has had a special array viewer for NumPy arrays since version 4.0, but it didn’t support pandas dataframes, which recently has started to be a highly voted feature. The new EAP enables the array viewer for pandas dataframes as well. Thanks to our user Yuli Fiterman, who contributed this feature!

Screen Shot 2016-06-16 at 14.48.42

Git/Mercurial Log viewer

The viewer now loads in the background on every change (a commit, fetch, rebase, etc). As a result, it’s always ready when you open it.

While the loading is in progress, there’s a thin stripe shown under the toolbar.idea_2016_2_log_progress

Custom Background Image for Every Project

While it may seem as something that doesn’t matter for productivity at all, a feature is still nice to have. Sometimes appearance is everything.

With the ‘Set Background Image’ (use Find action Shift-Cmd-A) action, you are able to set any image file as a background for PyCharm.

Screen Shot 2016-06-16 at 16.36.47

And now you can set different backgrounds for different projects!

That’s all for today, stay tuned!

Download PyCharm 2016.2 EAP build 162.917 for your platform from the project EAP page or update right from the previous PyCharm EAP installation with a patch update within the IDE. Please report any bugs and feature requests to our Issue Tracker, your feedback is very important for us to make PyCharm 2016.2 better.

Posted in Early Access Preview | 6 Comments

PyCharm 2016.2 EAP is Open

Today we’re excited to announce that the first PyCharm 2016.2 Early Access Preview (EAP) build 162.646 is finally available for download! Head on over to the PyCharm Early Access Preview (EAP) page and have a look at the download section where you can get it for your platform right now.

PyCharmPro_splash2016_2EAP

What is the Early Access Program (EAP)?
We at JetBrains believe that making tools for developers should greatly involve listening to developers. Early Access Programs allow you to try pre-release versions of our software to evaluate features that will be added to the next release, closely participate in discussions about PyCharm, and influence the release from early stages onwards.

Once you’ve downloaded the latest EAP build (from its official page), you’ll start getting updates with new features and improvements every week (to receive notifications about updates, make sure to set “Early Access Program” in your update settings). Your feedback is welcome and very much appreciated in our issue tracker or in the comments section of this blog post.

PyCharm 2016.2 EAP introduces a few new features, bug-fixes and improvements that are unique to PyCharm, as well as a lot of features and improvements from the Web and IntelliJ Platform sides. Here are some of the highlights:

  • Python Related Improvements
    • Function return values are now shown in the PyCharm’s debugger after stepping out from a function
    • Postfix code completion has been improved with the “main” and “print” postfix templates extracting your [expression].main and [expression].print code snippets respecting Python 2 and Python 3 versions
    • A number of fixes related to parsing requirements.txt files
  • General
    • The Inspection tool window has been redesigned for better ergonomics: now code location for each of the problems is displayed along with controls to either apply a quick-fix or suppress inspection
    • More friendly popup IDE notifications in the rightmost bottom corner which are now more compact, and grouped by the subsystem they came from
    • The brand new Set Background Image option that makes your IDE look different by choosing a background image for the editor and the IDE frame
    • The installer for Windows now comes with our custom JDK build where we address some issues with focus and font rendering
  •  Editor
    • Better spell-checking suggestions based on Damerau–Levenshtein distance
    • Support for font ligatures (highly voted request)
    • Coding assistance for Regex has been extended with better error reporting and added code completion for named group references
    • Language injection now works in YML files
  •  Version control integration
    • The Commit dialog now shows unversioned files (so you don’t forget to commit something important)
    • Patches can now be applied from the clipboard (the IDE will offer to apply the patch when it obtains focus), or by dragging a file into the IDE
    • If a file has been moved or renamed since a patch was created for it, the IDE will try to figure that out or let you manually choose the file to apply the patch to
    • The VCS log continues getting better with a handy loading progress indicator, a shortcut to navigate to the filter field, and tooltips for graph arrows
    • The Git Log can now display commit details for several selected commits (this works for Git and Mercurial)
    • Brand new Apply Patch dialog to apply the changes from patch using a Merge-like dialog
  •  Web development improvements
    • Small but neat improvements for TypeScript support
    • You can now configure whether you want IDE to add single, double or no quotes at all when autocompleting attributes in HTML and JSX
    • You can now drag a JavaScript or CSS file into the opened HTML file, and PyCharm will automatically create a script or a link tag inside the head tag for you.
    • Thanks to the integration with Angular CLI, you can now create new Angular 2 projects with ease. Install angular-cli globally via npm and create a new project in PyCharm.
    • Grunt, gulp or npm tasks as a Before launch task in a Run/Debug configuration

Download PyCharm 2016.2 preview build for your platform right from the project EAP page, and please report any bugs and feature request to our Issue Tracker.

-PyCharm Team
The Drive to Develop

Posted in Early Access Preview | 8 Comments

Meet the PyCharm Team at PyCon 2016

May 28th – June 5th, the JetBrains PyCharm Team will be in Portland, Oregon for PyCon 2016. As usual, we sponsor the event and will have a JetBrains PyCharm booth in the Expo Hall during the main conference days.

pyconlogo

The show represents a great opportunity to meet a large part of the PyCharm Team, learn about current developments, watch a live demo or just say hi. We invite you to stop by our booth with your questions and chat about your experiences with PyCharm and other JetBrains tools. We will be raffling PyCharm licenses so be sure to register and grab some of our cool giveaways.

PyCharm team members are going to attend two very important satellite events: Python Language Summit (May 28th) and Python Education Summit (May 29th). Andrey Vlasovskikh will give a talk on PEP 484 and Type Hinting adoption. Dmitry Filippov and Anna Morozova will be at Education Summit to talk about PyCharm Edu and current JetBrains educational initiatives. If you’ll be there, feel free to find and chat with us about latest trends in the Python world.

Also, some of our developers will join the Development Sprints (June 2nd – 5th). Would you like to join us or invite to your own sprint? Come to our booth to discuss things!

We’re looking forward to meeting you at PyCon!

-PyCharm team

 

Posted in Conferences & Events | 1 Comment

Announcing PyCharm 2016.1.4

PyCharm 2016.1.4 has been uploaded and is now available from the download page. It also will be available soon as a patch update from within the IDE.

The Release Notes lists all fixes for this update. This build doesn’t have any additional fixes comparing to the PyCharm 2016.1.4 Release Candidate build, so here’s the recap:

  • numerous performance improvements along with fixes for various hangs
  • a number of fixes for Docker-based python interpreters and Docker Compose support
  • a fix for exception while renaming a string format parameter
  • and much more

Download PyСharm 2016.1.4 for your platform from our website and please report any problem you found in the Issue Tracker.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in comments to this post and on twitter.

The Drive to Develop
PyCharm team

Posted in Release Announcements | 11 Comments

Python Developers Survey 2016: Findings

Recently we conducted a survey among more than 1,000 Python developers to identify the latest trends and gain insight into how the Python development world looks today.
We’re excited to share some of these results with you.

PC_Developer_Survey2016

It turns out Python is becoming the language of choice for more and more developers. At the same time time Python 3 is gaining popularity and is expected to overtake Python 2 very soon. Another interesting finding is that a huge part of Python developers appear to be engaged in scientific development and data analysis, which correlates with results from other independent research showing that the number of scientific Python developers is growing steadily and is projected to catch up with web development in the foreseeable future. Discover many more interesting findings from the Python world in our detailed report.

As usual we’re anxious to hear your feedback and discuss these survey findings in the comments to this blog post and on twitter.
_
Your PyCharm Team
The Drive to Develop

Posted in Cool Feature, Survey | 1 Comment

PyCharm 2016.1.4 RC is Available

Today we announce the PyCharm 2016.1.4 Release Candidate build #145.1503. The list of bug fixes and improvements for this build can be found in the release notes.

Some highlights of the PyCharm 2016.1.4 RC are:

  • numerous performance improvements along with fixes for various hangs
  • a number of fixes for Docker-based python interpreters and Docker Compose support
  • a fix for exception while renaming a string format parameter
  • and much more

Patch updates from the previous PyCharm releases should be available shortly.

Please give PyCharm 2016.1.4 RC a try before its official release and please report any bugs and feature request to our issue tracker.

-PyCharm team

Posted in Early Access Preview | 4 Comments

Announcing PyСharm 2016.1.3 with important fixes

PyCharm 2016.1.3 is now available from the download page. It also will be available shortly as a patch update from within the IDE.

Some notable highlights of this update include:

  • A number of fixes for Docker-based interpreters support
  • A lot of fixes for Docker Compose support
  • Fixes for various code inspections
  • Fixes for Debugger
  • Various performance improvements for the code insight and navigation engines
  • And much more

The list of numerous bug fixes and improvements for this build can be found in the Release Notes.

In addition to the usual functionality fixes, this update contains important fixes for two security issues inside IntelliJ Platform. The vulnerabilities, in various forms, are also present in older versions of PyCharm; therefore, patches for those are also available.

Find more information about the issues, the affected versions and ways to update in this special blog post: http://blog.jetbrains.com/blog/2016/05/security-update-for-intellij-based-ides.

It is strongly recommended for all users to install the update as soon as possible.

If you would like to discuss your experiences with PyCharm, we look forward to your feedback in the comments to this blog post and on twitter.

Your PyCharm Team
The Drive to Develop

Posted in Release Announcements | 2 Comments

Debugger Interview with PyDev and PyCharm

PyCharm’s visual debugger is one its most powerful and useful features. The debugger got a big speedup in the recent PyCharm, and has an interesting backstory: JetBrains collaborated with PyDev, the popular Python plugin for Eclipse, and funded the work around performance improvements for the common debugger’s backend.

To tell us more about the improvements as well as cross-project cooperation, we interviewed the principles: Fabio Zadrozny, creator of PyDev, and Dmitry Trofimov, Team Lead for PyCharm.

Let’s jump right into it. Tell us about the speedups in the latest pydevd release.

FZ: The performance has always been a major focus of the debugger. I think that’s actually a requisite for a pure-python debugger.

To give an example here: Python debuggers work through the Python tracing facility (i.e.: sys.settrace), by handling tracing calls and deciding what to do at each call.

Usually a debugger would be called at each step to decide what to do, but pydevd is actually able to completely disable the tracing for most contexts (any context that doesn’t have a breakpoint inside it should run untraced) and re-evaluate its assumptions if a breakpoint is added.

Now, even having the performance as a major focus, the latest release was still able to give really nice speedups (the plain Python version had a 40% speed improvement overall while the Cython version had a 140% increase).

I must say that at that point, there weren’t any low-hanging fruits for speeding up the debugger, so, the improvement actually came from many small improvements and Cython has shown that it can give a pretty nice improvement given just a few hints to it.

DT: The performance of the debugger was one of the top voted requests in PyCharm tracker. The latest release addresses this by implementing some parts of the debugger in Cython, which leads to huge performance improvements on all type of projects.

Was the Cython decision an easy one?

FZ: Actually, yes, it was a pretty straightforward decision…

The main selling point is that the Cython version is very similar to the Python version, so, the same codebase is used for Cython and plain Python code — the Cython version is generated from the plain Python version by preprocessing it with a mechanism analogous to #IFDEF statements in C/C++.

Also, this means that with the same codebase it’s possible to support CPython (which can have the Cython speedups) while also supporting Jython, PyPy, IronPython, etc. I even saw someone post about the debugger being used in a javascript implementation of Python.

DT: The idea was to make the debugger faster by rewriting the bottlenecks in C, but at the same time optional to have any compiled binaries, so that pure Python version would still work. Also, it was desirable to have as little code duplication as possible. Cython let us do all that perfectly, so it was a natural decision.

Let’s take a step back and discuss the 2014 decision to merge efforts. How did this conversation get started?

FZ: I was doing a crowdfunding for PyDev which had a profiler as one of its main points, which was something that PyCharm wanted to add too. Although the initial proposal didn’t come through, we started talking about what we already had in common, which was the debugger backend and how each version had different features at that point. I think PyCharm had just backported some of the changes I had done in the latest PyDev version at that time to its fork, and we agreed it would be really nice if we could actually work in the same codebase.

DT: We have used the fork of Pydev debugger since the beginning of the PyCharm and occasionally I would check what was going in Pydev branch to backport features and fixes from there to PyCharm. Meanwhile, Fabio does the same, taking the latest fixes from PyCharm branch. As time passed and branches diverged, it was getting more and more difficult to compare the branches and backport fixes from one another.

After one of the tough merges, I thought, maybe we’d better create a common project that would be used in both IDEs. So I decided to contact Fabio and was very happy when he supported the idea.

Did the merging/un-forking go as you planned, or were there technical or project challenges?

FZ: The merging did go as planned…

The main challenge was the different feature set each version had back then. For instance, PyDev had some improvements on dealing with exceptions, finding referrers, stackless and debugger reload, whereas PyCharm had things such as the multiprocessing, gevent and Django templates (and the final version had to support everything from both sides).

The major pain point on the whole merging was actually on the gevent support, because the debugger really needs threads to work and gevent has an option for monkey-patching the threading library, which made the debugger go haywire.

DT: The main challenge was to test all the fixes done for the PyCharm fork of the debugger for the possible regressions in the merged version. We had a set of tests for debugger, but the coverage, of course, wasn’t 100%. So we made the list of all debugger issues fixed for the last 3 years (around 150 issues,) and just tested them. That helped us to ensure that we won’t have regressions in a release.

Fabio, how did it go on your end, having JetBrains sponsor some of your work? Any pushback in your community?

FZ: I must say I didn’t really have any pushback from the community. I’ve always been pretty open-minded about the code on PyDev (which was being used early on in PyCharm for the debugger) and I believe IDEs are a really personal choice. So I’m happy that the code I did can reach more people, even if not directly inside PyDev. Also, I think the community saw it as a nice thing as the improvements in the debugger made both, PyDev and PyCharm, better IDEs.

The Python-oriented IDEs likely have some other areas where they face common needs. What do you think are some top issues for Python IDEs in 2016 and beyond?

FZ: I agree that there are many common needs on IDEs — they do have the same target after all, although with wildly different implementations 😉

Python code in particular is pretty hard to analyze in real-time — which contrasts with being simple and straightforward to read — and that’s something all “smart” Python IDEs have to deal with, so, there’s a fine balance on performance vs. features there, and that’s probably always going to be a top issue in any Python IDE.

Unfortunately, this is probably also a place where it’s pretty difficult to collaborate as the type inference engine is the heart of a Python IDE (and it’s also what makes it unique in a sense as each implementation ends up favoring one side or the other).

DT: The dynamic nature of Python was always the main challenge for IDEs to provide an assistance to developers. A huge step forward was done with Python 3.5, by adding a type hinting notation and typeshed repository from which we will all benefit a lot. But still this thing is in its early stage and we need to define and learn effective ways to adopt type hinting.

Python performance is also a challenge. In the Python world, when you care about performance, you switch from using pure Python to libraries written in C, like numpy. Or you try pypy. But in both cases performance and memory profiling becomes hard or even impossible with current standard tools and libraries. I think that tool developers can collaborate on that to provide better instruments for measuring and improving the performance of Python apps.

What’s in the future for pydevd, performance or otherwise?

FZ: I must say that performance wise, I think it has reached a nice balance on ease of development and speed, so, right now, the plan is not having any regression 😉

Regarding new development, I don’t personally have any new features planned — the focus right now is on making it rock-solid!

DT: One of the additions to pydevd from the PyCharm side is the ability to capture the types of the function arguments in the running program. PyCharm tries to use this information for code completion, but this feature now is optional and off by default. With the new type hinting in Python 3.5 this idea gets a new spin and the types collected in run-time could be used to annotate functions with types or verify the existing annotations. We are currently experimenting only with types, but it could be taken further to analyse call hierarchy etc.

Posted in Uncategorized | 5 Comments

PyCharm 2016.1.3 RC is available

It’s been a while since we released PyCharm 2016.1.2 and today we announce the Release Candidate build of PyCharm 2016.1.3.

Some highlights of the PyCharm 2016.1.3 RC build 145.969 are:

  • A number of fixes for Docker-based interpreters support
  • Fixes for Docker Compose support
  • Fixes for false positive warnings in strings inspections
  • Various performance improvements for the code insight engine
  • And much more

The list of numerous bug fixes and improvements for this build can be found in the Release Notes.

We encourage you to give PyCharm 2016.1.3 RC a try before its official release. Please change the updates channel to “Early Access Program” in the updates settings (Help|Check for updates…) to get the patch update to the EAP build from the recently released PyCharm 2016.1.2 or download the full installer for your platform. Please report any bugs and feature request to our issue tracker.

-PyCharm team

Posted in Early Access Preview | 1 Comment

PyCharm Migration Tutorial for Text Editors

If you’re a Python developer who uses a text editor such as Vim, Emacs, or Sublime Text, you might wonder what it takes to switch to PyCharm as an IDE for your development. We’ve written a helpful Migrating from Text Editors tutorial for just this topic.

The tutorial starts with the basic question of “What is an IDE?” The line between text editor and IDE can be blurry. PyCharm views the distinction as: a project-level view of your code and coding activities, with project-wide features such as coding assistance and refactoring.

This document then goes over some of the important points when migrating: the project-oriented UI, working with projects instead of files, Vim/Emacs specifics, keyboard shortcuts, customizing, and a discussion of facilities important to text editor users (multiple cursors, split windows, etc.) It then closes by discussing areas the IDE can really help, for example, the managed running and debugging of your code.

Of course, this document is just an overview. Vim and Emacs in particularly have decades of development and features, and PyCharm itself is now very mature with many features itself, so a complete comparison would break the Internet. If you have a specific question, feel free to comment, and we hope you find the tutorial helpful.

Posted in Tutorial | Leave a comment