Webinar: “Visual Debugging” on April 25

Are you a Python developer who knows you should use the debugger, but don’t yet? Or perhaps you are familiar with debugging, but know there is more productivity if you’d spend some time to learn?

pycharm_webinar_Visual_Debugging

PyCharm puts a visual face on debugging and we’d like to help more developers take the plunge and debug. In this webinar, Paul Everitt, PyCharm Developer Advocate, introduces the debugger and goes through many of the essentials in the context of writing a 2d game:

– Debugging without PyCharm’s debugger: print and pdb
– First use of the debugger (and the Cython speedups)
– Breakpoints
– Using the console/IPython at a breakpoint
– Exception and Conditional breakpoints
– All flavors of stepping through code, including filters
– Moving through stack frames
– Setting watches
– Django/Jinja2 template debugging
– Attaching to processes
– Debugging during testing
– Viewing numpy/pandas data frames
– Extracting type information

This webinar is aimed at developers who have never used the debugger, but even those familiar with PyCharm’s visual debugging will learn some new tricks and see some deeper usages.

Paul Everitt is the PyCharm Developer Advocate at JetBrains. Before that, Paul was a co-founder of Agendaless Consulting and a co-founder of Zope Corporation, taking the first open source application server through $14M of funding. Paul has bootstrapped both the Python Software Foundation and the Plone Foundation. Prior to that, Paul was an officer in the US Navy, starting in Python and launching www.navy.mil in 1993.

Posted in Uncategorized | 2 Comments

PyCharm 2017.1.1 Available Now

PyCharm 2017.1.1 is out now! Get it from our website

Some of the things we’ve improved:

  • The font rendering problem has been resolved
  • Many issues in the debugger, code inspections, and test runners have been fixed, including several additional issues since the last release candidate
  • Automatic deployment for new remote projects is now configured correctly
  • Remote Jupyter notebooks tweaks
  • And more: read our release notes for details

If you have any feedback about this version of PyCharm, please let us know! You can report bugs and request features on YouTrack. You can also reach us by leaving a comment on this blog post, by tweeting us at @pycharm or leaving a comment on our Facebook page.

PyCharm Team
-The Drive to Develop

Posted in Release Announcements | Tagged | 14 Comments

PyCharm 2017.1.1 RC out now

We’ve been fixing a lot of bugs, and are getting closer to the 2017.1.1 release. If you’d like to try this version before we release it, you can get it now from our confluence page.

Some of the things we’ve improved:

  • The font rendering problem has been resolved
  • Many issues in the debugger, code inspections, and test runners have been fixed
  • Automatic deployment for new remote projects is now configured correctly
  • Pyramid improvements
  • Remote Jupyter notebooks tweaks
  • And more: read our release notes for details

If you have any feedback about this version of PyCharm, please let us know! You can report bugs and request features on YouTrack. You can also reach us by leaving a comment on this blog post, by tweeting us at @pycharm or leaving a comment on our Facebook page.

PyCharm Team
-The Drive to Develop

Posted in Early Access Preview | Tagged | 6 Comments

PyCharm Edu: The Adaptive Python Course

Back in 2016, we partnered with Stepik to announce a brand-new Python course. It adapted to each student’s level of knowledge and interest and helped them stay more motivated and productive while learning Python. This course was the first step towards adopting Adaptive Learning, which we believe is one of the key trends in the future online education.

After an adoption period, we’ve fixed all the issues in the latest PyCharm Edu 3.5.1. The Adaptive Python course is now polished and ready to use! Let’s explore it in more detail.

The Adaptive Python Course

To start the course, go to File -> New Project -> Educational and choose Adaptive Python from the list:adaptive_python_1

Continue reading

Posted in Cool Feature | Tagged | 3 Comments

Docker-Compose: Getting Flask up and running

A couple of issues were resolved in PyCharm 2017.1, and Docker for Mac should now work out of the box. In this blog post we’ll show you how to set up a project with Docker Compose on a Mac. If you’re on Linux, the process is very similar. Docker Compose is supported on Windows from PyCharm 2017.2.

In this tutorial we’ll show how to create a very simple ‘Hello World’ Flask application, and then how to run it within a Docker container. For the full code, and to follow along, see GitHub: https://github.com/ErnstHaagsman/flask-compose.

First things first

Before we get started, let’s check a couple of things: make sure you are using PyCharm 2017.1 Professional Edition or later. Docker support is not available in PyCharm Community Edition. Then, please ensure your Docker and Docker Compose are up to date. To check, open a terminal, and run docker -v, and docker-compose -v:

Docker and Compose Versions

Then, although it is the default setting in PyCharm 2017.1, it never hurts to check that your Docker API URL (Preferences | Build, Execution, Deployment | Docker) is set to unix:///var/run/docker.sock.

Docker Settings

Now we can create a new Flask project. Let’s create a virtualenv so that PyCharm can stub out our Flask project before we have Docker configured. You can create a Virtualenv by using the ‘…’ button next to the interpreter dropdown.

Create a Flask Project

Dockerizing Flask

After you click create, you should see the standard “Hello World” Flask template. So let’s see if we can get Flask to show us “Hello World” from a Docker container. To do this, we’ll add four files:

  • requirements.txt, just put “Flask==0.12” here to install Flask
  • Dockerfile, where we will set up the Python environment for the Flask app
  • docker-compose.yml to set up how to run the Dockerfile, and add a database
  • docker-compose.dev.yml to make some changes for local development

In the Dockerfile, we’ll use the ‘python:3’ image, expose port 5000, install packages from requirements.txt, and afterwards copy the rest of our project into /app. See the full file on GitHub.

Then, we’ll create a Compose file where we define the Flask app as our only service (for now). Just use build: . to make Docker Compose build the container from the Dockerfile.

Docker will bake our code into the image, and that way the image is self-contained. If we wanted to, we could stop here and rebuild our image whenever we change the code. It makes development quite a bit faster to mount our code with a volume. So we’ll create an additional Compose file to do that while we’re developing.

In docker-compose.dev.yml, all we’re doing is adding a volume mapping for .:/app for the web service. This overlays the code in the container with a volume, making sure that code changes are applied immediately. Keep in mind that you will need to rebuild the image before pushing it anywhere.

Now that everything is configured, let’s quickly run docker-compose up in the Terminal to make sure that works. Open the terminal with Alt+F12 (in PyCharm) and run docker-compose up.

Docker-compose up on the terminal

As we can see, Docker built our container, and Flask is running! Let’s go and check out our ‘Hello World’ message!

A trap for young players!

Safari couldn't open the page

What happened here? Didn’t docker say that Flask was running?

For security reasons, many modern web frameworks actually limit incoming connections to only come from localhost. This means that our Flask app is running, but only accessible from within the Docker container, not from our Mac host. The easiest way to fix this is to add host='0.0.0.0' as a keyword argument to app.run() in flask-compose.py:

Flask, exposed

This change tells Flask to listen not only to requests from localhost (in this case the docker container), but on all network connections. In the case of our container this means we can access it from our host, and depending on configuration from other computers in our network.

If we now stop our container (Ctrl+C in the PyCharm Terminal), and rebuild it by running docker-compose up --build. We can see in the Docker Compose output that Flask is now listening on 0.0.0.0 instead of 127.0.0.1:

Flask is listening on 0.0.0.0

And reloading the page in Safari actually shows ‘Hello World’ now. So let’s stop the container (Ctrl+C in the Terminal), and then configure PyCharm.

Configuring PyCharm

To make our project run using a PyCharm run configuration, we need to set the project interpreter to the Docker Compose service. We can do this from the Project interpreter page in preferences: Preferences | Project: <Project Name> | Project Interpreter.

Click the little button next to the interpreter dropdown (the white one, not the blue one), and choose “Add Remote”. If you don’t see “Add Remote” here, you may be using PyCharm Community Edition, which doesn’t support remote interpreters.

Add Remote Interpreter

Now choose “docker-compose”, and almost everything will be pre-configured, I only had to add the docker-compose.dev.yml configuration to the list. We need to add it manually as this isn’t part of a standard docker project. You can do this with the ‘+’ button underneath the list of configuration files.

Compose Configuration in PyCharm

Now, we should set up the path mapping. We are inserting all our project code into the container’s “/app” directory. So let’s add this on the Project Interpreter screen, use the ‘…’ button next to the Path mappings field to change them.

Path mappings

After you close the preferences window, you can just use the regular Run and Debug buttons to start and debug your project:

Debugging Flask within Docker

To celebrate, let’s change “Hello World!” to “Hello from Docker within PyCharm!”

Flask in Docker-Compose

And that’s all there’s to it! Have fun developing with Docker Compose! If you’d like to learn more, you can read the second part of this tutorial about adding a database service.

PyCharm Team
-The Drive to Develop

Posted in Tutorial | Tagged | 38 Comments

PyCharm 2017.1 Out Now: Faster debugger, new test runners, and more

PyCharm 2017.1 is out now! Get it now for a much faster debugger, improved Python and JavaScript unit testing, and support for the six library.

  • The Python debugger got forty times faster for Python 3.6 projects, and up to two times faster for older versions of Python
  • We’ve added support for the six compatibility library
  • Unit test runners for Python have been rebuilt from the ground up: you can now run any test configuration with PyCharm
  • Are you a full stack developer? We’ve improved our JavaScript unit testing: gutter icons indicating whether a test passed and support for Jest, Facebook’s JS testing framework (only available in PyCharm Professional edition)
  • Zero-latency typing is now on by default: typing latencies for PyCharm 2017.1 are lower than those for Sublime Text and Emacs
  • Support for native Docker for Mac – no more need to use SOCAT! (only available in PyCharm Professional edition)
  • And more!

Get PyCharm 2017.1 now from our website

Please let us know what you think about PyCharm! You can reach us on Twitter, Facebook, and by leaving a comment on the blog.

PyCharm Team
-The Drive to Develop

Posted in Release Announcements | Tagged | 38 Comments

Why Postgres Should be your Document Database Webinar Recording

This Monday Jim Fulton, one of the first Python contributors, hosted a webinar about storing JSONB documents in PostgreSQL. Watch it now:

Known mostly for its mature SQL and data-at-scale infrastructure, the PostgreSQL project added a “JSONB” column type in its 9.4 release, then refined it over the next two releases. While using it is straightforward, combining it in hybrid structured/unstructured applications along with other facilities in the database can require skill.

In this webinar, Python and database consultant Jim Fulton shows us how to use JSONB and related machinery for pure and hybrid Python document-oriented applications. We also briefly discuss his long history back to the start of Python, and finish with his unique NewtDB library for native Python objects coupled to JSONB queries.

Jim uses PyCharm Professional during the webinar. PyCharm Professional bundles the database tools from JetBrains DataGrip, our database IDE. However, the webinar itself is focused on the concepts of JSONB.

You can find Jim’s code on GitHub: https://github.com/jimfulton/pycharm-170320

If you have any questions or comments about the webinar, feel free to leave them in the comments below, or you can reach us on Twitter. Jim is on Twitter as well, his Twitter handle is @j1mfulton.

-PyCharm Team
The Drive to Develop

Posted in Webinar | Tagged | 5 Comments

Inside the Debugger: Interview with Elizaveta Shashkova

PyCharm 2017.1 has several notable improvements, but there’s one that’s particularly fun to talk about: debugger speedups. PyCharm’s visual debugger regularly gets top billing as a feature our customer value the highest. Over the last year, the debugger saw a number of feature improvements and several very impressive speedups. In particular, for Python 3.6 projects, PyCharm can use a new Python API to close the gap with a non-debug run configuration.

pycharm_webinar_inside_the_debugger

If you’ve been to PyCon or EuroPython and come by our booth, chances are you’ve seen Elizaveta Shashkova talking about the debugger to a PyCharm user, or giving a conference talk. Let’s talk to Liza about her work on PyCharm, the debugger, and her upcoming talk at PyCon.

Can you share with us a bit of your background and what you do on the PyCharm development team?

I started my career at JetBrains as a Summer Intern two and half years ago – I implemented a debugger for Jinja2 templates and Dmitry Trofimov (the creator of the PyCharm’s debugger) was my mentor. After that, I joined the PyCharm Team as a Junior developer and implemented a Thread Concurrency Visualizer under the supervision of Andrey Vlasovskikh, and my graduation thesis was based on it.

At the moment I’m supporting the Debugger and the Console in PyCharm.

People really like PyCharm’s debugger. Can you describe how it works, behind the scenes?

The debugger consists of two main parts: the UI (written in Java and Kotlin) and the Python debugger (written in Python). The most interesting part is on the Python side – the pydevd module, which we share with PyDev (the Python plugin for Eclipse).

We don’t use the pdb standard debugger for Python, but we implement our own debugger. At first glance, it’s quite simple, because it’s based on the standard sys.settrace() function, and in fact it just handles events which the Python interpreter produces for every line in the running script.

Of course, there are a lot of interesting frameworks or Python modules, where debugging doesn’t work by default, that’s why we add special support inside the debugger: for PyQt threads, for interactive mode in pyplot, for creation new processes, for debugging in Docker and others.

Note: A year ago we did an interview with the creator of PyDev about the funded debugger speedups.

The Cython extensions gave a big speedup. Can you explain how it works and the performance benefit?

Yes, Cython speedups were implemented by Fabio Zadrozny and they gave a 40% speed improvement for the debugger. Fabio found the most significant debugger bottlenecks and optimized them. Some of them were rewritten in Cython and it gave even more – a 140% speed improvement. Fabio has done a really great job!

On to new stuff. The upcoming PyCharm, paired with Python 3.6, gives some more debugger speedups, right?

Yes, as I’ve already mentioned, for Python interpreters before version 3.6 we used to use the standard Python tracing function. But in Python 3.6 the new frame evaluation API was introduced and it gave us a great opportunity to avoid using tracing functions and instead implement a new mechanism for debugging.

And it gave us a really significant performance improvement, for example, in some cases the debugger has become 80 times faster than it used to be, and it has become at least 10 times faster in the worst case. In some partial cases, it has become almost as fast as running without debugging.

We have so many users’ reports about debugger’s slowness, and now I hope they will be happy to try the new fast version of the debugger. Unfortunately, it’s available for Python 3.6 only.

What changed in Python 3.6 to allow this?

The new frame evaluation API was introduced to CPython in PEP 523 and it allows to specify a per-interpreter function pointer to handle the evaluation of frames.

In other words, it means that we can get access to the code when entering a new frame, but before the execution of this frame has started. And this means that we can modify the frame’s code and introduce our breakpoints right into bytecode: execution of the frame hasn’t started yet so we won’t break anything.

When we used the tracing function, the idea was similar: when entering a new frame we checked if there are any breakpoints in the current frame and, if they exist, continue tracing for every line in the frame. And sometimes it led to the serious performance problems.

But in the new frame evaluation debugger, this problem was solved: we just introduce the breakpoint into the code and the other lines in the scope don’t matter. Instead of adding an additional call to the tracing function for every line in the frame, with Python 3.6 we add just one call to “breakpoint”, and that means that the script under debugging runs almost as fast as without the debugger.

Congratulations on your Python 3.6 Debugging talk being accepted for PyCon. Who will be interested in your talk?

This talk will be interesting for people who want to learn something new about the features of Python 3.6. Also, it will be useful for people who want to learn yet another reason to move to the Python 3.6: a fast debugger, which should appear in many Python IDEs.

Moreover, after the talk people will understand, how the PyCharm’s debugger works, and why such fast debugging wasn’t possible in the previous versions of Python.
This talk is for experienced Python developers, who aren’t afraid of calling Python’s C API functions and doing bytecode modifications. :)

What is the next big thing in debugging to work on in the next year?

We have a rather old and important problem in the debugger related to the evaluation and showing big objects in the debugger. This problem has existed from the beginning, but it has become really important during the last few years. I believe it gained visibility due to the increased number of scientists who use PyCharm and work with big data frames. At the moment we have some technical restrictions for implementing this feature, but we’re going to implement it in the near future.

Posted in Interview | Tagged | 2 Comments

PyCharm 2016.3.3 Released

Shortly after announcing PyCharm 2017.1 Release Candidate today we bring you some important backport fixes for the current PyCharm stable version.  Please welcome PyCharm 2016.3.3.

For the detailed list of changes please see the Release Notes.

If you’re running PyCharm 2016.3.2, use Help | Check for Updates to upgrade using a patch. Otherwise, feel free to download the update from our website, or using the ToolboxApp.

– PyCharm team

 

Posted in Release Announcements | Tagged | 11 Comments

PyCharm 2017.1 Release Candidate

Great news: We are now approaching the final steps towards the first major release in 2017. So today we’ve published the PyCharm 2017.1 Release Candidate. Download it now from our early access page.

PyCharm_Poff20171_splash

Since the previous early access preview build we’ve fixed a lot of bugs in various subsystems and also introduced some important improvements: details in the release notes.

In case you missed what’s new in upcoming PyCharm 2017.1, check out the early access page to learn about the features included in the new version. Some highlights:

  • The Python debugger got forty times faster for Python 3.6 projects, and up to two times faster for older versions of Python
  • Improved Python and JavaScript testing
  • Support for the six library
  • Zero-latency typing
  • Support for native Docker for macOS
  • And much more

We hope that there will be no major bugs in the RC build, however, should you encounter any problems, please report them to YouTrack – we’ll still have a bit of time to fix stuff before the final release.

Stay tuned for a PyCharm 2017.1 release announcement and follow us on Twitter.

-PyCharm team

Posted in Early Access Preview | Tagged | 12 Comments