Using Docker in PyCharm

Please note: this blog post was written in 2015, and parts of it have become outdated. If you’re using Docker Compose, please read our newer blog post on using Docker Compose.


Modern development workflows emphasize isolation and reproducability in development and production. Docker and its platform approach to containers has become very popular. With PyCharm Professional Edition 5, Docker is now supported as a remote interpreter.

Let’s take a look at PyCharm’s Docker integration in action.

Summary

  • Get a Django-oriented Docker image pulled locally
  • Make a sample Django project
  • Create a Docker interpeter to run it
  • The Django run configuration makes a new container on start and removes it on stop

Overview

In Python, when you run an application – a Django site, a database script, etc. – you are running it in an environment. Python has some tools to manage the environment and make it reproducible, such as virtual environments, pip requirements.txt files, and setup.py dependencies. But what about the non-Python parts of application?

Containers are one solution to this. In Docker, a container is an isolated universe with software, running inside your computer. They are fast and easy to create, start, stop, and destroy. This is ideal, not just for development, but for deployment as well.

PyCharm 5 provides the beginning of a series of steps towards productive development using containers and Docker. Namely, PyCharm 5 supports project interpreters that are executed in a Docker container.

Preparation

First, make sure docker and docker-machine are setup in your environment. The Docker installation is painless and the website docs are quite friendly. You’ll need a Docker “host” virtual machine setup and running, including on Linux. On Windows and OS X, installing the Docker Toolbox is effortless.

Next, we have to decide what software we want in our containers. In Docker, containers are built using “images”: collections of pre-installed software plus configuration that is called during container creation. Unlike other interpreters in PyCharm, you don’t visit the Project Interpreter preferences to add Python packages. All dependencies need to be baked into the Docker image you choose.

This blog post is based on the minimum/docker-django-mysite Docker image, which includes enough Python/Django software to run the PyCharm-generated Django starter project. You can fetch that onto your system with this command:

Once that Docker image is locally available and the Docker “host” is running and knows about it, PyCharm can make Docker interpreters for your project, as containers based on that image.

Alternatively, you can skip this step and later pull in the image while creating the Docker-based remote interpreter by simply typing in the image name during creation.

Note: You have the option of making your own image as well, using a Dockerfile. Once you make the image, you can tell PyCharm to make Docker interpreters based on it.

Create Django Project

Let’s make a Django project, then make a Docker interpreter for it. In PyCharm, choose File -> New Project, click on Django, and follow the normal process for making a Django project.

During project creation, you’ll have to use a local interpreter. If you try to make a Docker interpreter, PyCharm will give a warning saying you can only use a local interpreter.

The result of this step is a directory on your local computer with sample Django code and a Django-specific PyCharm Run Configuration.

Make PyCharm Docker Interpreter

On to the important part. We need to define a “remote interpreter” that runs in a PyCharm Docker container.

First, go to Preferences -> Project Interpreter and click on the gear to the right of Project Interpreter near the top. In the Configure Remote Python Interpreterdialog, click the Docker button.

Click the menu dropdown for Machine Name. It will likely have one entry named default. Choose that, and you will see the following:

docker_interpreter

Choosing default gives PyCharm a way to find the Docker daemon, which can let PyCharm know which images are available locally. Hence the dropdown for Images. Click on the dropdown and choose minimum/docker-django-mysite:latest as the image, then click Ok.

You now have a Docker container that will be used as your project interpreter. PyCharm features like autocompletion, code inspections, and checks will be driven by this interpreter.

Run Django

Now that we have a Docker container for our project interpreter, let’s use our Django-flavored Run Configuration to run it. Just to be safe, edit the run configuration to make sure it is using the Docker-oriented interpreter.

What does PyCharm do when it runs this configuration:

  1. First, it creates and starts a new container based on the image we named when creating the project interpreter.
  2. This container mounts your project directory into the container at /opt/project in the container. Note: On Linux, you currently have to perform this volume mapping manually.
  3. This container also mounts volumes needed for PyCharm to do its work: Python skeletons and Python library sources.
  4. It then executes the run configuration’s Python command.

If you are running PyCharm 5.0.2 or older, there is one fix you need to do:

  • Run -> Edit Configurations and select the PyCharm-generated Djano run configuration
  • In the Host field, enter 0.0.0.0

When you click the “Run” button, Django will be started. Yay, we are running a container! In fact, we can go on to test running, debugging, code coverage, profiling, and all the other ways to run a configuration. But first, let’s click the link in the console to open the Django site in our browser:

django_admin
You can confirm this using the following Docker commands:

This shows the most recently-run container. As you can see, our minimum/docker-django-mysite-based container is currently running.

Likewise, when you click the “Stop” button in PyCharm to stop the running Django configuration, PyCharm not only stops Django, but it also stops the Docker container. In fact, it deletes the container. Each time you run, PyCharm creates a new Docker container and starts it.

You can confirm this. After re-starting the Django run configuration, type:

The value in the CONTAINER ID column, as well as the NAMES value, is different from the previous values we got from running docker ps -l.

Current Limitations

Docker integration in PyCharm is already useful, but is still only a start. Much more is being done for future releases. For example:

  • Docker Compose. This is a big item and a frequently-requested feature. The effort for this is tracked in a YouTrack ticket in the PyCharm ticket system and work is already being scheduled. Please vote and leave any comments you might have.
  • Linux mounts. As noted in the PyCharm online help, on Linux, you have to mount your project directory manually via VirtualBox. This issue is obviated by work on the next point.
  • Customizing the container. Docker lets you pass arguments when creating/running a container. These arguments include volumes to mount and ports to forward. PyCharm doesn’t currently let you customize this in the run configuration, but this is being worked on.
  • Only Django for web ports. If you want a container with HTTP (or any other) ports available, the Django run configuration is the only one, and that is only for HTTP. Other run configurations (Flask, Pyramid, etc.) won’t forward any ports. Nor will attaching to a database, or orchestrating multi-container apps (unless your Dockerfile forwards the ports.) This is really just emphasizing the previous point, and as said there, help is on its way.

Conclusion

With this release we’ve shown the beginnings of what we can do with Docker integration, and the results are promising. Docker has the potential to be a first-class part of PyCharm’s development workflow.

This entry was posted in Cool Feature. Bookmark the permalink.

55 Responses to Using Docker in PyCharm

  1. Jonatas C D says:

    please, just ensure to tell people that this only works with docker-machine versions from 0.5.4 and after.

    they fixed here – https://github.com/docker/machine/issues/2587

  2. Mark McKelvey says:

    Any update on passing arguments when starting the docker container? I’d love to be able to debug my Flask app in a docker instance.

  3. Sean Brady says:

    I’m getting the error:

    7c9ebe40ddb5:python -u /opt/project/manage.py runserver 8000
    python: can’t open file ‘/opt/project/manage.py’: [Errno 2] No such file or directory

    I’m running PyCharm 5.0.4, Docker 1.10.2 on Mac OS X 10.11.2, with docker-machine configured to use a Ubuntu 14.04.4 LTS instance running on Digital Ocean with Docker 1.10.2 installed. PyCharm was able to configure the remote interpreter without an issue, however when I run the project I get the error above.

    Can you expand on the linking needed for Linux hosts? Does this apply in my case? If so can you shed some light on how I would do this?

    • alex_koshevoy says:

      Hi!

      Frequently Docker daemon is running either on a VM on the local host (for Windows and OS X) or the local host itself (for Linux). By local host I mean the same machine where PyCharm is running and the project sources are stored. In both these cases Docker Machine provides the ability to map folders on the local host to volumes on Docker containers. This is made by means of shared folders in case of VMs and by direct directories mounting in case of Linux.

      In your case Docker daemon is running on the remote server and mounting to the local host file system is not supported by Digital Ocean Docker’s provider. PyCharm needs upload the project sources to the cloud manually in your case, which is not yet implemented. There is no such issue in our tracker, please feel free to file it in https://youtrack.jetbrains.com/issues/PY with details of your use case.

    • patrick says:

      I have the same problem as yours, how did you solve that?

  4. Patrick Jones says:

    > You’ll need a Docker “host” virtual machine setup and running, including on Linux

    This is horrible, and keeps me from using this feature.

  5. Pingback: Docker教程:使用docker配置python开发环境 - IT大道

  6. romeo mihalcea says:

    My services run fine using docker-compose but when starting the app from PyCharm I can’t access it in the browser. Any idea why?

  7. Sebastian Cheung says:

    Can someone confirm that this is allowing me to run PyCharm within a docker image?

    • Paul Everitt says:

      The goal of this work is to manage the running of Python and related services (e.g. database) in a Docker image, not running the IDE itself.

  8. Sebastian Cheung says:

    Hi Paul (or anyone else)

    Does this also work for a MeteorJS project in PyCharm too?

    • Paul Everitt says:

      This is about getting a remote Python interpreter, where the Python interpreter is in a Docker container. Can you explain how you need to integrate Python with Meteor?

  9. Guillaume says:

    Hello, thanks for this tutorial, I’ve just followed every step, but it doesn’t work for me.
    I get an error : Can’t run remote python interpreter: Container command ‘python’ not found or does not exist.

    I really don’t know why…
    Still on it.

    • Paul Everitt says:

      If you use Docker to log into the container, is python on the path?

    • Lukasz says:

      Have you manage to get this sorted? I have the same issue.
      I can execute python within the container when i log into, as well as pycharm can see all python packages inside contaier. Not sure what is misconfigured

  10. Sergejs says:

    Hello!

    I have a following setup:
    1) A Windows box with a PyCharm installed on it
    2) an Ubuntu box with a docker inside it

    The sources of my project (.py files) are inside docker image (Linux as well) that is run on an Ubuntu box.

    Is it possible to configure PyCharm for this setting ?

    I would like to be able to run and debug .py files located in the docker container running on a remote Linux host.

    • Paul Everitt says:

      In a configuration like that, you’ll treat the Ubuntu box as a remote interpreter over SSH. Not really Docker, since it is a different machine than the Windows box.

  11. Tony says:

    Where is the docker build context when you launch a deployment? I’m having trouble specifying a requirements file and I believe it’s because “ADD . /var/app” is copying over the folders from wherever the docker command is being run from by the plugin. It is clearly not being run from project root as you might expect. Any idea where I can set the build path? Thanks!

  12. Evan Dale says:

    When will this be available in the python plugin for intellij idea ultimate?

  13. MJ says:

    Is it possible to deploy a docker image from a remote host machine to a local computer. I have tried the setting provided by pycharm webpage, but did not work. There is a certificates folder and it opens only local folders…

    Thank you in advance.

  14. Vu Ngoc Duong says:

    Everytime i make changes, I have to restart my container. Is there way to push changes to docker conatainer automatically?

  15. Shreyas Kulkarni says:

    Any plans to integrate with Docker for Mac instead of the old docker? I am sure many people who are using a Mac as their dev environment, will want to use pycharm with the latest, native docker on mac.

    • Dmitry Filippov says:

      We’re currently investigating this. It’s not easy to support it with the current implementation, but we have plans to improve Docker support in future versions of PyCharm.

  16. Nat says:

    I’d love to have Mac support as well.

  17. Vladimir Smelov says:

    When I open “Python Console” tab in Pycharm, it runs Python Interpreter

    5cf86d7d9de2:python /opt/.pycharm_helpers/pydev/pydevconsole.py 0 0
    PyDev console: starting.
    import sys; print(‘Python %s on %s’ % (sys.version, sys.platform))
    sys.path.extend([‘/home/vs/ML/GRUV’])

    But actually I cannot use any script from my project directory ‘/home/vs/ML/GRUV’
    >>> import os
    >>> os.listdir(‘/home/vs/ML/GRUV’)
    []

    There are also no ‘/opt/project’ directory
    >>> os.listdir(‘/opt’)
    [‘.pycharm_helpers’]

    Becuase, as far as I understand, Pycharm run Docker with mounting of project directory only in case of directly run project scripts.

    Is there any way to run Docker for Pycharm Interpreter with option
    -v PATH_TO_PROJECT:/opt/project
    ?

    Now i cannot run and debug my project via Pycharm Interpreter tab and need to manually run in terminal command
    docker run -t -i -v PATH_TO_PROJECT:/opt/project /bin/python

  18. Steven says:

    Has there been any progress on adding support for the latest Docker for Mac?

  19. Alex says:

    Hello,
    I’m trying to complete the tutorial but finaly get the following error:

    11cfbee45cb9:python -u C:/Users/Passanova/PycharmProjects/djangotestone/manage.py runserver 8000
    Process finished with exit code 128

    I’m on Win7 and latest Pycharm for evaluation. After playing a bit with the pats now they look like this:

    Working directory = \C\Users\Passanova\PycharmProjects\djangotestone\djangotestone
    Path mappings = /C/Users/Passanova/PycharmProjects/djangotestone=/opt/project
    Docker container settings = -v /C/Users/Passanova/PycharmProjects/djangotestone:/opt/project
    Environment variables = DJANGO_SETTINGS_MODULE(djangotestone.settings) and PYTHONUNBUFFERED(1)

    Finally I’m able to create container but it exits with the above 128 code before starting the server. I’m also unable to verify the container by “docker ps -l”. Every time I press the run button new container is created as expected but it never do the job.
    I can do “docker exec /bin/bash” whit no errors and “$python -V” returns as expected. The fowolling command from the terminal works as well:
    “docker run python -V” and can be verified with “docker ps -l”.

    This is my debug result:
    “ad0418d30d5a:python -u /opt/.pycharm_helpers/pydev/pydevd.py –multiproc –qt-support –save-threading –save-asyncio –client 10.0.2.2 –port 19383 –file C:/Users/Passanova/PycharmProjects/djangotestone/manage.py runserver 8000”

    I don’t have any problems with local interpreter. I’m not sure what the “-u” option means as in the “python –help” there is not much, but I noticed the equivalent comand with local interpreter does not includ it.

    I also tryed to change the image itself as it very much looks like an internal problem(I mean obvioulsy the paths are OK now and the proces is triggered) but no. The same exit with the official Django:latest image.

    I appreciate any help.

    • Ernst Haagsman says:

      Hi Alex,

      It looks like you’re specifying a path on your Windows install within the container. The volume mapping you specified with the “-v” option for Docker should synchonize “/opt/project” within your container with your project folder on your Windows host. What happens if you try to run “python /opt/project/manage.py runserver 0.0.0.0:8000”?

      Let me know if that helps!
      Ernst

  20. Markus says:

    Since I had a failry hard time getting the remote interpreter running, here is the short version for those using 64bit Windows 10 Pro, Enterprise and Education (1511 November update, Build 10586 or later) as specified here:
    https://docs.docker.com/engine/installation/windows/#/docker-for-windows
    And in the future hopefully all recent Windows 10 versions.

    There is no need to fiddle around with the docker-machine. When Docker is up and running:
    1. Make sure you share the C drive between Docker and your local machine
    2. When specifying the remote interpreter set tcp://localhost:2375 as API URL. That’s it.

    Regards,
    Markus

    • Hanan Shteingart says:

      Thanks Markus! you saved my day. I was looking for this for so long and couldn’t make the docker-machine stuff to work properly! Now I can use docker directly using the native approach!

    • M John says:

      Wow, also saved me.

      The pycharm docker support is a total confusing mess at least on Windows. They need to simplify this and stick to these types of approaches!

  21. kumR says:

    any comments/concerns re: running that docker build inside WSL(back on Windows) and then serving it up to Pycharm as an interpreter?

  22. Hanan Shteingart says:

    My pycharm screen looks totally different and it asks me to setup a “docker server”. What am I doing wrong?

    • Paul Everitt says:

      We’ve had several releases since that blog post was written, with lots of changes to the Docker support. For example, we didn’t have Docker Compose support then. Here are the most recent help pages, first for Docker Machine and then Docker Compose.

  23. Hanan Shteingart says:

    “Note: You have the option of making your own image as well, using a Dockerfile. Once you make the image, you can tell PyCharm to make Docker interpreters based on it.”

    How can I tell pycharm to treat a Dockerfile as the container and rebuild it if necessary (e.g. I added a necessary package)

  24. omar says:

    When I get to the choose machine name doesn’t display the default that says on the tutorial, docker an docker machine is installed but doesn’t appear on the dropdown
    any ideas on how to fix this?

  25. resolvewang says:

    Is there any way to use a running container rather than from the image?

  26. Russel says:

    I am running a flask app and I get the following message:
    INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

    I have been trying to access the website from my local browser and it’s not working. First, I tried the following and it didn’t work: “RUN > Edit configuration > Docker deployment > Container > Add port binding > container port=5000 > host port=5000”. Is this right? should I add something in Host IP?

    I also tried to access the API URL in the build, execution > docker, but it’s not working.

    What is the easier way to solve this problem? Thanks a lot for the help.

  27. While using debugger in PyCharm with remote interpreter in Docker, I ran into the following error:
    pydev debugger: warning: trying to add breakpoint to file that does not exist: /opt/…
    where /opt/… is the path in the docker.
    The path mapping is correctly set, I have checked multiple times.
    What could be wrong?

    • Ernst Haagsman says:

      Could you please start Bash in your Docker container and check if the file actually exists at the specified path? You can do this on the command line, or by opening the Docker tool window (View | Tool Windows | Docker), then you may have to double click on ‘Docker’ or ‘Docker Machine’ until it connects. When you see your containers, select the container you’re interested in, right click and choose ‘Exec’ and type ‘/bin/bash’. Then in the container ls to /opt/ and check the path.

Leave a Reply

Your email address will not be published. Required fields are marked *