PyCharm Blog

When you’ve written some great code, you might want to make this available for others to use as well. The pythonic way of sharing a package is making it available on PyPI. Let’s create a simple package and go through the process of publishing it!

Today’s Package: Left Pad

JavaScript developers are great at sharing code: if someone writes functionality, they share it so that others don’t have to reinvent the wheel (it doesn’t stop them from doing so however). A great example of this dedication to reuse is the excellent ‘left-pad’ library, and today we will write a clone of this library to bring this greatness to the Python community!

Someone has already been helpful and released a left-pad package on PyPI. However, his package has neither unit tests nor a GitHub repository so we can do better! We’ll make a drop-in replacement which we’ll name ‘pad-on-left’.

It is important to check if the intended name of your package is free on PyPI before you start. To do so, just use the search box on the top right of the PyPI page.

Structure of the Package

To make our package work with PyPI (which really means, to make it work with distutils), our package needs to have several files:

We’ll need to write a README file, the default for Python is to use ReStructuredText for these. You can read more about ReStructuredText in the Sphinx documentation. If you’d prefer to use Markdown or another file format, you need to add a setup.cfg file where you specify the name of the file.

Furthermore, you need to pick a license for your code. People can’t use your code without a license. In this case, we’ll pick a permissive open source license, I’ll use the Apache 2.0 license, but for your own project, you can use the MIT license or something else.

We’ll create the setup.py file later using PyCharm. Let’s start by creating a new Pure Python project, and writing the application.

Writing the Application

The function we need to write is simple: we get a string or a number, and if it is smaller than the specified length, we should add fill characters on the left side until we reach that length. Unlike JavaScript, the Python standard library has a method for this, and our ‘business logic’ is a single line of code:

Now let’s add some tests. If you go to the npm (Node package manager, pip for JavaScript) page for left-pad, there are several examples listed there. Let’s make sure that our code correctly executes these examples.

A great way to unit test Python is by using doctests. They allow you to write examples in the docstrings of your code, and then doctest will execute the examples and verify that the actual code matches your documentation:

To run the test, add a doctest run configuration: go to Run Configuration, and add a Python Tests | Doctests configuration. Then you can specify the path in several ways, I’ve chosen to configure it to look in my package’s folder. Afterwards you can run the tests and verify that the unit tests pass.

Let’s now generate a setup.py file, choose Tools | Create setup.py:

• Package name should already be filled, and is the name of our project. This is the name that users will later use to pip install this package
• Version: you need to provide the initial version of your package. This should be updated every time you want to push an update to PyPI. You should use semantic versioning if you can, otherwise, PEP 440 describes several ways of versioning packages.
• URL: this should be the homepage URL of your project. In most cases you can use the GitHub repo URL here.
• License: the license you’re using for your code, I’ll use ‘Apache 2.0’
• Author: Your name and surname
• Author Email: Your email address
• Description: A short description of what your package is for

Publishing the Code on GitHub

Let’s now create a git repo, and push the code to GitHub. Use Ctrl+Shift+A (Cmd+Shift+A on Mac) to open the Find Action window, and run “Create Git Repository”. Add a .gitignore file, select all your files, and then add them to git with Ctrl+Alt+A (Cmd+Alt+A). Then, commit (Ctrl+K / Cmd+K) your files to create the initial commit.

Now publish to GitHub: open Find Action again (Ctrl+Shift+A), and find ‘Share Project on GitHub’.

As we’re going to package and release this code as version 1.0.0, we can also tag this commit as version 1.0.0. We can do this by going to VCS | Git | Tag, I’ll use ‘v1.0.0’ as both the tag name, and the message. It’s recommended to use a message as well to create a full tag.

Tags are pushed separately, so we need to go to VCS | Git | Push, enable the ‘Push Tags’ checkbox, and Push.

Register for PyPI

At this point, we should register for both the real PyPI, and test PyPI. You can register here for test PyPI, and find the same form for production PyPI here.

To use these credentials, we need to create a .pypirc file. This is an ini-style configuration file, so let’s create a scratch file which we can then save in the correct place. Use Shift+Ctrl+Alt+Insert to start creating the scratch file, and then select ‘ini’ as the language.

After writing the contents of the file, use File | Save As, and then save it as .pypirc in your home directory (/home/yourname/.pypirc on Mac and Linux, C:\Users\YourName\.pypirc on Windows). Afterward, you may want to restrict the file to be only readable from your account: chmod 600 ~/.pypirc on Mac and Linux, on Windows right click the file, Properties | Security, and then make sure only your account has Full Control, and other accounts don’t have any rights.

Push to PyPI

We need to build our distribution package first. For this there is a setup.py command: setup.py sdist. The s in sdist is for ‘source’. You can also create pre-built wheels by using bdist, for more information about those, see PyPA’s page on packaging.

We’ll only build a source distribution today. We can run the setup.py task by choosing Tools | Run Setup.py task, and then choosing ‘sdist’. As we don’t need any additional command-line options, we can dismiss the dialog for those. We can see in the run output that our package has been built.

Now that we have our built package, we can proceed to upload our code. PyPA recommends using Twine rather than setup.py commands, as setup.py may transmit your passwords in plaintext. So we will need to install Twine: Settings | Project | Project Interpreter, and add Twine.

After Twine is installed, we can open the Terminal (Alt+F12) and run twine register dist/pad-on-left-1.0.0.tar.gz -r testpypi and then twine upload dist/* -r testpypi. If these commands executed correctly, you should be able to find your package on testpypi.

If everything looks good on testpypi, you can run the same commands without -r testpypi:

twine register dist/pad-on-left-1.0.0.tar.gz
twine upload dist/*

And after those commands completed successfully, you’ve uploaded your package to PyPI! Congratulations!

If you’ve ever visited JetBrains at a conference, you know that we always have handouts with keymaps for our products. You might also know that we have different keymaps for Windows/Linux and for Mac due to the different keyboard layouts. As sometimes people grab the wrong one and find out too late, they can’t use the keymap. Therefore we’ve decided to unify the keymaps into a single keymap.

Unfortunately, with a single keymap, we have less space than with separate keymaps so we will need to select a subset of hotkeys that we want to keep. As WebStorm recently already did this, let’s have a look at theirs to compare. It’d be helpful to find out which keys they selected, and more importantly, which keys they left out. To do this, let’s write a quick script.

If you’d like to follow along with the script, you can always have a look at the code on GitHub.

Python has built-in set operations so we can use these to do the hard work later. However, in order to use these, we have to parse our data. Our source materials are two CSV files that look like this:

So let’s define an object that can represent a hotkey:

Using sets

In this script, we’re going to use sets to compare the keymaps. Sets are meant for collections of unique objects where the order of the objects doesn’t matter. Hotkeys are a great example of these: they need to be unique (can’t bind two actions to a single key combination) and the order doesn’t matter (it doesn’t matter if we define Ctrl+Shift+A first, or Ctrl+Alt+O).

However, as a set should only contain unique objects, a set will check whether you already have an object that matches the current object upon insertion. We’d like to have hotkeys matched on the keystroke rather than the description (which may differ slightly), so we need to define the appropriate overrides (use Ctrl+O to see them all) on our Hotkey class:

• __eq__ which returns whether or not objects are equal
• __ne__ which returns whether or not objects are not equal
• __hash__ which should return a number that represents the object, this number should be the same for objects that are equal to each other, but ideally different for objects that are not equal to each other

For our class this will look like:

Our hash function here uses the recommended approach from the Python documentation: return a hash of a tuple of the objects that contribute to the equality check.

Parsing, first pass

Let’s start with a first, naive approach to parsing the files. We will expand this code later to deal with edge cases as they arise. To begin, let’s only parse the PyCharm file:

Exploring Results with the Debugger

Let’s run the code, and then have a look at the objects we created using the PyCharm debugger. Let’s first put a breakpoint at the last line of the main method (pycharm_hotkeys.add(to_add)). Then, if we right click anywhere in our code, we can then select ‘Debug’:

And now in the debugger, we see this:

Wouldn’t it be nice to see something clearer than <__main__.Hotkey object at 0x0000015A02D377B8> here? Well, thankfully that is quite easy: we just have to add an override for the __repr__ method of our object.

Now when we start to debug our code again, we get a much better result:

And we can immediately spot an issue: the key is incomplete: It is supposed to be “<Ctrl + Space> Basic code completion (the name of any class, method or variable)”. What’s happening?

This CSV file was saved using Excel, and we’re seeing an encoding issue, if you look at the ‘item’ line in the debugger, you can see that several strange characters occur before ‘Ctrl’. And if we look at the csvfile line, we can see that it’s attempting to decode cp1252, even though the file is UTF-8 with BOM. After a quick look in the Python docs, we find that we need to specify encoding='utf-8-sig' to the open method.

Handling special cases

At this point when we look at the keymap, and the results we’re getting in the debugger, we can see that there are a couple of hotkeys which are going to be hard to parse:

Ctrl +X, Shift + Delete - Cut current line or selected block to clipboard

Alt + F7 / Ctrl + F7 - Find usages / Find usages in file

These are either alternative hotkeys for the same action (indicated with the comma) or two related hotkeys on a single line (with the slash). As there are many of these on the WebStorm keymap, we’ll need to deal with them. Let’s first refactor our Hotkey parse code into a function. To help PyCharm, let’s first extract all of our code in the main clause into a main function. To do this select everything in the if __name__ == ‘__main__’ clause, and use Ctrl + Alt + M to extract the method.

Then, let’s select the lines from # Define the key to add until to_add.action = item[1], and use Ctrl + Alt + M again. Let’s name this function ‘parse_hotkey’. Let’s also manually rework the arguments to be a little more readable: keystroke and action instead of just a list item:

Now we can change our parse_hotkey method to return a list of hotkeys rather than a single hotkey: return [to_add], and change pycharm_hotkeys.add(to_add) to pycharm_hotkeys.update(to_add).

At this point, we can add code to handle the special cases in the parse_hotkey method. For the sake of brevity (hah), I’ll omit telling more about handing special cases, and refactoring the CSV reading method to easily read both files. You can see the full code on GitHub if you’d like. A couple of highlights:

• The any(iterable) method will return True if any of the items in the iterable is truthy
• List comprehensions are great, if you haven’t used one yet, check out this list comprehension tutorial.

Set Operations

At this point, we have two sets: pycharm_hotkeys, and webstorm_hotkeys. Now we can use several methods to analyze these sets:

• intersection: which elements are in both sets? In this case: which hotkeys are both on the PyCharm, and on the WebStorm keymap?
• difference: which elements are in set A, but not in set B (one way around)
• symmetric_difference: which elements are in set A, but not in set B; but also which elements are in set B, but not in set A (both ways)

In our case, I’m actually interested in the differences both ways, but I care about which way around it is. This leaves my main method looking like:

When we run this code, we get the results we’re interested in:

If you run the code you may notice that the keys appear in a different order every time, which is a result of sets being unordered.

To learn more about sets in Python, check out the Python set documentation.