Interview: Paul Craven on Python Gaming and Teaching

Writing games in Python is fun, so how about using it to teach computer programming? Paul Craven is both a professor and the creator of Arcade, a 2d game library for Python. He’s doing a webinar with us next week, so we talked to him about teaching Python, using Python 3 type hints, why another Python game library, and how IDEs fit into teaching.

paul_craven_splash

Thanks a bunch for doing the webinar next week. First, can you tell us a little bit about yourself?

I worked in the IT industry for 15 years before switching to teaching at Simpson College, a small 4-year college in Iowa. My main interest has been getting first-time programmers to realize that programming can be fun. That moment when a student cheers out loud because they finally figured out how to get sprites to move correctly on the screen? It is a beautiful thing to see.

You teach programming, and you created the Arcade game library to help. Before talking about Arcade, can you explain the motivation behind having a framework that you can teach?

Teaching is like engineering. Each semester you work to improve how you teach students. I had been using the Pygame library. But I wanted a library that I could improve on based on what I saw from students. For example:

Function and parameter names that students intuitively understand. Each year I had to teach them “set_mode” opens a window. Why not just have the function called “open_window”?
Support for functions students which ask for. In Pygame drawing ellipses with thick borders always had a moire pattern because of a bug in the library. And you can’t tilt ellipses when you draw them. Every year I have a student that wants to draw a football. And each year they were frustrated that it looked awful. I wanted a library where it just worked.
Students would download a graphic for their sprite. But it would be too large and there was no easy way to scale the image. That always resulted in hours of wasted time explaining how to do the scaling. What if the library just supported scaling?

After a while I collected such a long list of things like that, I decided to create a game library that where I wouldn’t have to teach “around” these issues.

Beyond using it for teaching, can you talk a bit about Arcade? What is it, how is it different, who might want to use it?

Arcade is great for sprite-based 2D games. It is simple and easy to get started with the library. There is no need to learn a complex framework. If you’ve wanted to create a game for fun, but Kivy, Unity, Unreal would just take more time to learn than what you’ve got, Arcade is a better choice. If you want to quickly create a scientific visualization without a lot of overhead code, Arcade can help you there too.

Arcade uses OpenGL and Pyglet. With OpenGL acceleration, it can draw a lot of sprites fast.

I use Arcade for my PyCharm tutorials and Arcade’s Python 3.6 type hinting is a big reason. Can you talk about your experience, as a library author and teacher, with type hinting?

New programmers often get confused when calling functions and methods. What data does it expect? And when the program doesn’t run, the students aren’t yet experts in reading stack traces. So they are stuck with a 500 line program that doesn’t work and they don’t know why. Frustrating.

Type hinting can sometimes tell students that they are passing unexpected data to the function. It does this before they run the program. Before they’ve even moved to writing the next line of code. It seems trivial, but it’s not. I found students able to create better, more complex programs because PyCharm and type hinting kept them from that error and allowed them to move on.

You also use PyCharm CE with your students. What’s been your experience having beginners start with an IDE?

I’ve taught students with an IDE and without an IDE. The biggest advantage is how the IDE can help students catch errors early. PyCharm’s built-in PEP-8 checking is huge. Also, built in spell-checking! Imagine trying to read program comments written without a spell-checker from today’s student. Students come up with some interesting ways to spell words.

Posted in Interview | Leave a comment

PyCharm 2017.2 RC

We’ve been putting the finishing touches on PyCharm 2017.2, and we have a release candidate ready! Go get it on our website

Fixes since the last EAP:

  • Docker Compose on Windows issues with malformatted environment variables
  • Various issues in Django project creation
  • Incorrect “Method may be static” inspection
  • AttributeError during package installation
  • And a couple more, see the release notes for details

As this is a release candidate, it does not come with a 30 day EAP license. If you don’t have a license for PyCharm Professional Edition you can use a trial license.

Even though this is not called an EAP version anymore, our EAP promotion still applies! If you find any issues in this version and report them on YouTrack, you can win prizes in our EAP competition.

To get all EAP builds as soon as we publish them, set your update channel to EAP (go to Help | Check for Updates, click the ‘Updates’ link, and then select ‘Early Access Program’ in the dropdown). If you’d like to keep all your JetBrains tools up to date, try JetBrains Toolbox!

-PyCharm Team
The Drive to Develop

Posted in Early Access Preview | Tagged | Leave a comment

Webinar: “Teaching Python 3.6 with Games” with Paul Craven, August 2nd

Want to teach Python? Want to use Python 3 type hinting in your applications? Want to write games? In this webinar, we look at all three, together, using the Arcade library for Python and its creator, Paul Craven.

  • Wednesday, August 2nd
  • 17:00 European Time, 11AM Eastern Daylight Time
  • Register here

paul_craven_splash

Paul Craven is the creator of Arcade, a 2d game library, and author of Program Arcade Games with Python and Pygame. He’s also a college professor teaching beginning programming using Python and game-writing.

He has settled on Arcade, Python 3.6 and PyCharm CE in his instruction. Arcade is written for the purpose of teaching, and thus heavily uses Python 3.6 type hinting. In this webinar, Paul will discuss:

  • Teaching programming using Python and game-writing
  • 2d games, and how Arcade was designed to help teach students
  • How Python 3.6 type hinting, combined with an IDE, helps both teaching and writing a framework

Speaking to you

Paul Vincent Craven (@professorcraven) is head of the Computer Science Department at Simpson College in Indianola, Iowa. He has a Ph.D. in computer science from the University of Idaho, and Masters from Missouri S&T. He worked for several years in IT before becoming a full-time professor. He finds that teaching students to program video games is a lot more fun than writing software to track mortgages.

Posted in Webinar | Tagged | 2 Comments

PyCharm 2017.2 EAP 7

The release of PyCharm 2017.2 is only a couple of weeks away. This week we have the seventh early access program (EAP) version ready for you, go to our website to get it now!

New in this version:

  • Many bugs have been fixed
  • Any virtualenvs detected for a certain project will now be visible only for that project
  • For more details, see the release notes.

Please let us know how you like it! Users who actively report about their experiences with the EAP can win prizes in our EAP competition. To participate: just report your findings on YouTrack, and help us improve PyCharm.

To get all EAP builds as soon as we publish them, set your update channel to EAP (go to Help | Check for Updates, click the ‘Updates’ link, and then select ‘Early Access Program’ in the dropdown). If you’d like to keep all your JetBrains tools updates, try JetBrains Toolbox!

-PyCharm Team
The Drive to Develop

Posted in Early Access Preview | Tagged | 3 Comments

PyCharm Edu: Tips & Tricks for Most Efficient Learning, Part I

Today we’re entering the home stretch with the PyCharm Edu 4 RC (build 172.3460) that is now available for download and try!

Learning something new is not only about getting new knowledge or mastering new skills, it is also about building new habits and getting the most joy out of something. That’s why with this blog post we wanted to start a series of posts covering learning methods and tips and tricks designed to help you to learn more effectively and make you more comfortable and excited with learning Python in PyCharm Edu. It may also help to set up productivity habits that will be quite useful for further professional Python development with PyCharm. So, let’s start!

Make your IDE feel like home

While coding, as well as learning how to code, it is very important to feel comfortable. The development environment should suit your needs and preferences and help you to stay focused and avoid distraction. You can use the default settings but the option is there for you to easily configure your environment in a way that makes sense to you if you want to.

Use keyboard shortcuts
Go to the dark side
Stay focused with a minimalistic UI
Find any action with ease

Continue reading

Posted in Cool Feature, Early Access Preview | Tagged | 1 Comment

PyCharm 2017.1.5 Out Now

We’ve released a minor update for PyCharm 2017.1. In this update several bugs have been resolved:

  • Python debugging would generate ‘unknown opcode’ errors.
  • Empty lines in test runner output
  • Bugs which caused PyCharm to freeze
  • For details, read the release notes

Update PyCharm now by downloading the new version from our website, or by using the automatic updates either in PyCharm, or JetBrains Toolbox.

PyCharm Team
-The Drive to Develop

Posted in Release Announcements | Tagged | 10 Comments

Remote Development on Raspberry Pi: Analyzing Ping Times (Part 2)

Last week we created a script that records ping times on a regular basis. We developed the script remotely on a Raspberry Pi, and then added it to Cron to make sure that times are recorded every 5 minutes into a PostgreSQL database.

This week we’ll work on visualizing the data we’ve recorded. For this we’ll create a basic Flask app where we use Matplotlib to create a graph. Furthermore, we’ll take a look at some cool PostgreSQL features.

Let’s see some results

It’s no good to just record pings if we can’t see some statistics about them, so let’s write a small Flask app, and use matplotlib to draw a graph of recent ping times. In our Flask app we’ll create two routes:

  • On ‘/’ we’ll list the destinations that we’ve pinged in the last hour with basic stats (min, average, max time in the last hour)
  • On ‘/graphs/<destination>’ we’ll draw a graph of the pings in the last 3 hours

The first route is simple, we just execute a query to get the data we’re interested in, and pass that to the template. See the full code on GitHub. Let’s make sure that everything works right by putting a breakpoint on the call to render_template:

Debug Flask Thumb

The graph route is a lot more complex, first we have to get the ping averages for the past three hours in reasonably sized bins (let’s say 10 minutes), and then we have to draw the graph.

To obtain those binned ping times, we could either get all times from the past three hours, and then use a scientific python library to handle the binning. Or we could write a monster SQL query which does everything for us. As I’ve recently read a book about PostgreSQL and got excited about it, I chose the second option.

Querying the Data

So the data we’re looking for is:

  • For each 10 minute period in the last 3 hours
  • Get the minimum, average, and maximum ping time to a specified destination

The first part makes this a fairly complex query. Even though PostgreSQL has support for intervals, date ranges, and a way to generate a series of dates, there is no way to generate a series of ranges (that I know of). One solution to this problem is a common table expression (CTE), this is a way to execute a subquery which you can later refer to as if it were a real table.

To get a series of timestamps in over the last three hours in 10 minute intervals is easy:

The generate_series function takes three arguments: begin, end, and step. The function works with numbers and with timestamps, so that makes it easy. If we wanted pings at exactly these times, we’d be done now. However, we need times between the two timestamps. So we can use another bit of SQL magic: window functions. Window functions allow us to do things with rows before or after the row that we’re currently on. So let’s add end_time to our query:

LEAD takes the value of the next row in the results, as ordered in the way specified in the over clause. You can use LAG to get the previous row in a similar way. So now we can wrap this query with WITH intervals as ( … query goes here … ) to make it a CTE. Then we can join our pings table and get the results we’re looking for:

Now you might think “That’s nice, but won’t it be incredibly slow?”, so let’s try it out! If you don’t see the ‘execute’ option when you right click an SQL query, you may need to click ‘Attach console’ first to let PyCharm know on which database you’d like to execute your query:

Execute Query Thumb

At the time of writing, my pings table has about 12,500 rows. And this query takes about 200-300ms. Although we could say that this is acceptable for our use case, let’s have a look at how we could speed this up. To see if there’s a way to improve the query, let’s have a look at the query plan:

Explain Analyze No Index

EXPLAIN ANALYZE shows us both how PostgreSQL decided to retrieve our results, and how long it took. We can see that the query took 471 ms. This is a bit painfully slow, and in the query plan we can see why: there’s a nested loop, and then a sequential scan. This means that for each of the 18 time buckets (6 buckets per hour, 3 hours), we do a full table scan. Right now the table fits in memory, so we first load the table in memory, and then we scan it 18 times in memory (you can see loops=18 on the materialize node). Imagine how slow this will be after the Pi collected a year’s worth of pings.

We can improve though, we’re querying our pings by the recorded_at column, using a ‘>=’ and a ‘<’ operator. A standard B-tree index supports these operations on a timestamptz column. So let’s add an index:

Now let’s look at the output of EXPLAIN ANALYZE again:

Explain Analyze With Index

5.7ms: much better.

Graphing the Data

After getting the data, matplotlib is used to generate a line graph with lines for the minimum, average, and maximum ping time per bin. Matplotlib makes it easy to plot time-based data using the plot_date function.

When the plot is ready, it’s ‘saved’ as a PNG to a StringIO object, which is then used to create an HTTP response. By setting the content_type header to image/png, everything is arranged.

So let’s take a look at the final result:

Ping page

If you want to see the full code, check out analyze.py on GitHub.

Querying the Data with Pandas

If the query above is a little much for you, you can achieve the same results with a couple of lines of code using the Pandas library. If we’d like to use Pandas, we can use a simple query to obtain the last three hours of pings, and then use the resample method to place the times in 10-minute buckets.

Important: To load PostgreSQL data into a Pandas dataframe, we need to have SQLAlchemy installed as well. Pandas needs SQLAlchemy for all database engines except SQLite.

Then we can do the same as with the SQL query, and use Matplotlib to plot it:

 

Now let’s have a look at what’s faster, the large SQL query, or a simple SQL query and Pandas. Pandas uses Numpy for math, which is largely written in native code for high performance. We add the code to get the appropriate data in both ways to benchmark.py, creating two functions: get_with_sql() and get_with_pandas(). We can use the Python standard library’s timeit function to run the methods 1000 times, and then get the total time it took to execute the function.

Let’s open the Python console, and take a look:Timeit Results

In other words: using Pandas it takes about 86ms to obtain the data, and with the large SQL statement it takes under 24ms. In other words: Pandas takes 262% longer. We’ve found that with a larger dataset this gap widens further.

In this specific case, it takes about 800ms to generate the graph, with the vast majority of that time taken by Matplotlib. So if we were really looking to improve performance, we’d hand off charting to a JavaScript library and just provide the data as a JSON object from the server.

Final Words

While working on this blog post, there have been a couple of times that I forgot that I was working on a remote computer. After setting up the remote interpreter, PyCharm handles everything in the background.

As you can see, PyCharm makes developing code for a remote server very easy. Let us know in the comments what projects you’re interested in running on remote servers! We’d also appreciate your feedback about SQL, let us know if you’d like to see more SQL content (or less of course) in further blog posts!

Posted in Tutorial | Tagged , , , , | 2 Comments

PyCharm 2017.2 EAP 6

We’re approaching the release of PyCharm 2017.2, and we’ve been busy putting the finishing touches on it. This week we have the sixth early access program (EAP) version ready for you, go to our website to get it now!

New in this version:

  • Many bugs have been fixed
  • React Native applications are now debugged using headless Chrome rather than Node.js
  • For more details, see the release notes.

Please let us know how you like it! Users who actively report about their experiences with the EAP can win prizes in our EAP competition. To participate: just report your findings on YouTrack, and help us improve PyCharm.

To get all EAP builds as soon as we publish them, set your update channel to EAP (go to Help | Check for Updates, click the ‘Updates’ link, and then select ‘Early Access Program’ in the dropdown). If you’d like to keep all your JetBrains tools updates, try JetBrains Toolbox!

-PyCharm Team
The Drive to Develop

Posted in Early Access Preview | Tagged | 2 Comments

Remote Development on Raspberry Pi: Checking Internet Quality (Part 1)

We all know that ISPs have a habit of overselling their connections, and this sometimes leads our connections to not be as good as we’d like them to be. Also, many of us have Raspberry Pi’s laying around waiting for cool projects. So let’s put that Pi to use to check on our internet connection!

One of the key metrics of an internet connection is its ping time to other hosts on the internet. So let’s write a program that regularly pings another host and records the results. Some of you may have heard of smokeping, which does exactly that, but that’s written in Perl and a pain to configure. So let’s go and reinvent the wheel, because we can.

PS: For those of you wanting to execute code remotely on other remote computers, like an AWS instance or a DigitalOcean droplet, the process is exactly the same.

Raspberry Ping

The app we will build consists of two parts: one part does the measurements, and the other visualizes previous measurements. To measure the results we can just call the ping command-line tool that every Linux machine (including the Raspberry Pi) ships with. We will then store the results in a PostgreSQL database.

In the part 2 of this blog post (coming next week) we’ll have a look at our results: we will view them using a Flask app, which uses Matplotlib to draw a graph of recent results.

Preparing the Pi

As we will want to be able to view the webpage with the results later, it’s important to give our Pi a fixed IP within our network. To do so, edit /etc/network/interfaces. See this tutorial for additional details. NOTE: don’t do this if you’re on a company network, your network administrator will cut your hands off with a rusty knife, don’t ask me how I know or how I’m typing this.

After you’ve set the Pi to use a static IP, use raspi-config on the command line. Go to Advanced Options, choose SSH, and choose Yes. When you’ve done this, you’re ready to get started with PyCharm.

Let’s connect PyCharm to the Raspberry Pi. Go to File | Create New Project, and choose Pure Python (we’ll add Flask later, so you could choose Flask here as well if you’d prefer). Then use the gear icon to add an SSH remote interpreter. Use the credentials that you’ve set up for your Raspberry Pi. I’m going to use the system interpreter. If you’d like to use a virtualenv instead, you can browse to the python executable within your virtualenv as well.

Create Project Thumb

After you’ve created the project, there are a couple things we need to take care of before we can start coding. So let’s open an SSH terminal to do these things. Within PyCharm press Ctrl+Shift+A, then type and choose ‘Start SSH session’, then pick your Raspberry Pi from the list, and you should be connected.

Start SSH Session Thumb

We now need to install several items:

  • PostgreSQL
  • Libpq-dev, needed for Psycopg2
  • Python-dev, needed to compile Psycopg2

Run sudo apt-get update && sudo apt-get install -y postgresql libpq-dev python-dev to install everything at once.

After installing the prerequisites, we now need to set up the permissions in PostgreSQL. The easiest way to do this is to go back to our SSH terminal, and run sudo -u postgres psql to get an SQL prompt as the postgres user. Now we’ll create a user (called a ‘role’ in Postgres terminology) with the same name as the user that we run the process with:

Make sure that the role in PostgreSQL has the same name as your linux username. You might also want to substitute a better password. It is important to end your SQL statements with a semicolon (;) in psql, because it will assume you’re writing a multi-line statement until you terminate with a semicolon. We’re granting the pi user login rights, which just means that the user can log in. Roles without login rights are used to create groups.

We also need to create a database. Let’s create a database named after the user (this makes running psql as pi very easy):

Now exit psql with \q.

Capturing the Pings

To get information on the quality of the internet connection, let’s ping a server using the system’s ping utility, and then read the result with a regex. So let’s take a look at the output of ping:

All lines with individual round trip times begin with ‘64 bytes from’. Let’s create a file ‘ping.py’, and start coding: we can first get the output of ping, and then iterate over the lines, picking the one that start with a number and ‘bytes from’.

At this point if you run the code (Ctrl+Shift+F10), you should see this code running, remotely on the Raspberry Pi:

Run code on SSH

If you get a file not found problem, you may want to check if the deployment settings are set up correctly: Tools | Deployment | Automatic Upload should be checked.

Storing the Pings

We wanted to store our pings in PostgreSQL, so let’s create a table for them. First, we need to connect to the database. As our database is only exposed to localhost, we will need to use an SSH tunnel:

Connect DB Thumb

After you’ve connected, create the table by executing the setup_db.sql script. To do this, copy paste from GitHub into the SQL console that opened up right after connecting, and then use the green play button.

Now that we’ve got this working, let’s expand our script to record the pings into the database. To connect to the database from Python we’ll need to install psycopg2, you can do this by going to File | Settings | Project Interpreter, and using the green ‘+’ icon to install the package. If you’d like to see the full script, you can have a look on GitHub.

Cron

To make sure that we actually regularly record the pings, we need to schedule this script to be run. For this we will use cron. As we’re using peer authentication to the database, we need to make sure that the script is run as the pi user. So let’s open an SSH session (making sure we’re logged in as pi), and then run crontab -e to edit our user crontab. Then at the bottom of the file add:

Make sure you have a newline at the end of the file.

The first */5 means that the script will be run every 5 minutes, if you’d like a different frequency you can learn more about crontabs on Wikipedia. Now we also need to create the log file and make sure that the script can write to it:

At this point you may want to grab a cup of coffee, and when you come back a while later some ping times should have been logged. To verify, let’s check with PyCharm’s database tools. Open the database tool window (on the right of the screen) and double click the table. You should see that it contains values:

Ping table

Read the second part of this blog post to learn how to analyze the ping times, and graph them with Matplotlib.

Posted in Tutorial | Tagged , , , , | 5 Comments

PyCharm 2017.2 EAP 5

Today, we’re happy to announce that the fifth early access program (EAP) version of PyCharm 2017.2 is now available! Go to our website to download it now.

New in this version:

  • We have fixed many bugs, especially in Python code inspections
  • We’re working on improving Jupyter Notebooks, you can try our experimental new run configuration now!
  • See the release notes for details

Please let us know how you like it! Users who actively report about their experiences with the EAP can win prizes in our EAP competition. To participate: just report your findings on YouTrack, and help us improve PyCharm.

To get all EAP builds as soon as we publish them, set your update channel to EAP (go to Help | Check for Updates, click the ‘Updates’ link, and then select ‘Early Access Program’ in the dropdown). If you’d like to keep all your JetBrains tools updates, try JetBrains Toolbox!

-PyCharm Team
The Drive to Develop

Posted in Early Access Preview | Tagged | 2 Comments