Interview: Oliver Bestwalter for tox webinar next week
Python has long distinguished itself with a culture of testing. In the last decade, two libraries have combined to give powerful testing in isolation — pytest and tox. The latter combines easily with pytest to give you a clean environment across test runs, including across multiple versions of Python.
tox certainly counts as one of those things lots of PyCharm customers know they should know, but don’t yet know. To make it easy to break the ice we’ve invited Oliver Bestwalter to introduce tox in a PyCharm webinar. Oliver is the maintainer of tox and advocate for release automation in projects.
Here’s our interview with Oliver. As background, Oliver was previously interviewed on The Mouse vs. Python blog and the Test & Code podcast.
Give us a sneak peak on what you’re going to discuss in the webinar and what audience it is aimed at.
I want to do a time lapse journey through a hypothetical project that grows tests and automation over its lifetime. The focus will be on how to use tox to bundle all these development and automation tasks into a workflow that works well locally and on CI.
Let’s go back in time. Can you give us your Python origin story?
The short version is: I stumbled over the tutorial in 2006 and fell in love. The longer version is in the interview you linked :)
You are a core maintainer of tox. What’s it like running a large, popular open source project?
When I joined the project, Holger Krekel (the original author) and the first generation of contributors had started to move on to other projects. Work around tox had cooled down, so I thought I’d help out a bit. There were quite a few open pull requests and issues that needed addressing. So I did that and after a while Holger asked me if I want to handle releases as well. I also made contact with plugin authors to gather tox plugins into the tox-dev organization to raise their visibility. My main focus was on keeping the project alive, fixing annoying bugs and creating a welcoming atmosphere. That’s a lot of work that doesn’t result in very much code, but I still think it was the right thing to do rather than to just hack away at the code.
Over the past few months I wasn’t quite able to even follow what is going on in the issue tracker for a number of private and professional reasons. At the moment my activities are concentrated more on teaching Python with a focus on testing and automation (with pytest and tox obviously). Luckily Bernát Gábor joined as a maintainer this year and is currently very active in the project (thank you Bernát!).
So to answer the question: in the beginning it was a good feeling to keep an important project alive and improve its processes, but for quite a while now it has also meant feeling guilty a lot of the time, because I can’t do more "direct" work for tox a lot of the time.
Testing is one part of your release automation vision. Can you discuss how other parts — pre-commit hooks, black, CI — fit together?
I think of all test and automation tools as helpers to build lines of defense (against bugs) in a development process. The first line of defense is static code analysis. Ideally the most obvious problems are already pointed out directly in the editor when they happen. PyCharm inspections and quick fixes are great for that if you crank them all up to 11. The next line of defense are linters and automatic fixers (like flake8 and black). The pre-commit framework wraps that and more into a neat package that can prevent you from accidentally committing code that has obvious problems. If you don’t use it locally the first stage in CI will fail where the same checks are run. The next line of defense are automatic tests. Depending on the nature of the project there might be more (like running tests against the deployed environment on different stages).
The role that CI plays for me is quite simple: CI is just a task execution and report collection tool and should contain as little process knowledge as possible. So nothing fancy should happen there that isn’t easily reproduced on a developer box. CI simply runs everything that the developers run locally and makes artifacts available for release. This means that the process knowledge needs to be encoded in the tox configuration and the automation scripts that live alongside the production code. The real world is often a bit more complicated and this isn’t always completely possible (often due to security considerations) but that’s the aim.
What’s been your experience using an IDE like PyCharm to visualize testing and the other pieces?
I am with Brian Okken on this: PyCharm is a really good GUI for pytest (see or better listen: Episode 48 of Test & Code)
The Test runner tab provides a good overview of the tests that were run and I also appreciate the many ways how I can navigate from the results back in to the relevant code sections. What I appreciate even more is the convenience of running the tests right from the code rather than having to switch to the command line. Also: auto-test – very handy. It’s also great that PyCharm even supports code completion and navigation for pytest fixtures in recent releases.
Look ahead 3 years from now. What’s next for tox and the release automation field?
My crystal ball is at the repair shop at the moment, but I’ll try :)
In respect to tox that really depends who steps up to the plate and champions the development of new features and improvements. If I get the chance, I want to improve interpreter discovery and squash some bugs. I also want to improve the documentation and make tox more accessible to newcomers. Regarding the recent changes in packaging (PEP-517 and PEP-518) tox already has everything in place. Projects using flit and other tools that leverage the sudden freedom from setup.py and setuptools can use tox, but I am sure there will be some kinks to iron out along the way. tox also needs to grow native support for pyproject.toml (meaning that you won’t need a tox.ini anymore and configure it there alongside all the other tools). In three years tox will also be 100% bug-free and powered by renewable energy :)
Regarding the greater landscape I really hope that the development will go even further into the direction we are already heading: making build, test and deploy automation as easy and accessible to the masses as possible. I’ll do my best to help with that.
Subscribe to PyCharm Blog updates
