Interview: Python Development at OpenStack with Swapnil Kulkarni
In this interview with Swapnil Kulkarni, a Solutions Architect at Opcito Technologies where he contributes to different OS initiatives, we’re going to look under the hood of the OpenStack project, where more than 95% of the entire codebase is in Python. Swapnil is a long-time core contributor to a few major subsystems and the core reviewer in the requirements team at OpenStack. We’re going to learn about the project, how they develop there, what their technology preferences are and some of the reasons they chose Python as the main language. Swapnil is also managing hundreds of PyCharm licenses at OpenStack, so we’ll ask him what developers value the most from it.
- Hi Swapnil, could you tell us a little bit about yourself?
I have 9+ years software development experience. I started my career as an intern and made my graduate project with a local embedded software development company. Since then I have worked for a range of different companies like Persistent Systems, Calsoft, Red Hat, and Xoriant. I’ve worked with multiple different technologies from Java, .NET to Python and in a lot of different roles from L3 Support to my current position as a Solutions Architect at Opcito where I am a contributor to Open Source initiatives and cater to different solution requirements for cloud, container, devops projects. I was part of OpenStack Storage projects and Compute projects earlier. I have been working with the containers team in the OpenStack community for more than 3 years now, and have worked on multiple use cases to utilize the container ecosystem in OpenStack. One of the most popular use-cases is the containerized deployment of the entire OpenStack with the project Kolla. I also work with the OpenStack requirements team as a core reviewer.
- For those of us who have probably heard of OpenStack, but haven’t spent much time with it, could you tell us what it is all about?
As you might have heard earlier, OpenStack started as an Open Source cloud operating system with initial contributions from NASA and Rackspace. It provided a much-needed alternative platform for enterprises to look at an Open Source private cloud alternative. It started with only a Compute (Nova) and Object Storage (Swift) component and has since then acquired more than 250+ active technology partners with active developers contributing to different features to make it a viable private cloud deployment player. OpenStack has always had the edge with emerging technologies and always pulled the cutting edge technologies into its projects for the users to use. E.g. The way OpenStack projects adopted containers before its general availability has provided a range of different use-cases already developed in the ecosystem.
- Tell us about how OpenStack is organized as an open source project?
The different projects, initiatives, and events in the OpenStack ecosystem are governed by the OpenStack Foundation. The source code is mirrored on Github, and everything is reviewed with Gerrit. The technical decisions related to projects and community are done by a Technical Committee formed by a community-wide open election process from all active contributors. There are 100+ actively contributed projects in the ecosystem that are categorized into multiple streams such as Compute, Storage, Networking, QA, Deployment. Each stream has a number of project groups. Each project group has contributors in a similar domain. The most active and consistent contributors are responsible for maintaining the projects in the group with merge rights and are called Core reviewers. With more than thousands of incoming reviews every week the community thrives on the infrastructure contributed by multiple companies like RackSpace, and Vexhost to perform the continuous integration and delivery jobs.
- How big is the OpenStack core team?
Each OpenStack project has its own core team. The size is dependant on the number of diverse contributors and incoming project reviews. It could range from a small core team of 5, to a team of 50 or more for some of the larger project groups. Each project group is part of a broader stream with community-wide goals. Apart from the development teams, there are special interest groups like the Operators group who collaborate and focus on requirements and issues with OpenStack deployments. We have a user-committee for getting feedback from OpenStack users. And a Technical Committee which works as a catalyst between the users, developers, operators, and the Foundation board members.
- Could you give us a high-level overview of the projects at OpenStack?
OpenStack community has projects for multiple requirements for the needs of a complete private cloud deployment. The projects are classified into multiple streams like Compute, Storage, Networking, etc as discussed earlier. There are core projects which are required for the cloud deployment and then there are supporting projects for deployment and monitoring. There is also a class of research projects which utilize the emerging trends in technology and map the use cases to the user requirements in the private cloud space. If you’re interested in more details, there’s a nice project navigator on the OpenStack website which guides you through the whole ecosystem.
- What are the main languages and technologies generally used at OpenStack?
More than 95% of entire OpenStack codebase is in Python. For any requirements related to web development, we use the Django framework. The rest are the supporting components which include tools like shell script, ansible, puppet, chef or configuration management, and deployment.
- What’s is so special about Python and why did you choose it as the main language?
First of all, Python is great for rapid development. It is much faster to develop a new feature than from all the other languages around… Ok, it’s not the only one, but the point is – deliverability. OpenStack has a very fast paced development model and it often needs to work with subsystems which require a language which has rich technical features and robust libraries. The language is required to be object-oriented and dynamically typed. Python satisfies those needs 100%. It’s also clean and easy to read and comprehend, that’s why we absolutely love it. Also, the decision to choose Python was mostly driven during the early stage of OpenStack development based on the regional availability of resources, compatibility with partners, security, as well as the personal expertise of the devs at the time.
- What kind of Python development are you and/or your team doing?
There are multiple streams of Python development in OpenStack teams. We have the APIs for each module, the scheduling algorithms, the components to give native calls to different hardware components such as hypervisors, and storage subsystems. Python is also used for different requirements in the web dashboard development with the Django framework.
- Which Python version is currently in use at OpenStack?
In the initial days, OpenStack started with Python 2.6. The entire code base was then converted to Python 2.7 and 3.x compatible. Python 2.6 is now no longer supported and hasn’t been since the last few releases; there is a separate effort to implement complete support of Python 3.x in all OpenStack components and the required libraries.
- What do you think are the main development challenges for OpenStack developers?
Working with a huge project like OpenStack requires integration points at multiple internal projects as well as external supporting components. It requires single as well as cross-project debugging, integration with Git/Gerrit, managing requirements, real-time debugging, using integrations like vagrant and docker are just some of the major challenges.
- There are many core developers at OpenStack using PyCharm for their development. How does PyCharm help them be more productive?
There are around 200 core developers at OpenStack making use of OS PyCharm licenses generously provided by JetBrains. I’m managing the internal distribution of licenses. The number of PyCharm users in the community varies a lot since many users do not wish to use the IDE all the time and sometimes prefer to use native editors like Vim. Some users also like to use tools like Eclipse or VS Code which they are comfortable with from their previous development environments.
PyCharm is an IDE, so it always has tons of advantages when compared to text editors in terms of supported functionality. With respect to Python development, PyCharm definitely stands out with features like remote debugging, code quality checks, and integrations with third party software like Docker. The features most prominently used with PyCharm within my team are remote debugging and Docker integration.
- Does PyCharm help boost team productivity on your project?
PyCharm works very well for most users. It provides support for project-wide code style conformance and version control. Many developers also find the quick-fixes quite useful and save a lot of time when fixing small bugs and doing code reformatting. Additionally, the debugging support makes the development teams life easier to verify the feature development. You can also use the Gerrit plugins to manage your code reviews from the IDE itself.
- What about your individual productivity?
I am a contributor to multiple containerization projects within OpenStack, to start with having an IDE with support for Docker integration really makes it a lot easier for me. It also helps me do some additional tests with containers for some Python projects which are not containerized yet.
- How did you personally first learn about PyCharm?
I came from a Java development background, and I have mostly used Eclipse for my development. When I switched to Python for OpenStack development, I wanted to have an IDE that I could rely on. I started with Eclipse and a Python plugin, which helped me initially but I needed something which is specifically designed for Python, and that’s when I first came across PyCharm.
Thank you for the interview, Swapnil!
Thanks for having me!
To stay in touch with Swapnil, please follow him on Twitter.
Subscribe to PyCharm Blog updates
