Research Hub: a new frontier of research
Maria Antropova, Head of the Market Research and Analytics Team, spoke with her colleagues — Cognitive Research Lead and Marketing Analyst Yanina Ledovaya and UX researcher Giulshan Dzhafarova — about changes in the team, new projects, and plans for the future.
Four years have passed since the last interview, in which you talked about yourself and marketing research at JetBrains. This time, we would like to talk about the transformation of the team. How has the team changed since then?
We are gradually transforming into a Research Hub – a research center that conducts research while also helping other employees at the company do their own independent research. We also communicate with university academics and carry out research together.
The team itself has expanded and split into several groups. Currently, the Market Research and Analytics team includes such sub-teams as Business & Market Insights, Surveys, UX-research, ResearchOps, Data Science, Research Core, Business Intelligence, Cognitive Research. Each one of them is focused on different aspects of research. For example, the Market Research and Analytics team used to deal with UX research tasks, but now there are more of them, and they are handled on a separate track – this is how the sub-team that deals with this area came about.
There has been an increase in the number of internal studies in which we look at the processes, the tasks, and the company from within. We carry out these tasks for HR and other teams when we need to answer certain questions about the company, and JetBrains serves as the subject of our research.
We have also developed the field of trending and naming research.
What other notable transformations have taken place?
Administrative work has expanded considerably. We have created and maintained our own panel, which helps us recruit respondents for qualitative and quantitative research.
We have automated some internal processes, such as giving out prizes to respondents for their participation in our research.
Has your interaction with other teams in the company changed?
As the team grew and sub-teams began to form, we needed to reorganize the way we worked. Each sub-team works independently, but at the same time they are all engaged in joint projects. It is very important not to lose this cohesion, which means we’re currently facing a major important task – building up relationships and processes between sub-teams.
One of the team’s big projects is the annual Developer Ecosystem survey, which helps us find out what’s going on in the development world. How has DevEco transformed in the past four years, and what’s it like now?
It has become very large, both in size and in the amount of effort that we put into it. When the first interview came out, we could no longer test it manually: the survey was so voluminous that it was almost impossible to go through all logical branches manually so as not to miss out on anything. Therefore, we have various automations that we use to test a survey. But it certainly doesn’t give us 100% protection from mistakes. So we spend the first couple of weeks after launch urgently fixing whatever minor issues come up.
We also have more infographic sections. We’re adding new technologies and generally changing the format of sections. Compared to the first Developer Ecosystem, we now have fairly fine-grained weighting mechanisms to avoid data bias that might otherwise be dictated by the characteristics of our sample and by how we collect this data.
We also have a productive partnership with the Python Software Foundation. For the last five years, we’ve conducted an annual survey for Python Developers. We collect data through official PSF channels and receive about 25,000 to 30,000 responses each year. Because of this, we consider this report to be quite representative.
In addition, we have been conducting a Django Developer Survey in partnership with the Django Software Foundation since 2021.
These surveys are a win-win for us and the PSF and DSF – they can learn about the community and identify directions for further improvement, and we can refine our products to better meet user needs.
Are there any approximate statistics as to how many surveys the team conducts on a monthly basis?
More than 15 a month.
Tell us, how has the technology used by the team changed over the past four years?
For data processing we still use R and Python. Now we are testing our own product, DataSpell, instead of R Studio. Also we have our own set of automations. This is a system that allows us to automatically create purchase requests for various gift certificates that we use to reward respondents for taking part in our research. We also have an application which was written by one of the team members. It manages the research panel and the contact details within it.
What specialists are part of the team?
We have a multidisciplinary team: There are developers, psychologists, economists, data analysts, anthropologists, and sociologists. So we have a fairly wide range of specialists who tackle various tasks.
Do you plan to bring anyone else on board this coming year?
We have a constant need for specialists. Now there is a lot of interest in the field of trend studies, so we will look for more people to satisfy this demand. We also really need a data journalist – a specialist who will write journalistic materials based on the data collected and analyzed sent in to them.
We also plan to develop a scientific track. Most likely, it will be based on the principle of cooperation with research institutes and universities that have relevant research departments, and we will solve various problems with them. Applied Research works in a similar way at Microsoft.
What other projects, besides research, does the team participate in?
We have infographics that we publish in the public domain: on the one hand, this is a study, and on the other, it’s a product. There are analytical blog posts that we started writing relatively recently. At this point we have already published two articles for which we collected data on various technologies and consulted with experts.
There are also personal recommendations – this is an individualized distribution of the results of the Developer Ecosystem. Each respondent receives an email about how his answers compare with the answers of the other participants.
Every month, we prepare an overview of important external analytical reports, which is included in the company’s internal digest.
We also launched a quick research format. Colleagues come to us with questions from our area of expertise, and we get back to them with an answer in a couple of hours or days. Typically, in this format, we deal with questions that we can find answers to in previous studies or available existing data. For example, we answered questions such as: “How many developers currently use WebAssembly?”, “How many developers are engaged in pair programming and on what basis (weekly, monthly)?” etc.
In addition, the Research Coffee format was launched. Colleagues can come and chat with us online about research, psychology, anthropology, sociology, and other social sciences. Since we have a lot of psychologists on our team, we can talk about the use of psychological methods and techniques in solving some applied problems. We are also ready to talk about other topics, ranging from procrastination to anthropological issues, such as “Why anthropology is not just about skulls and bones”.
How and what does the team learn to improve their skills?
We have internal ways of learning: These are retrospectives on the results of research in different subteams and workshops.
One of these is the training of research coordinators. At some point, my colleague and I realized that we needed to expand our expertise on the incoming task stream. We created a school of research coordinators within the team, where various issues were discussed in the case-study format for two weeks. We tried to do without lectures on how to coordinate research, and we used an interactive format. We simply prepared a list of questions and topics. In fact, we have outlined all the stages our research passes through. We recalled the problems that we encountered at one stage or another, discussed them, and formulated the principles of each stage. Now we meet once a month in the format of a support group and discuss the problems that arise in the process of work, as well as ways to solve them.
We also try to do a little learning outside the team. Colleagues take courses and bring that knowledge to the team.
Sometimes we also have team training. Last year we had an in-depth interview course. We took online facilitation training this year because we felt a strong need for these skills during the stages of discussing problems with initiators and applying research results. There are other training needs, which the Learning Development department helps us to take care of.
Are there any conferences you would like to attend?
We used to go to BigSurv, a conference dedicated to surveys. Our guys participate in Data Science conferences. In addition, some colleagues hope to attend such conferences as Advancing Research, EPIC, QRCA Annual Conference, Design Research, Why the World Needs Anthropologists, Cognitive Science Society, BigSurv.
As head of the team, what is it that helps you develop your team?
We have a lot of freedom at our company. And I, as Head of the Market Research and Analytics Team, also have the freedom to choose the direction of our research, аnd how we develop it – naturally and based on the needs of the company. It’s actually a very interesting creative process. You can do a lot of cool things, but the possibilities for team development are limited, relatively speaking, by our imagination, professionalism, and understanding of subject area trends.
Therefore, one of the important tasks that I am still striving to solve is how to keep my imagination stimulated, and my colleagues are doing the same. They look around, always in communication with colleagues from other companies, observing who has what and how it’s organized, what areas they are researching, and what are the trends in these areas. This helps us come up with new ideas. Moreover, it’s not enough just to look at the area you’re researching. You have to look at others as well – sometimes even the ones you’d never expect.
I try my best to maintain a good mood and culture in the team, so that the spirit of equality reigns inside, and everyone can realize their interests and opportunities. This is of paramount importance to me.
How do you imagine the team’s development in the future?
I hope we will focus on the scientific field, develop trend research, and integrate more closely with product teams. Now we are redeploying research coordinators to work with specific teams. This is so that in the future, each product team will have its own specialist on the research team who will be responsible for dealing with all of the team’s requests.
Over the past year, we have adjusted the information architecture within the team. We have guidelines that describe most of our processes. Good instructions were created for onboarding new employees. There is the Research CookBook that we wrote for the colleagues at the company who do research on their own but are not analysts. That is to say, product managers and colleagues from product teams. In a separate guide, we described all our internal systems, statistics and data sources. This helps new employees who are just joining the company to understand what research and analytics opportunities we have.
Fantastic! Thank you!
 |  |
