JetBrains BuildServer FeedBack Day: How Internal Feedback Helps Us Develop TeamCity

Alexander Rassokhin

We want everyone to have the best possible experience with TeamCity, regardless of the size of their team, the complexity of their project, or the technologies they use. In this post, I want to share how we integrate internal feedback from JetBrains teams into our development process.

Modern software development practices rely on short iterations and continuous feedback, which allow you to quickly validate your ideas against real users:

• A common way to run usability research is to recruit a bunch of people, give them a test scenario, and observe their experience while they try to complete your tasks.
• Another approach is diary research, in which participants live with your product for a week or two, write down their everyday impressions, and then you analyze their diaries and interview them to learn what went right and what went wrong.

The problem with the above methods is that they work great for products that are designed for individual use, but it’s hard to scale them and use them for teamware. Every team is unique, and has its own processes, habits, and expectations. Different people have different opinions, and the organizational difficulty of UX research grows exponentially with the number of participants. Finding a team that is ready to change their internal process and migrate to a new tool just for the sake of someone’s research is practically impossible.

But luckily, we have an amazing opportunity to test Teamcity on other teams inside our company.

Testing nightly builds inside JetBrains

TeamCity is used in thousands of projects across JetBrains, including such behemoths as Kotlin and IntelliJ IDEA. Our internal BuildServer has over 40,000 build configurations and runs over 300,000 builds every day of the year, giving us a tremendous amount of usage statistics and access to over 800 users that work with it on a weekly basis.

BuildServer runs exclusively on nightly builds of TeamCity, which means that teams inside JetBrains get all the latest changes shortly after they appear in the TeamCity master branch. If something is broken, or not performing well, we immediately get overwhelmed with complaints in our issue tracker or via Slack. The same happens when we add a new feature and people can’t figure out how to use it. Having so many users that are ready to provide logs, explain why something doesn’t meet their expectations, and help you to find the right solution is just priceless. The feedback allows us to be confident that our EAP releases are stable and that all important scenarios work correctly.

If you’re interested in learning more about how we at JetBrains use our own tools to develop our own tools, check out this blog post about how we do dogfooding.

Taking internal feedback further

However, this process is not ideal either. Feedback suffers from an inherent, fundamental problem in that it is always provided only by the most active users. And behind them, there always is a “silent majority” – users who prefer to find workarounds instead of reporting bugs and suggesting improvements. We never hear anything from them, and never know what kind of problems they may have.

So how do we give a voice to this silent majority? To make it easier for them and for us, we recently organized an internal BuildServer Feedback Day, where everyone from JetBrains can learn what’s happening in TeamCity, participate in an open discussion, and share their experience. Most colleagues joined us online, but some lucky ones (those working in our Munich office, which is not locked down) could come in person and not only talk to us, but also enjoy some tasty snacks and Bavarian beer. 🥨 🍺

This worked like a charm! Some people joined the event as listeners – at first they were not ready to speak out about their problems, even with direct questioning. As time went on, everyone became more relaxed and used to this format, and it became easier to involve them in the conversation. Once this happened, they started sharing their personal feedback, and it was absolutely invaluable.

As an outcome of our BuildServer Feedback Day, we have created or re-prioritized over 30 issues in our YouTrack project. Here’s the list:

• TW-68000 – No way to choose multiple filters for the builds lists
• TW-67656 – Dependencies Timeline: show queued builds
• TW-68050 – Simplified version of the IDEA plugin
• TW-58812 – Kotlin DSL configuration – path in the repository for versioned settings should be configurable
• TW-64859 – Branch selector is reset after changing the build page to project/build configuration page
• TW-68016 – Improve "My investigations" order
• TW-68015 – Free space isn’t optimally used on projects \ build configuration history pages in the experimental UI
• TW-68004 – Favorites page: projects and build configurations sections are moving during loading the page
• TW-68013 – Newly added branches are shown in UI only after page refresh
• TW-67622 – Add the text "Show" for the hidden sidebar
• TW-68001 – Button "Select all" for tests on the Build Overview page is confusing
• TW-68007 – Build Configuration Home/ Project Home links in build configuration/project settings are not obvious
• TW-68011 – Run custom build dialog: allow specifying an arbitrary branch name
• TW-68009 – Build page, failed tests section: consider renaming "show all" to "show all failed"
• TW-68006 – Information in the inactive (opened in new tab) tab isn’t loaded in experimental UI
• TW-62232 – Consider keeping the Favorites section always at the top
• TW-59968 – No need to change the projects list in the sidebar if the search field is selected and empty. Only after typing the letters
• TW-68002 – Actions button for tests and build problems is unnoticeable
• TW-67270 – Add buttons "Create project…" and "Configure sidebar…" to Favorite Projects page
• TW-64315 – New Queue page
• TW-67990 – Promote build dialog: don’t close the window after clicking on the run button
• TW-55062 – Show tests in the failed order in the build page
• TW-67988 – No agents from the profile are shown on the agents’ page while the cloud profile is loading (or is disabled)
• TW-66130 – Build log tab in tail mode by default is inconvenient for running builds
• TW-62983 – Reason for build failure is collapsed in the build log in the experimental UI
• TW-66159 – Implement full log text search in new UI
• TW-67986 – Viewing full test stack trace on build results requires additional clicks/scrolling
• TW-67999 – It’s not obvious that information about build problems can be expanded
• TW-49886 – Support transitive promote
• TW-67989 – Build log: Home shortcut not working
• TW-67642 – Slow rendering of builds on the Dependencies tab (new UI)
• TW-67092 – Wrong number of pending changes are displayed in the tab counter after changing the branch
• TW-63950 – Show build dependants in new UI
• TW-67649 – Build page, dependencies tab: let the user switch the status filter regardless of the page loading progress
• TW-59315 – Prevent outdated investigations

Subscribe to TeamCity Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!