.NET Tools
Essential productivity kit for .NET and game developers
A Story About .csproj, Large Solutions and Memory Usage
We discussed the motivations and our push toward running Rider on .NET Core in our previous post. As part of that effort, we are looking into converting projects in the ReSharper and Rider solution to using the new, simpler, SDK-based projects introduced with .NET Core.
In this post, we will see why we want to do this migration. We’ll talk about how we ran into some development-time issues when using the newer project style with large solutions, and how that led into finding a similar issue with the “old .csproj” format. We’ll also share a workaround with you. Let’s dive in!
Moving ReSharper and Rider to SDK-Style Project Format
SDK-style projects are built on top of the Common Project System (CPS), and are a much simpler format than the “old .csproj” format. They typically have an Sdk="..." attribute defined, which specifies the tools and additional build targets that will be loaded for a project. For .NET Core projects, the SDK will often be Microsoft.NET.Sdk or Microsoft.NET.Sdk.Web.
Referencing these SDKs simplifies project files a lot. Many details about how our project should be built are abstracted away in the SDK. In “classic” projects, all files in a project would be listed separately. In an SDK-style project, files are included by convention, and only files that deviate from default rules have to be specified in full.
SDK-style projects also make multi-targeting easier. Adding a target framework to the TargetFrameworks property in a project file is sufficient for many projects to support multiple .NET targets. Of course, there will still be conditional dependencies, and we’ll have to use preprocessor directives to conditionally include other portions of code in many places. But overall, the project structure will become simpler, even when multiple target frameworks are involved.
Right now, we have already converted several projects to the new SDK-style project format, but it takes time to validate and verify. So while we’re nowhere near completing this migration, we do expect it to simplify our build toolchain and development process when we finish.
Migrating to using SDK-style projects is technically not required, yet it will help us compile Rider against .NET Core, while we can keep ReSharper on .NET Framework. And thanks to less complex project files, we also expect to have fewer conflicts while merging branches in our codebase!
Memory Usage with Large Solutions
As you may know, Rider shares a lot of its codebase with ReSharper. This is great, as functionality can be shared easily! For example, when we write a new code inspection, it will be available in both Rider and ReSharper. Of course, changes in the project and build infrastructure in one product, also impact the other.
The ReSharper solution consists of close to 700 projects, with many code files in each of them. For cross-project refactorings and changes, we have to work with that 700-project solution.
Our developers don’t have to open all projects in our monorepo all of the time, though. We generate smaller solution files that contain just the necessary projects to work on one product. For example, the generated Rider backend solution contains 162 projects (250 if we add unit test projects). For ReSharper, the generated solution has 294 projects:
Tip: Try to generate a solution with 300 projects, each containing 200 classes, if you want something that compares in size.
Our ReSharper and Rider teams work in their IDE of choice. We ran into an issue with the ReSharper code base when opening it in Visual Studio, after converting some shared projects to using the SDK-style project format. Visual Studio would be unresponsive for longer periods of time, or crash completely. Memory usage with the newer project format was also consistently higher.
Investigating the issue further, we found that this was caused by excessive allocations in the Common Project System (CPS). After taking a closer look, we found the issue was also present with the “old .csproj”, albeit slightly better – which probably explains why we did not notice it before. We reported this to Microsoft, who are looking into this based on a repro we provided.
After opening the 162-project Rider solution in vanilla Visual Studio (without ReSharper installed), the process uses ~2 GB of memory, with 1.12 GB reserved for .NET. Much of the memory usage goes to string duplicates and C# syntax trees, as we can see in dotMemory:
Tip: In dotMemory, either attach to the running “devenv.exe” process and capture a snapshot, or load a memory dump file (.dmp) for further analysis.
Keep in mind that this is an IDE, where we want coding assistance and all kinds of analyzers that help us write better code, faster. That means there is a solution model that keeps track of a project tree, and that for each project all of the source code (strings) has to be parsed into syntax trees to be able to do something useful. These objects will typically be long-lived, ending up on higher memory generations. As such, the memory usage is not entirely unexpected.
Things get more interesting when we realize all this is in a 32-bit process, where memory is not unlimited. There’s a lot of memory pressure, and a lot of garbage collection going on as a result.
Reducing Memory Usage with Large Solutions
We’re working on moving ReSharper out of process to overcome part of the memory pressure problem we saw above, to make sure we’re not adding extra memory pressure. But we’re not there yet.
Our own teams are using ReSharper (and Rider) to develop ReSharper (and Rider), which means that they don’t use some of the default features in Visual Studio at design-time. ReSharper overrides them anyway, so why load them in memory?
We found a solution, or rather a workaround, to reduce the overall memory footprint of Visual Studio, by disabling some defaults:
- Code analysis, and related quick-fixes and refactorings
- Code completion
- Navigation
While this cannot be done directly, we can do this by overriding how design-time builds “see” our project. By removing all items that are marked as “compile”, and instead including them as embedded resources during a design-time build, we are essentially preventing these files being included in Visual Studio’s code analysis and IntelliSense.
In MSBuild format:
When this snippet is added into a Directory.Build.targets file next to our solution file, this will apply to all project files in our solution. This works for SDK-style projects, and for “old .csproj” projects (MSBuild 15 or newer).
Note: It’s probably not a good idea to commit this snippet to source control, unless it is intended to enable this technique for everyone on the team.
We also recommend enabling Color Identifiers in the ReSharper options (under Code Inspection | Settings), as otherwise some highlighting may be missing.
The Results
In vanilla Visual Studio, this technique reduces memory usage by 169 MB on a 300-project sample solution we generated. The .NET memory footprint was reduced by 18%, from 909 MB to 740 MB. Using this technique in vanilla Visual Studio is obviously not the best idea, as many IDE features will stop working.
With ReSharper enabled, we’ll still have code completion, code analysis, quick-fixes, and refactorings. On a similar solution, Visual Studio with ReSharper was using 1293 MB of memory. Using this Directory.Build.targets technique, we saw a reduction of 213 MB, or 16%, dropping memory usage to 1080 MB.
Another solution we tried this with was the Roslyn compiler solution. It has 66 projects, but lots of code and references. Without using our Directory.Build.targets file, managed memory usage is ~1.11 GB:
Adding the Directory.Build.targets file, we see a significant drop in managed memory usage! From ~1.11 GB, down to ~687 MB. That’s a 37% reduction!
Conclusion
Keep in mind that when using this technique, code completion, code analysis, quick-fixes, and refactorings are provided only by ReSharper. Custom Roslyn code analyzers will also not be available, as this technique effectively hides code from Visual Studio. Another side effect is that Code Lens will not be available.
Should you be using this in your solutions? As always, it depends. If you are using Visual Studio and ReSharper for a solution with many projects, it never hurts to try, as long as you and your team are okay with the above constraints.
If you do use our Directory.Build.targets file, keep us informed about how it goes! And if you encounter any issues with it, let us know here.
Subscribe to a monthly digest curated from the .NET Tools blog:
