Make It Workflow — Part 11: Updating Batches of Issues
I’m happy to announce that after almost a year-long break I’m back with the “Make It Workflow” series. Back then, we published a whole load of posts dedicated to talking you through various YouTrack usage scenarios and how to set up the workflows for them. Through this series so far, we’ve covered topics such as restricting issue visibility, generating time reports, and enhancing mailbox integration. You are welcome to check out the “Make it Workflow” series if you haven’t done so already.
We are reviving Make it Workflow today with our next installment in the series: updating issues. You may be wondering why this topic in particular? After all, it is part of the core functionality of the workflows, and we have discussed it already many, many times before. This post though is going to be a little bit different, today we are all about scaling the process and altering many issues at once.
Replacing Commands with Silent On-schedule Rules
Imagine the following scenario: you have a project which utilizes Assignee and Subsystem fields. And at some point, you decide to enable the “Subsystem Assignee” workflow, which allows setting the Assignee based on a Subsystem value. Doing so will save a lot of time for your team, as the users don’t usually know who is responsible for which subsystem. However, this workflow is not retroactive: it doesn’t affect issues where users have set Subsystem field before enabling it. So now you have a bunch of unassigned issues which you need to assign according to their subsystems.
How can such an update be performed? I hear you frustratedly scream. Well, it is possible with the help of search and commands:
- Search for issues by `#Unresolved #Unassigned Subsystem: {Landing page}`.
- Execute a command `Assignee mariyadavydova`, using notification-less command option.
- Repeat for the next subsystem.
This solution is very useful if you have a small number of subsystems. But there are still perils to be aware of such as clicking on a wrong button in the command dialog and burying your colleagues in a mountain of notification clutter. Also, as the issues will be manually updated, the default issue order (by updated date) will lose all its sense for several weeks if not months.
Again, it’s workflows to the rescue. In YouTrack 2018.2 we introduced an incredibly useful `muteUpdateNotifications` property for on-schedule rules. It makes them “silent”: the server doesn’t send update notifications on the changes made with these rules. The changes are out of sight out of mind, so you won’t have to deal with a load of disgruntled colleagues.
Using a silent on-schedule rule in this scenario has a few terrific benefits:
- No manual updates: you write the rule, enable it for the project, wait until all the issues are updated, and then detach it.
- No issue order breakage: changes by on-schedule rules do not modify the updated date.
- No notifications: users don’t get hammered by email notifications because of the `muteUpdateNotifications` property.
The rule looks like this:
var entities = require('@jetbrains/youtrack-scripting-api/entities');
var search = require('@jetbrains/youtrack-scripting-api/search');
exports.rule = entities.Issue.onSchedule({
title: 'Set Assignees by Subsystems',
cron: '0 * * * * ?',
search: '#WS-1',
muteUpdateNotifications: true,
action: function(ctx) {
var searchQuery = '#Unresolved has: Subsystem has:-Assignee sort by: {issue id} asc';
var issues = search.search(ctx.issue.project, searchQuery);
var entries = issues.entries();
var i = entries.next();
var n = 0;
var firstIssueId = i.done ? '' : i.value.id;
while (n < 100 && !i.done) {
var issue = i.value;
issue.fields[ctx.Assignee.name] = issue.fields[ctx.Subsystem.name].owner;
n += 1;
i = entries.next();
}
var name = ctx.issue.project.name + ' : Set Assignees from Subsystems : ';
if (n) {
console.log(name + n + ' issues are processed, starting with ' + firstIssueId);
} else {
console.log(name + 'no issues are processed, as nothing is left to process');
}
},
requirements: {
Subsystem: {
type: entities.OwnedField.fieldType
},
Assignee: {
type: entities.User.fieldType
}
}
});
The code looks a bit daunting, let me try and help by explaining how it works. The main idea is we have defined an on-schedule rule which runs once a minute (`cron: ‘0 * * * * ?’`) and this updates one hundred issues or less (`while (n < 100 && !i.done)`) on every run. The number you choose to iterate over depends on the complexity of a single issue update. For example, in this case, we can safely update 200-300 issues every minute without risking overloading the YouTrack server, because all we are doing is simply reading a Subsystem and setting an Assignee. If the update procedure was more complex and involved a lot of operations, it would make sense to decrease the single run update limit to 100 or maybe even 50 issues per minute.
This rule uses an “anchor issue” approach (`search: ‘#WS-1’`). The anchor issue can be pretty helpful, it lets you pull the project that the issue belongs to into the context and iterate over other issues in the project. This guarantees that the rule runs exactly once per scheduled execution. For an anchor issue, create an issue with a description like “Please don’t ever delete this issue!” and set it to a resolved state. You can then reference its ID in the search property of your on-schedule rule. This technique was covered in detail in the ‘Generating Time Reports’ post of this series.
In the action section the rule looks for all the issues meeting our criteria (`’#Unresolved has: Subsystem has:-Assignee sort by: {issue id} asc’`), then updates the first one hundred or less of them, and logs the results in the workflow console. We sort issues by id in this search to sort the log entries also by issue id.
Now, if you attach this rule to your project and then check out the rule console after a few minutes, you will see something like this:
workflow_user_1990547510381747193 4 Mar 2019 15:54 Data Modifications : Set Assignees from Subsystems : 100 issues are processed, starting with WS-3 workflow_user_1990547510381747193 4 Mar 2019 15:55 Data Modifications : Set Assignees from Subsystems : 100 issues are processed, starting with WS-157 workflow_user_1990547510381747193 4 Mar 2019 15:56 Data Modifications : Set Assignees from Subsystems : 100 issues are processed, starting with WS-345 workflow_user_1990547510381747193 4 Mar 2019 15:57 Data Modifications : Set Assignees from Subsystems : 43 issues are processed, starting with WS-482 workflow_user_1990547510381747193 4 Mar 2019 15:58 Data Modifications : Set Assignees from Subsystems : no issues are processed, as nothing is left to process
The last line tells you that the issue processing has finished, and you can deactivate the rule.
Taking It One Step Further
Imagine now, that after running the workflow rule from the previous section you found that there are still a lot of unassigned issues in your project, because nobody has chosen a subsystem for them. But, you want to have all your issues assigned to a subsystem and a person. Is there a solution to this kind of problem other than taking on the terrible task of going through and updating all these issues manually?
Of course there is. Choosing the correct subsystem is sometimes difficult even for humans, nevermind machines, but it is still possible to reduce the amount of manual work. As a simple example, we can search for specific words in the issue summary and assign a related subsystem if we find them.
The rule for this solution looks very much like the previous one, everything that is except the issue update procedure. I have put it in a separate function `guessSubsystem` at the top of the script:
var entities = require('@jetbrains/youtrack-scripting-api/entities');
var search = require('@jetbrains/youtrack-scripting-api/search');
var guessSubsystem = function(ctx, issue) {
var summary = issue.summary.toLowerCase();
var value = null;
if (summary.includes('welcom') || summary.includes('landing')) {
value = ctx.Subsystem.Landing;
} else if (summary.includes('product') || summary.includes('catalogue')) {
value = ctx.Subsystem.Catalogue;
} else if (summary.includes('profile')) {
value = ctx.Subsystem.Profile;
} else if (summary.includes('cart')) {
value = ctx.Subsystem.Cart;
}
issue.fields[ctx.Subsystem.name] = value;
};
exports.rule = entities.Issue.onSchedule({
title: 'Guess Subsystem from summary',
cron: '0 * * * * ?',
search: '#WS-1',
muteUpdateNotifications: true,
action: function(ctx) {
var searchQuery = 'has:-Subsystem sort by: {issue id} asc';
var issues = search.search(ctx.issue.project, searchQuery);
var entries = issues.entries();
var i = entries.next();
var n = 0;
var firstIssueId = i.done ? '' : i.value.id;
while (n < 100 && !i.done) {
var issue = i.value;
guessSubsystem(ctx, issue);
n += 1;
i = entries.next();
}
var name = ctx.issue.project.name + ' : Set Subsystem from summary : ';
if (n) {
console.log(name + n + ' issues are processed, starting with ' + firstIssueId);
} else {
console.log(name + 'no issues are processed, as nothing is left to process');
}
},
requirements: {
Subsystem: {
type: entities.OwnedField.fieldType,
Landing: {},
Catalogue: {},
Profile: {},
Cart: {}
}
}
});
Script template
As you can see, these rules have a pretty similar structure. So I wrote the following template that you can steal. All you have to do is update the parts of this code marked with comments to your needs:
var entities = require('@jetbrains/youtrack-scripting-api/entities');
var search = require('@jetbrains/youtrack-scripting-api/search');
var TITLE = ''; /* Add a rule title. */
var SEARCH = ''; /* Add a search query for the issues you want to update. */
var LIMIT = 100;
var CRON = '0 * * * * ?';
var ANCHOR = ''; /* Add an id of an issue from the project you want to update. */
var updateIssue = function(ctx, issue) {
/* Update an `issue` here. */
};
exports.rule = entities.Issue.onSchedule({
title: TITLE,
cron: CRON,
search: ANCHOR,
muteUpdateNotifications: true,
action: function(ctx) {
var issues = search.search(ctx.issue.project, SEARCH);
var entries = issues.entries();
var i = entries.next();
var n = 0;
var firstIssueId = i.done ? '' : i.value.id;
while (n < LIMIT && !i.done) {
var issue = i.value;
updateIssue(ctx, issue);
n += 1;
i = entries.next();
}
var name = ctx.issue.project.name + ' : ' + TITLE + ' : ';
if (n) {
console.log(name + n + ' issues are processed, starting with ' + firstIssueId);
} else {
console.log(name + 'no issues are processed, as nothing is left to process');
}
},
requirements: {
/* Add rule requirements. */
}
});
Conclusion
There are many cases where such approaches can be applied. In our first post in this series, we discussed how to restrict issue visibility. If you wanted to apply the described scheme to an existing project, you could write a similar rule and update all the issues. If you chose to use the template above for this, the top part of the script may something look like this:
var TITLE = 'Restrict issue visibility';
var SEARCH = 'has: -{visible to}';
var LIMIT = 100;
var CRON = '0 * * * * ?';
var ANCHOR = '#WS-1';
var updateIssue = function(ctx, issue) {
issue.permittedGroups.add(ctx.viewers);
issue.permittedUsers.add(issue.fields[ctx.Assignee.name]);
};
Another example would be if you wanted to perform a transformation on large amounts of issues – tens or even hundreds of thousands. Recently we had to update over 260 thousand issues on our YouTrack server, and because of the possibility to automate this process it only took a couple of minutes to write the rule; YouTrack itself handled everything else.
Okay, that’s it for today. There is no strict schedule set out for the series, so I recommend that if you want to get informed about these posts as soon as they are out, that you to subscribe to our blog so that you don’t miss anything. Also, don’t forget to visit the YouTrack Community Slack, where you can speak to my colleagues and me and see how other users are using workflows to support their business logic.
Subscribe to YouTrack Blog updates
