Make It Workflow — Part 12: Storing Data between Workflow Runs

Mariya Davydova

Every now and then the question as to whether it is possible to store data between workflow executions comes up. For example, when a team lead wants to show an informational message to the entire team, once to each person. The most straightforward way to achieve this would be to add a flag “hasSeenTheMessage” to each user account. However, YouTrack doesn’t allow you to define properties like this via the workflow, though we are planning to support such use-cases in some form or another in the future.

In the meantime, I want to show you how to work around this problem using the existing workflow capabilities. OK, so before we dive into the code, let me first briefly explain how different types of workflow-accessible data are stored in YouTrack.

Storing data

From each workflow you can operate with three types of data:

Database

This type of data includes every issue, project, user, and other YouTrack entry. Every time you change anything in UI, an isolated snapshot is created and provided to the workflow rules. If all rules related to the change run without errors, the snapshot is then checked for consistency and dumped to the actual database. The crucial feature of this type of data is that it is persistent – it is all preserved after YouTrack restarts.

Workflow rule global variables

These kinds of data are better explained with an example. Let’s take a look at the start of the script template from my previous post:

var entities = require('@jetbrains/youtrack-scripting-api/entities');
var search = require('@jetbrains/youtrack-scripting-api/search');

var TITLE = ''; /* Add a rule title. */
var SEARCH = ''; /* Add a search query for the issues you want to update. */
var LIMIT = 100;
var CRON = '0 * * * * ?';
var ANCHOR = ''; /* Add an id of an issue from the project you want to update. */

All the variables in this snippet belong to the rule scope. That means they are defined once every time the rule is saved or the YouTrack server is started, and are not redefined on every workflow rule execution. As you might have seen, people often declare rule constants as global.

However, I strongly discourage you from modifying them inside the workflow rules. Not only are they reset often – which is especially true for InCloud servers because it is out of your control when the team restarts them – but they are also not thread-safe. That means that if two people change an issue simultaneously, it is impossible to predict the effect it will have on the global variable.

Workflow rule local variables

Local variables are data defined inside code blocks, like guard and action functions, and are limited to these blocks. For example, the on-change rule template that we have in YouTrack contains the following code:

action: function(ctx) {
  var issue = ctx.issue;
  // TODO: specify what to do when a change is applied to an issue
}

Here, the issue variable references an issue which is being processed by the workflow rule, which is different every time YouTrack runs this rule.

Showing a message to colleagues

So, as you can see, the only way for data to survive all the restarts is to keep it in a database. But, also, as I’ve said, YouTrack won’t let you add custom properties to database entries via APIs. Don’t worry though; I won’t leave you stranded, I have a workaround for the situation which uses the tools we have at our disposal.

In short, the idea is to save data in the issue description. This issue should be invisible to almost all users to avoid any accidental (or not so accidental) modifications. The data should also be saved in a format which is easy to read, write, and modify inside the workflow rules.

Let’s try it out on a simple example. Imagine that you want to surprise your colleges and congratulate them on a successful release with a message in YouTrack. You can write a rule, which will show a message on each modification, but these messages will soon become pretty annoying. A better solution would be to display a message to each person exactly once, so to implement this we need to store some data.

Let’s have a look at the rule:

var entities = require('@jetbrains/youtrack-scripting-api/entities');
var workflow = require('@jetbrains/youtrack-scripting-api/workflow');

exports.rule = entities.Issue.onChange({
  title: 'Congratulations!',
  guard: function(ctx) {
    return ctx.issue.isReported && !ctx.issue.becomesReported;
  },
  action: function(ctx) {
    // Retrieve data
    ctx.db.applyCommand('visible to All Users', ctx.db.reporter);
    var data = ctx.db.description;
    var map = JSON.parse(data);
    if (!map) {
      map = {};
    }

    // Do stuff
    if (!map[ctx.currentUser.login]) {
      workflow.message('Congratulations with amazing release!'); 
    }
    map[ctx.currentUser.login] = true; 
    // Save data 
    ctx.db.description = JSON.stringify(map, null, ' '); 
    ctx.db.applyCommand('visible to ' + ctx.db.reporter.login, ctx.db.reporter); 
    }, 
requirements: { 
    db: { 
     type: entities.Issue, 
     id: 'WS-280' 
    } 
    } 
});

I have mentioned a couple of requirements above; this rule covers both:

Issue visibility

Before executing the rule, we first create an issue (WS-280 in my case) with an empty description and make it visible only to the reporter. You may also want to set the summary to something like “Don’t touch this issue, seriously, I am not kidding, leave it alone!” and the State to Fixed, well, just in case.

Now, at the beginning of the action rule section, we open an issue which is visible to everyone. This trick is possible because in the workflow you can apply the command on behalf of any user (in this case, the issue reporter). The description is now accessible.

At the end of the action, we set the issue back to being visible to the reporter only. As both modifications happen inside one rule action, the issue is never actually visible to anyone else but the reporter (neither from UI nor via other workflows or REST calls).

Data storage format

We store the data as a JavaScript object, which is simply a collection of key-value pairs (where each value may be another object as well). JavaScript is good at reading and writing objects via JSON.parse(data) and JSON.stringify(map), respectively.

After reading the data, we will then have either an empty object ({}) or something that looks like this:

{
  'mariya.davydova': true,
  'scott.adams': true,
  'natasha.katson': true
}

Each entry means that a person with this login has already seen the message, and we do not want to show them it again.

Counting the number of changes

Okay, how about more complex use case?

Imagine the following scenario: as a team leader you want to analyze your team’s activity in your YouTrack project. To be precise, you want to know who does the most changes in issues every month. (I bet, it’s your QA team!)

To achieve this goal, we need to write two rules: the first one will count the number of changes, and the second one will send you a report on the 1st day of each month.

Let’s look at the rule which counts the changes. It is very similar to the previous rule, so I will just leave it here, but I won’t go into it:

var entities = require('@jetbrains/youtrack-scripting-api/entities');

exports.rule = entities.Issue.onChange({
  title: 'Counter',
  guard: function(ctx) {
    return ctx.issue.isReported;
  },
  action: function(ctx) {
    // Retrieve data
    ctx.db.applyCommand('visible to All Users', ctx.db.reporter);
    var data = ctx.db.description;
    var map = JSON.parse(data);
    if (!map) {
      map = {};
    }

    // Do stuff
    console.log(map);
    var month = new Date().getMonth();
    if (!map[month]) {
      map[month] = {};
    }
    console.log(map[month]);
    var login = ctx.currentUser.login;
    if (map[month][login] || map[month][login] === 0) {
      map[month][login] += 1;
    } else {
      map[month][login] = 0;
    }
    console.log(map[month][login]);

    // Save data
    ctx.db.description = JSON.stringify(map, null, '  ');
    ctx.db.applyCommand('visible to ' + ctx.db.reporter.login, ctx.db.reporter);
  },
  requirements: {
    db: {
      type: entities.Issue,
      id: 'WS-279'
    }
  }
});

The second rule, however, is slightly different:

var entities = require('@jetbrains/youtrack-scripting-api/entities');

exports.rule = entities.Issue.onSchedule({
  title: 'Reporter',
  search: '#WS-279',
  cron: '0 0 18 LW * ?',  
  action: function(ctx) {
    var data = ctx.issue.description;
    var map = JSON.parse(data);
    if (!map) {
      map = {};
    }
    var month = new Date().getMonth();
    var report = '';
    if (!map[month]) {
      report = 'No changes were made in previous month.';
    } else {
      var max = 0;
      for (var login in map[month]) {
        var line = 'User ' + login + ' made ' + map[month][login] + ' changes.\n';
        report += line;
        max = Math.max(max, map[month][login]);
      }
      var winners = [];
      for (login in map[month]) {
        if (map[month][login] === max) {
          winners.push(login);
        }
      }
      var winnerLine = winners.length === 1 ? 'The winner is ' + login + '.\n' : 
        'The winners are ' + winners.join(', ') + '.\n';
      report += winnerLine;
    }    
    ctx.issue.project.leader.notify('Activity report', report);
  }
});

This rule runs in the evening of the last working day of every month (cron: '0 0 18 LW * ?') and sends an activity report to the project leader. The report will look like this:

User mariya.davydova made 325 changes.
User scott.adams made 217 changes.
User natasha.katson made 286 changes.
The winner is mariya.davydova.

In this rule, I am exploiting the fact that on-scheduled rules can run on behalf of a dedicated system user, which can see any issue in any project. Therefore, the search property points to where the data is saved (in which issue), making it accessible via context (var data = ctx.issue.description).

The other code in this rule is an example of how you can go about manipulating the JavaScript objects.

Conclusion

We went through how to implement the two most popular scenarios when we need to store data between the workflow executions:

• Performing an operation a limited number of times per user, issue, project, or other limiting objects.
• Collecting information over time.

You may also want to collect some information over the list of issues. In this case, you don’t need to use data storage; instead, you can traverse over a bunch of issues in one rule. I showcased this approach in the post devoted to generating time reports.

That’s it for today. If you have any questions about this post or YouTrack Workflows in general, you are very welcome to join our YouTrack Community Slack. As usual, the source code for this installment can be found in our YouTrack Custom Workflow Repository.

Subscribe to YouTrack Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!