Multi-team Collaboration
This document provide a framework as part of the Inner Sourcing initiative about how to measure the repositories contributions and describe the used concepts behind it.
Problem Description
We need to boost the contributions between different teams and people to different repositories, because:
- we want to to share the knowledge about the repositories more among the teams.
- we want to make the teams more self-sufficient and autonomous when they need to change something.
- we want to encourage the developers to improve the code of other repositories.
The problem is how to measure these level of contributions in a sensible way.
Currently, we have an organizational structure based on teams. For a team perspective, the work is managed using JIRA tickets and the teams usually works on some specific repositories.
Introducing Code Ownership, we also have a parallel structure based on code owners for every repository. From a code owners/contributors perspective, the work revolves around the repositories and people.
We usually propose the group aims using Key Performance Indicators (KPI) based on the teams performance, but the want to measure the contributions, that follow the code ownership structure, where there are people as responsible of the repository (code owners), a contributor and some additional reviewers, not all of them necessary in the same team.
Background
Usually, the teams have a backlog based on the requirements of the company. These requirements used to be about the repositories that the team has assigned. For example, for FX teams, they work with BOS and similar repositories.
We have some team working on repositories like eburyonline that usually requires to have changes in the BOS API to develop new features, but they need to delegate these kind of tasks to teams working on BOS.
In many cases, the developers are focused on some repositories, but they do not have the opportunity to know and understand well the repositories around theirs repositories.
The code owners concept allow us to establish people that take care about every repository and this is good, but also is good to have diversity of contributions from different teams to a repository and the autonomy to propose changes in a external repository to your usual ones and get the approval from this code owners.
Also the code ownership is a way to interact with other teams, guiding the development in the repository and marking coherency in the code. When the code owners are people out of your team, the vision is more enriching.
How to measure the contributions to repositories and the level of the qualities is a challenge, and how to drive the teams to collaborate with other teams is also complex to drive with KPI and targets.
We have some sources to extract information to measure these kind of things, from the git repository information to the Bitbucket API or the JIRA API with the tickets information. The first step is to know what we want to achieve and how to handle it. You can see in the next section that we have proposed a solution based on the teams collaboration, but there are other solutions based on other concepts.
We believe that it is important to measure the teams as creator of contributions and also as reviewers of contributions, looking for the interaction with other teams. In the next section we expose the proposed solution.
Solution
We propose to measure collaboration measuring contributions between teams.
We will consider only those pull requests directed against the integration branch, that were merged within a specific time range. We will ignore those repositories whose usage is limited to trigger deployments.
The key concept is the "social contribution": every merged pull request that was approved by people external to the author team.
We will take into account the team membership at the moment of merge.
The Code Ownership initiative involves configuration in repositories and enforcing default reviewers. That means that contributions from other teams will be automatically considered as a "social contribution".
Even if code owners are individuals and not teams, we are explicitly considering that getting approvals only from your teammates is not qualifying a contribution as "social".
The proposed metric contains two numbers that scores for the teams:
- the percentage of contributions that are social, among the ones authored by someone in the team.
- the percentage of contributions that are social, among the ones in which the team participates with at least one approval.
Metric 1 measures the collaborative aim of a team, and metric 2 measures the effort put into welcoming contributions.
To implement this solution we need the next base information available:
- Which person is on each team in every period (person email / team / started date).
- Which contributions has being done for the repository and the involved people (when / repository / author / approvers )
With this information, we can process the next data:
- For every contribution, the involved teams based on the approval people and the author team based on the creator person.
- For every contribution, if it is a social contribution and the team was involved as author or approval.
Finally, we can have something like this:
| Team | Social contribution as author | Social contribution as reviewers |
|---|---|---|
| ABC | 30% | 60% |
| XYZ | 20% | 10% |
In this example, we can see that the ABC team probably has some reviewers that are code owners or are required in reviews for other teams, usually are reviewed by other teams and had some individual tickets. In the case of XYZ team, the are very isolated in the contributions, the code written by them is rarely reviewed by external people and the do not review external code.
Please note that these two metrics do not add up to 100%.
Computation
To get these metrics, it is required to have some automatic way to extract the base information and then we could calculate the results in a spreadsheet based on both concepts.
The first sheet must be about team membership. Every row represents a name that belongs to a team, and the moment this starts. When someone changes teams, a new row can be added. This information shall be maintained by the teams.
The second sheet is the historical information about contributions. For every row, at least: the moment when pull requests were merged, the author email, and the list of approvers in following cells. This information can be extracted from the code repositories.
The third sheet is a computation over the second. Every author and approver's name is replaced by the team acronym. With this formula, we get the team that belongs to an name at a given moment: =iferror(VLOOKUP(contributions!D2, filter('team membership'!$A$2:$B, 'team membership'!$C$2:$C < $A2), 2), ""). We can group all the approvers teams with =join(";", UNIQUE($G2:2)). The social flag is computed only by checking the number of teams involved: =COUNTUNIQUE(F2:P2)>1.
The fourth sheet contains a summary. For every team (=UNIQUE('team membership'!B2:B)), we need the total (and social) contributions as author. Total: =countif('contributions-computed'!$F$2:$F, concat("=",$A2)). Social: =countif(filter('contributions-computed'!$F$2:$F, 'contributions-computed'!$D$2:$D), concat("=",$A2))). Also, the total (and social) contributions as a reviewer. Total: =counta(filter('contributions-computed'!$E$2:$E, REGEXMATCH('contributions-computed'!$E$2:$E, $A2))) and. Social: =counta(filter('contributions-computed'!$E$2:$E, 'contributions-computed'!$D$2:$D, REGEXMATCH('contributions-computed'!$E$2:$E, $A2))).
With these four numbers, a new pair of columns (here or in another sheet) can host the metrics: =iferror(summary!$C2/summary!$B2,"--") and =iferror(summary!$E2/summary!$D2,"--").
Alternatives
As alternatives, we have considered the next ones:
- Measure the contributions to a repository considering the tickets in JIRA completed by a team that usually does not work with this repository. In this case you need to have a list of usual repositories by team and fill always the component field of every ticket to do a reliable metric. We trust more in the Bitbucket registry than the manual filling in JIRA.
- Measure the contributions based on the git repository. In this case, you only have the commit messages with the approvals content and you need to know about which branch is the integration branch in every repository. We propose to use the Bitbucket API to extract the information about the approvals and creator using the integration branch concept.
- Measure the number of commits or number of lines in a contribution. We want to count the contribution as a single point. We should have contributions as small as possible with the pull request philosophy. The number of commits or lines depends on the way every developer works.
- Measure the quality of every contribution. We are going to count the number of contributions and we are going to discard initially the story points of the JIRA story or the kind of contribution.
Caveats
The proposed solution is not fully automated, so it requires some manual work. Some caveats are related to the possible corner cases in the formula. It could be improved with the usage.
Operation
These metrics will be available for the team leads and there must be a responsible to update the spreadsheet based on the scripts.
Security Impact
From a data security perspective, the sensible information is related to developers emails and the team to which they belong.
Performance Impact
N/A
Developer Impact
From the developer perspective, the expected impact is to measure up the collaboration with other developers out of the developer's own team. Also, it should encourage to make changes to other repositories and finally to have more code owners in every team.
Data Consumer Impact
If the base information is extracted in an automated way and set available for internal usage for distinct consumers, we will increase the possibilities to create new metrics based on this information. This proposal does not change existing data, so there is not an impact on other components.
Deployment
N/A
Dependencies
N/A
References
N/A