In this article we explore the results of using the user story analysis tool, ScopeMaster on an example set of user stories (access originals here). The stories we have used are published by Mike Cohn on the Mountain Goat website.  The stories describe the functions of an interactive website to be built.

Example User Stories - analysed by ScopeMaster

About the Example User Stories

We have chosen these example user stories as they are freely available and provided by a reknowned expert and author of books on the subject of writing user stories. They are examples that anyone can download. (The PDF actually contains examples for three different projects). For this exercise we have just looked at the stories of the ScrumAlliance website .

The purpose the exercise is to explore what ScopeMaster can reveal about a typical set of user stories.

Preparation

Step 1. Convert the PDF into a CSV

The PDF document contains a bullet point list of user stories. To prepare the list for import, we first copied the each user story onto a row in a spreadsheet. The spreadsheet was then saved as CSV ready for importing into ScopeMaster.

Fortunately most user stories these days are contained in a structure that can readily be formatted as a CSV file and so this step would typically be avoided. In total there are 98 user stories. We took the opportunity to generate a unique reference id for each one before importing them. You can re-use the csv that we created if you wish, here: Original Mountain Goat example user stories – ScrumAlliance CSV

Step 2. Import into ScopeMaster

We then created a new project in ScopeMaster and selected the “import CSV” option

Use Case Model Diagram from example user stories set

We recommend the separation of the functional part of the user story from the benefits text, so we asked ScopeMaster’s import to do this for us.

Step 3. Analyse All

We then kicked off the automated analysis. Just by pressing the green button.

Use Case Model Diagram from example user stories set

This is where the language analysis happens, each user story in turn is then interpreted, analysed, tested, sized and cross-referenced.

In just under 3 minutes, ScopeMaster analysed every word of the user stories and performed nearly 112,000 tests.

Use Case Model Diagram from example user stories set
ScopeMaster analysed the entire backlog of 98 user stories in less than 3 minutes

Results

Now let’s take a look at what ScopeMaster tells us about these user stories.

Each User Story: Interpreted, Tested and Sized (if possible)

  • ScopeMaster examined each word of each user story to determine if it can establish clear functional intent. Is it clear?
  • If so, ScopeMaster determined who is the user and what they want/need to achieve. ie. Is it user-oriented and functional?
  • From the functional intent determined above, ScopeMaster estimated the functional size in ISO standards COSMIC function points. What is the size?

Insight of the Example User Stories

The use case model diagram was generated from the interpretation of the set of user stories. This diagram instantly reveals insight into many quality aspects of the set of user stories that are just not visible in requirements repositories such as Trello, Jira, Azure DevOPs. This diagram reveals, the logical relationship between user stories. This diagram helps you to determine if you have the right requirements?

Use Case Model Diagram from example user stories set
The functional interpretation of the set of user stories is very insightful and dynamically maintained.

From User Stories to Data Model

From the functional intents determined by ScopeMaster a suggested class diagram of all the probable classes, and their methods is determined from the text of the user storiesThis can help developers understand what data they need to consider and how it might be stored and handled.  This is not a definitive class diagram but a very useful starting point.  This diagram helps to determine how to organise and structure the data to meet the requirements?

Partial
Autosuggested classes and methods from the set of user stories

Generated Tests Directly from User Stories

Example pseudo code for test scenarios, generated from the text of the user story

Determining the Backlog Size

For all teams, all projects, user stories vary in size and the effort to deliver them. This variability can be very high. ScopeMaster was designed from the outset to automate functional sizing of written software requirements. This means it was designed to eliminate or reduce the administrative work of reading requirements to determine a formal functional size estimate of the software that they describe. Functional size is a technology agnostic, user-oriented indicator of magnitude. ScopeMaster estimates functional size in the two main ISO standards for sizing software, namely COSMIC Function Points and IFPUG Function Points. We recommend the former as it is more suited to modern software architectures and handles incomplete requirements better. ScopeMaster estimates the functional size of all the stories in your entire backlog in a matter of minutes, saving the team from wasting time on valueless discussions about story points or T-shirt sizing.

Automated sizing of a set of user stories
ScopeMaster automatically sizes user stories in standardised COSMIC Function Points

Quality Analysis

In just under 3 minutes ScopeMaster identified hundreds of problems with the user stories.

Improved reporting of defects detected (on the summary screen)
ScopeMaster automatically sizes user stories in standardised COSMIC Function Points

Ambiguities

51 of the 98 user stories had ambiguous functional meaning, which if left unresolved would lead to bugs. 51 defects identified.

Missing Basic Security User stories

There was no mention of login, authenticate, authorize, permission or logout throughout the user stories. This is evidently an important omission. We typically expect this to require at least several user stories. (although the “forgotten password” story is there). 5 defects identified:

• Login
• Validate email
• Change password
• Logout
• And then with each role sensitive user story there woul need to be a role and/or permissions check

The original set of user stories did not completely describe the required functionality.

Inconsistent terms for Users/Personas

ScopeMaster identified 26 potential user types. It is likely that there are actually only 10 distinct personas. 16 defects identified.

Inconsistent terms for Users/Personas

Similarly ScopeMaster identified 41 potential object types, whereas there are probably only about 25-30. 16 defects identified.

Completeness of Data Maintenance (CRUD analysis)

For each valid Object type there should be at least 1 function for each CRUD event. After eliminating the redundant object types, this leads to about 80 missing functional user stories. 80 defects identified.

Capability Outliers

The Use Case Model Diagram revealed further missing requirements based on role/personas. We estimate at least 1 or 2 per user type. 10 defects identified

Value Statements

ScopeMaster identified that only 58 of the 98 have used the phrase “so that”, however all but 3 do include the term “so”. 3- 40 defects identified

NFRS

Furthermore, ScopeMaster’s Non-Functional Requirement detection identified that there are several relevant categories of NFRs that were not mentioned in this set of user stories: accessibility, auditability, observability. 3 Defects identified

Quality Results Summary

In just 3 minutes, ScopeMaster identified 236 likely problems with these user stories. It has also inferred via the CRUD analysis that only half the probable necessary user stories are actually listed. (this example set of user stories may have been deliberately trimmed before publishing).

Furthermore ScopeMaster generated over 140 essential test scenarios, which will help speed up the functional testing initiative, once the functionality has been created.

The purpose of this exercise was not to criticise the example user stories but to highlight the value of using ScopeMaster on any set of user stories.

We welcome any comments by the folks at Mountain Goat.

Conclusion

From this exercise, we can see that ScopeMaster performs a valuable job on this example set of user stories.  It has:

  • Revealed over 200 problems that can be readily resolved before being the cause of rework.
  • Revealed missing scope
  • Revealed a potential data design
  • Produced usable foundation test scenarios for each user story, including both positive and negatives tests.
  • Estimated the functional size of each story, (including the missing ones) in ISO standards of functional size.

All this adds up to a substantial amount of useful, insightful, shift-left work that helps to bring certainty to the scope (and reduce risk) of any software project.

It would be great to hear from Mike Cohn or anyone who worked on the ScrumAlliance website of their experience of using these user stories and about their thoughts on ScopeMaster’s analysis.  So far nobody from his team has been available to comment.