In this article we explore the results of using the user story analysis tool, ScopeMaster on an example set of user stories (access originals here). The stories we have used are published by Mike Cohn on the Mountain Goat website, they describe the functions of an interactive website to be built.
About the Example User Stories
We have chosen these example user stories as they are freely available and are published by a well known author of books on the subject of writing software user stories. They are examples that anyone can download. (The PDF actually contains examples for three different projects). For this exercise we have just looked at the stories of the ScrumAlliance website . Download the CSV with the example user stories
The purpose the exercise is to explore what ScopeMaster can reveal about a typical set of user stories.
Preparation
Step 1. Convert the PDF into a CSV
The PDF document contains a bullet point list of user stories. To prepare the list for import, we first copied the each user story onto a row in a spreadsheet. The spreadsheet was then saved as CSV ready for importing into ScopeMaster. This was the most time-consuming part of the whole exercise! Fortunately most user stories these days are contained in a structure that can readily be formatted as a CSV file and so this step would typically be avoided.
In total there are 98 user stories. We took the opportunity to generate a unique reference id for each one before importing them. You can re-use the csv that we created if you wish, here: Original Mountain Goat example user stories – ScrumAlliance CSV
Step 2. Import into ScopeMaster
We then created a new project in ScopeMaster and selected the “import CSV” option
We recommend the separation of the functional part of the user story from the benefits text, so we asked ScopeMaster’s import to do this for us.
Step 3. Analyse All
We then kicked off the automated analysis. Just by pressing the green button.
This is where the magic happens. ScopeMaster’s analysis engine works on each user story in turn, it is interpreted, analysed, tested, sized and cross-referenced.
In just under 3 minutes, ScopeMaster analysed every word of all 98 user stories and performed nearly 112,000 tests.
Results
Now let’s take a look at what ScopeMaster tells us about these user stories.
Each User Story: Interpreted, Tested and Sized (if possible)
- ScopeMaster examined each word of each user story to determine if it can establish clear functional intent. Is it clear?
- If so, ScopeMaster determined who is the user and what they want/need to achieve. ie. Is it user-oriented and functional?
- From the functional intent determined above, ScopeMaster estimated the functional size in ISO standards COSMIC function points. What is the size?
Insight of the Example User Stories
The use case model diagram was generated from the interpretation of the set of user stories. This diagram instantly reveals insight into many quality aspects of the set of user stories that are just not visible in requirements repositories such as Trello, Jira, Azure DevOPs. This diagram reveals, the logical relationship between user stories. This diagram helps you to determine if you have the right requirements?
From User Stories to Data Model
From the functional intents determined by ScopeMaster a suggested class diagram of all the probable classes, and their methods is determined from the text of the user stories. This can help developers understand what data they need to consider and how it might be stored and handled. This is not a definitive class diagram but a very useful starting point. This diagram helps to determine how to organise and structure the data to meet the requirements?
Generated Tests Directly from User Stories
Determining the Backlog Size
For all teams, all projects, user stories vary in size and the effort to deliver them. This variability can be very high. ScopeMaster was designed from the outset to automate functional sizing of written software requirements. This means it was designed to eliminate or reduce the administrative work of reading requirements to determine a formal functional size estimate of the software that they describe. Functional size is a technology agnostic, user-oriented indicator of magnitude. ScopeMaster estimates functional size in the two main ISO standards for sizing software, namely COSMIC Function Points and IFPUG Function Points. We recommend the former as it is more suited to modern software architectures and handles incomplete requirements better. ScopeMaster estimates the functional size of all the stories in your entire backlog in a matter of minutes, saving the team from wasting time on valueless discussions about story points or T-shirt sizing.
Quality Analysis
In just under 3 minutes ScopeMaster identified hundreds of problems with the user stories.
Ambiguities
51 of the 98 user stories had ambiguous functional meaning, which if left unresolved would lead to bugs. 51 defects identified.
Missing Basic Security User stories
There was no mention of login, authenticate, authorize, permission or logout throughout the user stories. This is evidently an important omission. We typically expect this to require at least several user stories. (although the “forgotten password” story is there). 5 defects identified:
• Login
• Validate email
• Change password
• Logout
• And then with each role sensitive user story there woul need to be a role and/or permissions check
The original set of user stories did not completely describe the required functionality.
Inconsistent terms for Users/Personas
ScopeMaster identified 26 potential user types. It is likely that there are actually only 10 distinct personas. 16 defects identified.
Inconsistent terms for Users/Personas
Similarly ScopeMaster identified 41 potential object types, whereas there are probably only about 25-30. 16 defects identified.
Completeness of Data Maintenance (CRUD analysis)
For each valid Object type there should be at least 1 function for each CRUD event. After eliminating the redundant object types, this leads to about 80 missing functional user stories. 80 defects identified.
Capability Outliers
The Use Case Model Diagram revealed further missing requirements based on role/personas. We estimate at least 1 or 2 per user type. 10 defects identified
Value Statements
ScopeMaster identified that only 58 of the 98 have used the phrase “so that”, however all but 3 do include the term “so”. 3- 40 defects identified
NFRS
Furthermore, ScopeMaster’s Non-Functional Requirement detection identified that there are several relevant categories of NFRs that were not mentioned in this set of user stories: accessibility, auditability, observability. 3 Defects identified
Quality Results Summary
In just 3 minutes, ScopeMaster identified 211 likely problems (97 + 114) with these user stories. It has also inferred via the CRUD analysis that only half the probable necessary user stories are actually listed. (this example set of user stories may have been deliberately trimmed before publishing).
Furthermore ScopeMaster generated over 140 essential test scenarios, which will help speed up the functional testing initiative, once the functionality has been created.
The purpose of this exercise was not to criticise the example user stories but to highlight the value of using ScopeMaster on any set of user stories.
We welcome any comments by the folks at Mountain Goat.
Conclusion
From this exercise, we can see that ScopeMaster performs a valuable job on this example set of user stories. It has:
- Revealed over 200 problems that can be readily resolved before inadvertently becoming the cause of rework.
- Revealed missing scope
- Revealed a potential data design
- Produced usable foundation test scenarios for each user story, including both positive and negatives tests.
- Estimated the functional size of each story, (including the missing ones) in ISO standards of functional size.
All this adds up to a substantial amount of useful, insightful, shift-left work that helps to bring certainty to the scope and reduce risk of any software project.
It would be great to hear from Mike Cohn or anyone who worked on the ScrumAlliance website of their experience of using these user stories and about their thoughts on ScopeMaster’s analysis. So far nobody from his team has been available to comment.