Monte Carlo Simulation is a powerful statistical method for modeling the probability of various outcomes in different processes. Named after the famous Monte Carlo Casino in Monaco, this technique employs random sampling or historical data to provide highly accurate probabilistic estimates. For example, the graph of potential investment outcomes shown by your stock broker is likely the result of a Monte Carlo simulation.

Monte Carlo simulation can also used to predict project delivery dates or product release dates with high accuracy. This guide will focus on using Monte Carlo Simulation to forecast delivery dates. For a broader strategic overview, read Unlocking the secrets to accurately forecast product releases.

Cycle Time

Before diving into the details, let’s briefly talk about cycle time. Cycle time is the total time from the beginning to the end of a process, including processing time and any waiting time. It is a critical metric in manufacturing, software development, IT service management – in fact, in any department or industry interested in tracking delivery speed. By using Monte Carlo Simulation, we can model and predict cycle time variability, helping us manage expectations and margins.

Probability Distributions

Once you have your cycle time data, create a histogram to understand the probability distribution of your process. For service delivery or software engineering, your distribution is likely similar to the one outlined here. This is not a normal distribution. However, if most tasks are completed around the average (or median), with some variation, you’ll get a familiar bell curve which is a normal distribution

While this step is not strictly necessary for Monte Carlo simulation modeling, it is helpful for understanding the data—specifically, the average, median, range, and variation. This analysis provides insights into the effectiveness of current processes and helps identify areas for improvement.

Creating the Monte Carlo Simulation

To implement Monte Carlo simulation, Python is highly recommended for its versatility and power. Although I’m not a programmer by trade, I find it accessible, powerful and easy to use. Alternatively, you can use Excel.

Understanding The Logic Behind The Monte Carlo Method

Let’s explore the thought process and logic behind the Monte Carlo method, focusing on predicting delivery dates.

You likely have estimated delivery dates based on work estimates or story point velocity. Let’s validate these using Monte Carlo simulation. As outlined above, you are tracking cycle time of completed work. Let’s use that.

Now refer to the glass jar filled with numbered balls. Each ball contains a unique number representing the cycle time of each completed task. I like to wait until 5-10 tasks are complete before beginning to validate the projected end dates.

• Pick a ball from the jar (representing the cycle time of a completed task).
• Note the number and return the ball to the jar.
• Shake the jar and repeat the process.
• Perform this process 10,000 times or more.
• Repeat for all incomplete tasks.

Once you have simulated the cycle times for each incomplete task, plot the probability distribution of all the incomplete tasks using random sampling. The x-axis will represent time (days, weeks, or months), depending on your data.

Interpreting the Monte Carlo Simulation Results

• Mean (Average) Completion Time: 7.65 weeks.
• Median Completion Time: 7.64 weeks, indicating a symmetric distribution.
• Standard Deviation: 0.64 weeks, showing less variability.

As a Product or Delivery Manager, you now have probabilistic estimates to set expectations and understand impacts on margins. Shift left for a more aggressive date with lower confidence levels or shift right for a date with higher confidence levels. And now to tell a story. Here’s an example:

Based on the rolling 6-week average completion rate of 8 items per week, we have an 85% chance of achieving a release/delivery between 8 and 10 weeks (replace with dates). Key factors impacting our weekly throughput include ambiguity in scope and acceptance criteria, and the reduced allocation of key team members’ time on the product. Here are three items that need executive attention…

Final Thoughts

Monte Carlo Simulation is a versatile and powerful tool that can be applied across various industries and processes. Whether you are managing software development projects, manufacturing lines, or service deliveries, this technique provides valuable insights into your processes’ performance and future outcomes. By adopting Monte Carlo Simulation, you can improve your project management practices, enhance predictability, and make data-driven decisions that lead to better outcomes.

This guide is a starting point, and as you become more familiar with Monte Carlo Simulation, you can customize and expand your models to fit your specific needs and challenges.

By following the steps outlined in this guide, you are now equipped to implement Monte Carlo Simulation and leverage its benefits to drive success in your projects.

The Python Script

The full set of Python scripts can be downloaded and modified for free in my Github repo. The following are snippets of the Python code the logic can be applied if you choose to implement this using a spreadsheet.

Since all my clients have used Jira as their workflow management tool, the code has a Jira slant. Moreover, since all my clients have usually refused to invest further into the Jira marketplace tools, I’ve had to write my own code to extract the data and perform the analysis.

Step 1 – Collect Data

Assumption: 
Your underlying data is contained within a csv with the following format:

[["Jira Key","Summary","IssueType","Current Status","Ticket Created On", "From status", "To status", "Date changed","Time in From Status (days)","Release","Components","Labels","Sprint","Date Completed","Epic Link","Age in the last WIP Status","Jira Change ID", "WIP Category","Done Year", "Done Week", "Year Week"]]

Step 2 – Read the csv

The code assumes you are interested in the current release. Adjust the code as per your requirements. We are also filtering out user-defined ticket types and workflow states. If your process includes “work types” or “classes of service”, that will need to be coded in. Feel free to reach out to me. I would be interested in understanding what Jira fields you use to track “class of service”.

# The following python libraries will be needed
import numpy as np
import pandas as pd
from datetime import datetime, timedelta
import subprocess
import ast
import json
import matplotlib.pyplot as plt
import scipy.stats as stats
import sys
from collections import defaultdict
import os
import math

# Step 2: Read the CSV File and Filter tickets
def read_csv(file_path,releases, exclude_from_status, issue_types, wip_category_included,excluded_epics):
    #read the csv file
    try:
        df = pd.read_csv(file_path)
    except:
        print('The csv file containing historical jira tickets can\'t be found')
        sys.exit()

        # filter in only the releavant releases
    if (releases != ""):
        df['Release']=df['Release'].apply(ast.literal_eval)
        value_to_find = releases
        df = df[df['Release'].apply(lambda x: value_to_find in x)]
   
    
    filtered_df = df[df['WIP Category'].isin(wip_category_included) &
                    ~df['IssueType'].isin(issue_types) &
                    ~ df["From status"].isin(exclude_from_status) 
                    ]

    return filtered_df

Step 3 – Calculate the cycle times for each ticket

# Step 3: Calculate Cycle times for the filtered data
def sum_cycle_times(df):
    # Ensure the time spent in each state is numeric
    df = df.copy()
    df['Time in From Status (days)'] = pd.to_numeric(df['Time in From Status (days)'], errors='coerce')

    cycle_times = [] # initialize an empty list to store cycle times
    devops_cycle_times = []

    # Group by ticket_id and sum the time_in_state for contributing states
    grouped = df.groupby('Jira Key')['Time in From Status (days)'].sum().reset_index()
    
    # create an array of cycle times for all tickets
    for _, row in grouped.iterrows():
        total_cycle_time = row['Time in From Status (days)']
        cycle_times.append(total_cycle_time)  # Correct usage of append on a list    

    return(np.array(cycle_times))

Step 4 – Calculate the weekly throughput

We also need to calculate the weekly throughput, i.e., the count of tickets completed each week. Throughput is a function of all impediments (or lack thereof) the team faces in its delivery journey. It is also a function of how the team plans and sizes their work.

The logic is simple – if the team has large story sizes, throughput will be lower, and vice versa. Usually, teams have a combination of sizes, and they will show up in the histogram we created above. Large work sizes will get reflected in the long tail, providing the team with an improvement opportunity.

# Step 4: Calculate weekly throughput of completed tickets
def avg_weekly_throughput(df,weeksForRollAvg):
    # remove the WIP from completed data
    filtered_df= df[(df['WIP Category'] == 'Done')]

    # remove duplicate tickets since year week will contain the weekly 
    df_done_unique = filtered_df.drop_duplicates(subset=['Jira Key', 'Done Year', 'Done Week'])
    df_done_unique.to_csv('csv/debug-Unique_done_tickets.csv')
    # Count the number of tickets closed by week
    grouped = df_done_unique.groupby('Year Week')['Jira Key'].count().reset_index()
   
    # create a new column 'takt time". divide the count of tickets completed in a week by 5
    # ... to get tickets completed per day
    grouped['Takt Time'] = grouped['Jira Key']/5
    
    weekly_rolling_throughput_average = grouped['Jira Key'].tail(weeksForRollAvg).mean()

    return grouped, weekly_rolling_throughput_average, len(df_done_unique)

Step 5 – Run the Monte Carlo simulation

For most use cases, 10,000 simulations suffice. However, you can increase the n_simulations value to 100,000 or more for greater accuracy. Use the last 6 weeks’ throughput rolling average as one of the model inputs. This can be adjusted based on team culture, though I would caution the use of rolling averages greater than 8 weeks – it smoothens out the variation which can be detrimental to the final results.

# Step 5: run the monte carlo simulation
def takt_time_simulations(weekly_takt_time_list, remaining_tickets, weeksForRollAvg, n_simulations = 1000):
    weeks_to_complete = np.zeros(n_simulations)
    
    # use the last configured weeks for rolling average 
    # logic is to use the most recent throughput data as it's the realistic expection of the team output
    historical_tickets_completed = np.array(weekly_takt_time_list['Jira Key'])[-weeksForRollAvg:]
    takt_times = np.array(weekly_takt_time_list['Takt Time'])[-weeksForRollAvg:]

    
    for i in range(n_simulations):
    # Randomly sample from historical data for each simulation
        sampled_takt_times = np.random.choice(takt_times, size=remaining_tickets, replace=True)
        
        # Adding up all the days in the sample
        total_time_required = sampled_takt_times.sum() 

        # Estimate weeks to complete by dividing total time by average weekly capacity        
        avg_weekly_capacity = np.mean(historical_tickets_completed) * np.mean(takt_times)
        weeks_to_complete[i] = total_time_required / avg_weekly_capacity

    return weeks_to_complete

Snippet of the main program

Once the Monte Carlo simulations are executed, we need to tell a story and that is what this piece of code does

# get inner and outer bound days to complete - assume 5 working days/week

outer_bound_days_to_complete = np.percentile(weeks_to_complete,confidence) * 5
inner_bound_days_to_complete = np.percentile(weeks_to_complete,5) * 5

outer_bound_date =datetime.today()+timedelta(days=outer_bound_days_to_complete)
inner_bound_date =datetime.today()+timedelta(days=inner_bound_days_to_complete)

if release == "":
      release = "'Completed tickets in all releases'"
        
print(f"\nThe following emperical data for the Release: {release} is used to forecast release dates: \n"
      f"- {wip_category_included} are included\n"
      f"- Time spent in {exclude_from_status} queues are NOT included\n"
      f"- {issue_types} ticket types are NOT included\n"
      f"- {completed_tickets_count} tickets have been completed so far\n "
      f"-- with a median cycle time of {np.median(cycle_time_history):.2f} working days per ticket and "
      f"standard deviation of {np.std(cycle_time_history):.2f} working days \n "
      f"-- with a rolling {rollingAvgWeeks} week average completion "
      f"rate of {throughput:.2f} tickets/week \n\n"
      f"Forecast: There's a {confidence}% chance that the remaining {openTickets} (+10% additional tickets to account for unknown unknowns) tickets "
      f"is expected to be delivered between "
      f"{inner_bound_date.strftime('%d %B, %Y')} and {outer_bound_date.strftime('%d %B, %Y')}\n\n")