|
Key Takeaways
|
|
In the modern development landscape, we are seeing a shift in how work is managed. The rise of AI-assisted development and highly distributed teams means that work is moving faster than ever before. However, this increased velocity often comes with a hidden tax: complexity. We are seeing more parallel work streams, more intricate dependencies, and a constant stream of shifting priorities. In this environment, simply moving fast is not enough to guarantee success. If we are moving at full speed but heading toward a deadline that we cannot mathematically hit, we are not winning; we are just accelerating toward a late delivery.
To address these challenges, we are excited to introduce the new milestone delivery confidence capability in Rally. This framework is designed to help teams move away from a reliance on "gut feel" and optimistic guessing, and start doing objective, real-time risk management. We want to move away from the traditional status meeting where everyone hopes for the best, and move toward a model in which data tells the story of our delivery health.
The problem with "status-by-sentiment"
We have all been in those meetings. A milestone (which references a significant point in time, such as a market launch or a compliance deadline) is approaching. For weeks, the project leads report that everything is "green." However, two weeks before the date, the entire effort suddenly turns "red." This phenomenon usually stems from two main issues: Optimism bias and a lack of visibility into competing work.
When we rely on manual status updates, we are often getting a report on how the team feels about their progress rather than the reality of the work remaining. This is often compounded by a lack of insight into what else is on the team's plate. Teams might be working diligently on milestone items, but if they are also being pulled into high-priority "fire drills" that are not part of that milestone, their delivery capacity is being siphoned away. Often, these conflicts aren't immediately visible on a standard tracking board. Given all this, the key question becomes “How can we establish a data-driven approach to predicting software project delivery dates?”
Data-driven risk management with the delivery confidence score
What is Rally’s new milestone delivery confidence capability and how does it work? The core of the new capability in Rally is the delivery confidence score. With this capability, we are moving the needle from subjective updates to data-driven forecasts.
The delivery confidence score is calculated using a Monte Carlo simulation—a mathematical model that runs thousands of delivery scenarios based on a team's actual historical throughput from the last 52 weeks.
Instead of asking team leads to guess if they will finish on time, Rally enables you to look at the remaining backlog and see the statistical probability of success. We categorize this into three clear risk ranges:
-
Low risk (green status): This category indicates there’s a probability of 75% or higher, meaning there’s high confidence that the team will complete all assigned work by the target date.
-
Medium risk (yellow status): With a score ranging between 50% and 74%, there’s a moderate level of confidence. Delivery success depends on velocity and scope stability.
-
High risk (red status): This category signifies low confidence, with a score of less than 50%. This means delivery is unlikely without changes to scope, capacity, or priorities.
Figure 1: An example of how the delivery confidence score is displayed for a milestone that has an at-risk target date.
By using historical performance to predict future outcomes, we remove the "hero culture" mentality and replace it with a realistic view of what is achievable. This allows us to have difficult conversations early, when we still have time to adjust our strategy.
Identifying the "invisible competition"
One of the most powerful aspects of this new view is the ability to expose competing work. Often, a team is working at peak efficiency, yet their milestone work is in danger. Rally now identifies work items that are either ranked higher or planned earlier than your milestone items but are not part of that specific milestone.
Figure 2: An example highlighting work that competes with work in the milestone, representing a drain on team capacity.
This is the "invisible competition" for a team's capacity. By revealing this, we empower leaders to make faster trade-off decisions. If a milestone is at risk, you can immediately see if the solution is to deprioritize non-milestone work or to rebalance capacity across the organization. It transforms the conversation from "Why are you behind?" to "Which of these competing priorities is actually the most important to the business?"
Pinpointing the bottlenecks
In a complex enterprise environment, a single milestone might involve twenty different teams across multiple value streams. (Value streams refer to the end-to-end sequence of activities that deliver value to a customer). As a release train engineer (RTE) or a product owner, you don't have the time to audit every single team's backlog.
Figure 3: Weakest Project Confidence highlights which team(s) are causing the most severe delivery bottlenecks.
The Milestone Delivery Confidence view automatically reveals the weakest project. By pinpointing exactly which team or workstream is jeopardizing the deadline, leaders can focus their coaching and intervention where it is needed most. We can see which team has the lowest confidence score and immediately drill down into their specific "beyond target date" work to understand the bottleneck.
Real-time action and sustainable flow
This isn't just a dashboard for observation; it is an active management workspace. The page is designed to facilitate real-time action. Within the view, we can take immediate steps to improve our delivery outlook:
-
Re-plan beyond target date work: We can identify items scheduled to finish after the deadline and pull them back into an earlier iteration (a time-boxed period of development, often called a sprint).
-
Fix unplanned work: We can identify milestone items that are missing release or iteration values. Assigning these correctly provides immediate clarity into the simulation and often improves the confidence score instantly.
-
Reprioritize in place: We can use a drag-and-drop interface to move milestone work into the "green gone" based on priority, ensuring that the most critical value is delivered first.
Our ultimate goal with this new capability is to foster a culture of transparency and sustainable delivery. By using a shared, data-driven source of truth, we can align teams and stakeholders on what is actually feasible. By catching these "late surprises" months or weeks in advance, we can move away from the stress of end-of-cycle heroics and toward a more predictable, healthy flow of value.
We are currently in the early adoption phase for milestone delivery confidence. We encourage you to reach out to your account team to see how these tools can help your teams build the future with true, data-driven confidence. To learn more, see our YouTube video, “Milestone Delivery Confidence in Rally.”
Frequently asked questions
What is the delivery confidence score and how is it calculated?
The delivery confidence score is a data-driven forecast based on a Monte Carlo simulation. This mathematical model runs thousands of delivery scenarios, drawing on a team's actual historical throughput from the last 52 weeks to determine the statistical probability of success.
How does Rally define the three risk ranges for delivery confidence?
The software categorizes delivery health into three distinct ranges based on probability: Low risk (greater than or equal to 75% confidence of on-time completion), medium risk (50-74% confidence), and high risk (less than 50% confidence).
What is "competing work" and how does it affect a milestone?
Competing work consists of items that are ranked higher or planned earlier than your milestone items but are not actually part of that specific milestone. This creates an "invisible competition" for a team's capacity, siphoning away their focus, even when they appear to be working at peak efficiency.
What real-time actions can leaders take to fix a failing timeline?
The solution features an active management workspace. Through this workspace, leaders can perform a range of tasks, including the following:
-
Immediately pull work scheduled beyond the target date into earlier iterations.
-
Fix unplanned work by correctly assigning missing release or iteration values.
- Use a drag-and-drop interface to reprioritize milestone work.
Eric Nash
Eric Nash is an Agile expert and Rally resident advisor who is happiest when he can coach and share his passion with others, whether it’s working on digital and Agile transformations, junior golf or fishing. When not on the water he works with organizations in industries from manufacturing to publishing to defense...
Other resources you might be interested in
Speed with Confidence: Managing Delivery Risk in an AI-driven Development World
Stop relying on hunches and guesswork to track and manage delivery progress. Rally's milestone delivery confidence capability offers a data-driven approach.
Rally Office Hours: May 28, 2026
Discover the latest Rally product updates and event news, including tips for migrating from legacy custom pages to custom views.
Rally Office Hours: May 21, 2026
Explore the latest Rally updates, including support for OAuth and portfolio-level Kanban boards. Get expert tips for reserving capacity for unplanned work.
Rally Office Hours: May 14, 2026
This video from May 14, 2026 details Rally product updates, including new dependency settings. It also offers a Q&A on custom views and details events.
Rally Office Hours: May 7, 2026
This video details various Rally updates, including a tutorial on using AI for custom widget creation and news on upcoming community networking events.
Rally Office Hours: April 30, 2026
This Rally office hours video covers a new milestone delivery confidence framework, user Q&A on features like ranking, and upcoming event news.
Rally Office Hours: April 23, 2026
Learn about new tools Rally provides for moving legacy pages to custom views. Listen to technical Q&A and find out about upcoming events.
ValueOps ConnectALL: Synchronize Jira and Rally for Frictionless Cost Accounting
This course teaches you how to integrate Jira data into Rally for the purpose of frictionless cost accounting in Clarity.
Rally Office Hours: April 16, 2026
Join Rally Office Hours to get expert tips and the latest product news. Explore new AI controls, Monte Carlo simulation for milestones, and more.