Category: Software Engineering

When Agility Breeds Entropy: The Hidden Cost of the AGILE Process

~ cjois

Agile has transformed software development. It replaced rigid waterfall structures with flexible sprints, rapid feedback, and constant iteration. Teams move faster, deliver sooner, and adapt to change with ease. But beneath this celebrated adaptability lies a subtle and growing problem – entropy. The very qualities that make Agile powerful can also accelerate the disorder of software code.

In an Agile environment, developers are encouraged to prioritize delivering working software over exhaustive documentation and long-term architectural planning. This is great for short-term progress – but over multiple sprints, it often leads to shortcuts, fragmented designs, and inconsistent coding patterns. Each sprint adds a new layer of functionality, sometimes built on incomplete refactors or temporary fixes. Because the focus is on “delivering value now,” deeper architectural integrity can be deferred indefinitely. Over time, the codebase starts to resemble a geological formation – with layers of old design decisions, patched logic, and duplicated functionality. The result? Increased entropy: a system that works today but resists adaptation tomorrow.

Agile also introduces entropy through team dynamics. As teams rotate, priorities shift, and user stories evolve, institutional memory weakens. The rationale for design decisions fades, documentation lags behind, and technical debt accumulates quietly sprint after sprint. Ironically, the very agility that enables rapid evolution also erodes long-term stability. The cumulative effect is a codebase that becomes progressively harder to maintain, extend, or even understand.

To counter this entropy, Agile teams must embrace sustainable agility – balancing iteration speed with architectural stewardship. This means treating refactoring and technical debt reduction as core deliverables, not optional chores. It means embedding architecture reviews, code quality metrics, and documentation updates into the sprint rhythm. Agile, at its best, is not chaos – it’s disciplined flexibility. But without mindful engineering, it can devolve into a cycle of entropy masked by velocity.

In essence, Agile doesn’t create entropy — people do, when they mistake iteration for improvisation. The key is to use Agile not just to build fast, but to build well, ensuring that with each sprint, the system grows in both functionality and structural coherence.

CP Jois

Software Engineering – too much velocity?

~ cjois

Velocity – this is something we hear about almost so frequently in the software engineering world. There is an ever-increasing demand for speeding up software development, automating code engineering or testing, and more recently, even speaking into a microphone so that code could be generated automatically.

It’s about time that as an industry we slow down a little and ask whether we need all this velocity and how much is good enough. Moderation is a good thing and it applies to all aspects of life, including software engineering. This endless pursuit of speedier software development is beginning to cause damage in multiple ways – to products and people. More often than not, it has resulted in nothing more than poor quality. While it may be fashionable to speak about digital transformation, velocity, speed and the like, it is beneficial to stop and look occasionally – such a break will most likely reveal that unfettered speed in any context is mostly unecessary and left uncontrolled leads to dissipated energy. 

In the context of software, its leading to wasted cycles of productivity. In the name of AGILE, process is thrown to the winds and quality suffers. There are very few – really a small percentage – of software outcomes that are AGILE. The rest of those efforts are simply following no process at all and riding thier luck – and because its no more fashionable to call software processes anything else, we want to call it AGILE, even if we are failing to execute waterfall projects successfully. Most teams are merely cleaning up defects from a prior sprint in the next sprint – but call themselves AGILE teams becuase that is what sells. 

Software is incredibly pervasive… and thats a powerful thing. However, if software engineers dont pace themselves appropriately, this very pervasiveness could end up being a curse. 

/CPJ

Quick Method Software Estimation and Project Planning

~ cjois 
Earlier this week, I conducted a simple and what I believe, was an engaging and effective workshop on estimation and ROM sizing - a critical aspect of proposing software projects. The audience was a group of project managers. 
 
Any such invite on the calendar asking for participation in a project management activity is viewed with some skepticism. ‘One more meeting…” is the thought that must have run through the minds of those that were invited to the session. 

Regardless, they all showed up. The end result was in my opinion a very engaging session. They all participated. We all learnt from the session. 
 
This post is to share some of the learnings from the session. 
What did we set out to do - Essentially, I wanted to communicate the power of separating “Effort' estimation, from 'Elapsed Time' computation; while further separating both of those from ‘Resourcing'.  

My sole focus was to engage the team in a hands-on activity that would convey the reality around the wide variance of software estimates even when conducted by different individuals in the same room against the same spec. It would tell us us what clients experience when they see proposals created by different members in the same group. 
 
With the exercise around "Effort" estimation completed, my intent was to use their estimate and have them perform an "Elapsed Time" computation. After those two steps were completed, we went on to put together a "Resourcing Plan" with the right skills and corresponding capacity.
 
Before we got started, we needed a simple but complete business need. We just picked a sample requirement for this purpose. 
 
Outputs from the exercise: 
There were 15 people in the workshop.  We received 15 estimates. In any such exercise, there is bound to be a spread. All that we can strive for is narrow variance. The data gathered is charted below for your reference. The data, even being a small sample, when charted followed a near-normal distribution. That is a good sign. However, the data was also skewed to the left of center, indicating the general bias to estimate lower.  
Using these estimates we then moved on to computing Elapsed Time.  
But before we did this, we needed to set some contextual parameters: 
A) we adopted the AGILE process model for this task 
B) we time-locked the Sprints at 2 calendar weeks
C) we capacity-locked the Sprints to 80 capacity hours  
D) this left 'Scope' to be the variable element

To get a little more room to compute the Elapsed Time computation, we asked each participant to take their estimates and multiply it by 10. 

For the example shown below, the PM had his effort at 410 hours.  
This computation indicates a 12 calendar week timeline. 

Now for the important thing... note that till now, we haven’t spoken about resources at all. And thats for a reason. 

The resource staffing matrix or the ’staffing plan’ as it is called, is not relevant till this point. 

The important and critical learning here is that the Effort is the effort. It has nothing to do with elapsed time or the staffing plan. There may be some influencers such as team experience levels, productivity, tech stack, architecture construct, programming environment, tools etc., but Effort remains the effort. 

Like wise the delivery model has little or nothing to do with Effort. The work remains the same. Whether we deliver via iterations or sprints, Effort does not change. If work breakdown changes, then due to granularity, effort may appear to change but it really hasn’t. 

Building out a resource model involved composing a team to have a total capacity of 80 hours per Sprint - aka - over a 2 calendar-week period (remember earlier on, that is the number we locked our Sprints to - 2 calendar weeks).
These 3 steps provide a reliable, repeatable method to arriving at Effort, Elapsed Time and the Staffing Plan.

Until next time…
CP Jois

Why AGILE fails to be agile?

~ cjois

The term ‘agile’ is much used today in many different contexts – so used that its bordering on overuse or misuse. The software industry is perhaps the one that uses it the most – so much so that it coined an entirely new process model known as AGILE hoping to jumpstart a new revolution in software engineering. While there has been a lot of talk around it, this revolution has barely provided the uptick in software project success it was meant to.

Why?

There are many reason for this. One of the reasons is that while labels have changed, basic behavioral aspects have hardly changed. Software engineers don’t do much different today, software managers hardly understand the nuances between process models/methodologies. Even more importantly, customers cant seem to change their modalities as participants in the process. Transformative change requires change at all levels, in every stakeholder role. For example, AGILE requires that customers/clients become participants in the daily activities of an AGILE effort. This calls for deep commitment and necessary time adjustments. While easy to state, this isn easy to achieve. Funding models have to change. The term ‘project’ and AGILE don’t go together. AGILE efforts are on going efforts, burning down a backlog of requirements. Projects have fixed scope, time and budget – the triad. AGILE efforts – as per the very AGILE manifesto – are meant to take on change late in the process. The typical ‘Project’ attempts to control scope and rigidly guard its execution. Looked at from a high level, these are counter to each other.

The majority of software engineers hardly understand these nuances. The typical software manager struggled to deal with traditional project execution, let alone have to deal with brand new terminology and safeguards. The very definition of a requirements changed with time. From writing 100s of pages of software requirements, the ‘Iterative’ process cycle asks for use-cases in an Actor-Ability syntax. On the other hand, AGILE demands User Stories. Speak to the average software developer and it becomes apparent that the nuances are barely even understood. The situation is analogous to the time when object oriented programming models came along but it took a long time before the industry truly wrote any real object oriented code. Even today, it isn’t unusual to see engineers writing long segments of procedural code while using advanced object oriented programming languages. Software is still a nascent industry. Success is still a matter of striving till it gets done. The ‘soft’ nature of the outcome makes it very hard to measure. Traditional practices have yet to stabilize. Complexity has been on the rise. In fact inter-operable systems and cooperating systems have become the norm.

One of the fundamental goals of AGILE was to get more features out, quicker. Getting a quick Sprint 0 done has become an obsession – most AGILE projects don’t care to build solid foundations. This concept of a MVP may be very lucrative however when observed closely, most MVP start off wanting to be something and end up being barely one-tenth of that vision. I have performed many due diligence exercises and one observation is common – most AGILE sprints are more about cleaning up the mess left behind by the previous sprint and less about feature roll outs. Hardly are there teams that truly measure that basic goal of “getting more features out, quicker”

For AGILE to truly delivery on its promise, core behavior must change. Until then, it shall remain a fancy label. Core behavior change begins by educating teams (software engineers and customer) to understand the basic tenets of AGILE – starting with the manifesto, the process itself, and the definition of an outcome in AGILE.

Until next time…

CPJ