Category: Technology Operations

When Agility Breeds Entropy: The Hidden Cost of the AGILE Process

~ cjois

Agile has transformed software development. It replaced rigid waterfall structures with flexible sprints, rapid feedback, and constant iteration. Teams move faster, deliver sooner, and adapt to change with ease. But beneath this celebrated adaptability lies a subtle and growing problem – entropy. The very qualities that make Agile powerful can also accelerate the disorder of software code.

In an Agile environment, developers are encouraged to prioritize delivering working software over exhaustive documentation and long-term architectural planning. This is great for short-term progress – but over multiple sprints, it often leads to shortcuts, fragmented designs, and inconsistent coding patterns. Each sprint adds a new layer of functionality, sometimes built on incomplete refactors or temporary fixes. Because the focus is on “delivering value now,” deeper architectural integrity can be deferred indefinitely. Over time, the codebase starts to resemble a geological formation – with layers of old design decisions, patched logic, and duplicated functionality. The result? Increased entropy: a system that works today but resists adaptation tomorrow.

Agile also introduces entropy through team dynamics. As teams rotate, priorities shift, and user stories evolve, institutional memory weakens. The rationale for design decisions fades, documentation lags behind, and technical debt accumulates quietly sprint after sprint. Ironically, the very agility that enables rapid evolution also erodes long-term stability. The cumulative effect is a codebase that becomes progressively harder to maintain, extend, or even understand.

To counter this entropy, Agile teams must embrace sustainable agility – balancing iteration speed with architectural stewardship. This means treating refactoring and technical debt reduction as core deliverables, not optional chores. It means embedding architecture reviews, code quality metrics, and documentation updates into the sprint rhythm. Agile, at its best, is not chaos – it’s disciplined flexibility. But without mindful engineering, it can devolve into a cycle of entropy masked by velocity.

In essence, Agile doesn’t create entropy — people do, when they mistake iteration for improvisation. The key is to use Agile not just to build fast, but to build well, ensuring that with each sprint, the system grows in both functionality and structural coherence.

CP Jois

Technology Portfolio Choices

~ cjois

Technology portfolios grow out of control rapidly. More so in today’s environment than before. Additions to and deletions from portfolios are not casual or trivial activities. Unfortunately, time pressures on technology projects turn them into casual decisions.

Recently I was helping a team decide between a newly acquired technology and the use of native legacy technology. All facts indicated that the new acquisition, while more sophisticated technology, had a track record of delaying projects. There was clear evidence that the team was having a hard time finding people with skills in the new tech. Despite that the team had persisted for 8 months, without making any progress.

Indeed, its hard to disband a new foundational technology. However, as technologists, we must always keep the facts in view and allow data to speak the story. All facts indicated that we were being slowed down the technology rather gaining any momentum. The right decision was to disband the choice and move forward. Thats what we finally concluded.

Always remember, let the facts guide the decision. Its never too late to remedy a situation.

CP

Operations and Innovation – Shifting the Needle

~ cjois

Shifting the needle is essential to generating growth. Left to itself any process either sustains or degrades. Effort is needed to overcome the inertia. Such is the case with Support processes and costs. ‘Keeping the Lights On’ is pretty much a bulk of any operating budget. Left as it, if we are lucky, KLO costs remain where they are… most likely they get worse. We are speaking $$ but the same applies to capacity (hours). As entropy occurs, at one point, all available capacity will be consumed by KLO actions. In fact, KLO will demand even more capacity than we typically have.

So where does this leave Innovation?

It leaves it bereft of any space on the radar. This is the reason why so many companies talk about innovation, but nothing ever happens. The reality is that there is no $$ or capacity left for innovation to even be spoken about, let alone acted upon.

What does that mean?

It means that for any company finding themselves in a rut, in status quo, on the verge of obsolescence, the first step then is to shift the needle as to where their $$ are going. The first step is to begin optimizing the operation in every which way possible in order to gain a different distribution of KLO and Innovation $$. Most companies find themselves with a 100:0 – KLO:Innovation – ratio. The goal is to shift it the other way…. 90:10; 70;30 to 50:50, at least. This then gives us a good shot at ideating, incubating, iterating and industrializing new ideas and bringing them to market.

CP Jois

Machine Learning & IT Operations

~ cjois

Machine Learning is a much spoken topic nowadays – perhaps more spoken about than available capabilities allow for. There is nothing new about machine learning, neural networks, or artificial intelligence. These concepts have existed for a while with early discoveries in the 1700s. Even recently, there was a huge push in research in AI/ML in 70s and 80s.

The last 5-7 years have witnessed an upsurge in activity in this realm. The reason for this increase in discussion and action is primarily the rapid decrease in compute costs, increase in compute efficiencies and tool that allow for simple, relatively non-programmatic approaches to get these insights.

Insights have driven operations for a long time now. At one point, such action was purely reactive. Then it slowly moved towards being pre-emptive. And now it trending towards being predictive. Not every situation requires or will benefit from predictive insights. While insights must be actionable, not all need predictive traits. And even when predictive, how far predictive is another question all together. While technology groups are being asked to play a role in the move to predictive insights, the percentage of technology organizations that apply such philosophies to their own operations is not all that high. There are few who believe that an organization needs to be of a certain size for it to use prediction capabilities. Others believe that its unaffordable. The reality is that organizations of any size, operations of any scale will benefit for being able to use predictive capabilities.

Technology operations can leverage machine learning techniques generate predictive insights in a variety areas. From predicting where the next alert on their server farm will come to predicting the code from where the next software fault will arise to predicting the set of test cases to use in regression test cycle, technology operations can leverage prediction capabilities. The pre-requisite is that heuristics exist, can be funneled to a data container where insights can be distilled from. Of course, an important element is that the organization carries data science skills in its human resource base. The lack today is more around the skills area than any other. The lack is also around leaders and managers having the vision to stretch themselves and their employees to think differently than in the past.

For those of us to remember, it took years for our software developers (formerly generally called programmers) to move from procedural code to object oriented thinking. For decades, we would find procedural code being written inside of object methods. I wont say that this has gone away completely. I still see literally 100s of lines of code written inside of methods which by the very nature of object oriented thinking were meant to be short and sweet, focused and dedicated. Likewise, our leaders and managers mostly still manage operations like we did 20 years ago, despite the availability of tools that can help us do it differently. We still run 100s of test cases just because they are in the suite, even though we know that the likelihood of many of them in finding a defect is low to none. We still suffer what is known as ‘alert fatigue’ – which then leads to a real alert actually being missed. We still wait for the problem to occur before we can even begin to prepare for it.

Machine learning is a compelling tool. If leveraged in simple and effective ways, it can lead to cost optimization, effort optimization, focus optimization and in the end make for happier technology employees… and of course happier customers.

CP Jois