There is a time to be an Agile Purist

agile purist
There is always another mountain to climb, and there is always something to improve on – by an agile purist 🙂

As someone who is conscious of other people’s needs, I don’t see myself as a dogmatic agile purist. Instead, I pride myself of the ability to handpick practices and tools for clients based on their maturity and environment. So you can imaging the surprise I had when one of a close client team member called me an agile purist in a half joking way. Knowing that any joke is based on some kind of truth and being the curious yours truly, I convinced her to tell me more about the reason. It turns out she and some of the other team members think I have been advocating things that is “out of their control”. Things like signing up for their own tasks and stories are difficult when there are more than one manager on the team that doesn’t want to let go of control totally. Stopping the habit of over-committing work into the sprint were also met with resistance as the project manager has a schedule based on WBS to follow and development manager would commit on behalf of the team all in the interest of keeping things on schedule. So, how can I, as a coach, push them to do these things when they are not allowed to do so? These are all real issues and I felt terrible for putting the team in that position. Then the thinking started – why are we in such a situation and what can I do about it. That’s when I realized that I have done something about it already. In fact, I have had private conversations with various managers regarding the topic of self-organization. There are just so many hesitation because the team is relatively new and not familiar with the product. So the reality is this change is going to take time. Now should I stop advocating it to the team members in the meantime? The conclusion is NO. Because we need to get out this victim mentality to make any sustainable change. Should we always wait for the organization to change first? Should we wait for a memo from the CEO’s office before acting? If you ask yourself these questions, the answer will become fairly evident. Each of us controls a part of our destiny. Like Sir Sultan Muhammad Shah Aga Khan III said “Struggle is the meaning of life; defeat or victory is in the hands of God. But struggle itself is man’s duty and should be his joy.” Each of us, from top to bottom in the organizational structure has the right and responsibility to voice your opinion. Each of us is part of the problem domain just as much as your colleague and manager. So, keep voicing your opinions and when a lot of us does that, something amazing will happen.

Now, this is not to say our leaders gets off free. Change is most effective when it is supported from the top. Leaders bear additional responsibility because you are not just responsible for yourself but also others. A successful agile transformation requires push from both top and bottom.

So, I will continue to be an agile purist in mindset and a pragmatic and practical agile practitioner. What about you?

How to start your own agile transformation in a large enterprise

In my last blog, I talked about the emphasis of large enterprise world toward lean agile software development methodologies. In addition, people are favouring “owning” their agile transformation rather than relying on external consultants to hand them one!

What does that mean to all those people out there still looking for help and answers on their quest for agile transformation, lost in the vast sea of enterprise IT? First and foremost, define your needs/challenges. What is the problem we try to solve? Defining the right problem has several advantages, one of which is easier to get management buy-in. Then get enough management support! This is another one of these crucial success factors. While it is possible in small organization for grass root agile movement to take hold, it is almost impossible to do so without management buy-in within a mid-large organization. You don’t necessarily to ask for “Agile” transformation, but ask for a chance to explore possible solutions to your needs/challenges in the first step. The next step is to find that guiding light. As with many other things, Google can be your friend or foe. You may decide to do the research yourself and start the transformation from the things you learned. Internet is full of resources on this, but information overload and misleading opinions also makes it very dangerous to place your bet on any literature you read online. As a sports enthusiast, I can assure you that the difference between learning any sport with or without coach is night and day. The top problem is that it is hard to unlearn bad habits. The time and money wasted on learning and then unlearn the bad habits is way more costly than paying for a good coach even before considering the damage done by the bad habits. So, getting someone to help you start the groundwork getting everything moving is the next critical step. Now, be aware of anyone or organization that promised the silver bullet solution. That doesn’t exist. Just like there is no magic herb that can treat all diseases, there is no one agile methodology or technique that can fit your organization right out of the box. If you have bought any COTS solution that supposed to work “out of box” for enterprise, they never do. Instead, look for someone with deep knowledge, patience, listening skills that can ask the right questions! As Peter Druker, father of management consulting, had proven time and time again, you hold the key to the true answer, consultants are just the torch to help you find the right door to open.

Hopefully, once the right help is found, you would have a great trip down the agile lane. But what happens when the Coach leaves, as they suppose to? I will follow up with my next blog.

Enterprise Agility – Focus of 2014

There have been a number of interesting observations in this year’s Toronto Agile & Software 2014 conference.

Enterprise Agility is the hottest topic of the conference. The conference itself took an interesting approach this year by using crowd voting as a way to do session selection. A large number of votes went to sessions representing large scale enterprise topics. Six out of the final twenty-one sessions focused on this trendy topic – how to do agile in large enterprise effectively?

In addition, aside from the usual big guns like Scott Ambler on DAD, more presenters appears to be from internal employees rather than outside consultants. This is another interesting trend, and IMHO, a great trend showing the affect of agile movement is truly taking foothold in larger enterprises. It is no longer expert consultants and coaches trying to “sell” the ideas to the enterprise, but company staff actively seeking change which is always more powerful and sustainable.

Furthermore, one resounding conclusion drawn by all presenters on this topic is that agile transformation cannot be “bought” from an outside consultancy, but must be “owned” by the company. Even the consultants, e.g., Deloitte, supported this idea! This does not mean don’t hire expert consultants. On the contrary, all of them had hired help to give them a kick start, a guiding light, a set of useful tools. That is the value of consultants. However, one should not expect consultants to come in and hand you answers to all your woes, a “recipe” for successful agile transformation or even a certified large scale agile framework to be implemented by the letter. There is no such thing! Success is born from experimentation, learning from past experience, from disciplined execution by the owners of the process – YOU.

Having a consultant or coach is very valuable, because it will save you from some well known traps and start you off on a path with the highest chance of success. Once you started down that path, strategy must be formed, decisions must be made, and tools must be selected that is suitable to an organization’s unique culture and environment. I often say agile is a mindset, a philosophy, an art based on science. We have to embrace the core value of agile of inspect and adapt like so many great management gurus before us have discovered already. Only then, will we have a real chance of enjoying the fruit of success. I will sort out my other thoughts in the next few days and discuss the following topics in follow up posts.

  1. What does that mean to all those people out there still looking for help and answer on their quest to agile transformation, lost in the vast sea of enterprise IT?
  2. How to find the right coach/consultant?
  3. What to do after the consultants leave and you have taken on the ownership?

Is #scaled-agile frameworks the next step or a mis-step?

Agile software development has become the mainstream for a few years, now. With the help of its successes in smaller and younger companies, larger and more traditional companies like banks are starting to experiment with them as well. Lately, scaled agile seems to be a hot topic. More structured frameworks like SAFe and DAD are becoming more popular by the day. Yet, there is strong resistance, and sometimes almost resentment, towards them from some agile practitioners. Everyone agrees agile needs to work at scale, the disagreement is whether that needs to be done under a seemingly more waterfall-like framework.

Interestingly, both frameworks mentioned above are started with people used to work in a large enterprise environment. Should we trust frameworks that came out of such traditionally non-agile settings? We all know that one of the major reputation issues with agile is people practicing wagile (waterfall agile) or scrumbut. What makes these self made frameworks better than the rest that people look down upon? Well, it turns out, a lot of the original Agile Manifesto signers worked in large enterprises. That proves good things can come out of these environments. In addition, having consulted in one of these enterprises, I can say that people who work in large enterprises are just as talented, creative, and sometimes free spirited as the rest. However, they also constrained by a lot more external factors like regulations, industry rules and standards, social pressure.

An experience agile coach Joe Little had an interesting correspondence with James O Coplien through one of his blog post on this very subject.  While James is obviously very knowledgeable in agile and especially Scrum, I agree with Joe’s point that explaining, much less convincing, most people about expanding the agile practices in a self-similar pattern is a quick way to send people to the clouds. It will be even worse talking to business executives like that who already have a preconceived notion that structure and rigid processes provide security and quality.

The biggest advantage I see in these frameworks is that it provides a more gradual transition path from waterfall to agile development. By implementing a more structures top layer (at portfolio or product line or business line level), they provides predictability and discipline required of public companies from the investor. That also gives more freedom to teams and business units to make decisions on lower level details.

It’s a balancing act. Just like one of my favorite quotes in agile, “take one bite at a time”. Doesn’t it make sense to approach agile at a large scale in the same way for some environments?

In a follow up discussion on this post, a number of very interesting points were posted here. Please continue reading the comments. The conclusion/ take away took a slight turn. Instead of evaluating the scaled agile frameworks, it is the detailed things we do and the ways we think that makes a difference. The frameworks, like a lot of other large frameworks, can be used as sounding board to reflect on our ways.

Enterprise Agility – How far would you go?

One of the sessions in the Agile Coach Camp Canada 2014, in open space format, was on Scaled Agile Framework (SAFe). The session attracted many participants who works in large enterprises. The interesting thing is that rather than talking about the specific implementation and experiences people had with SAFe, the discussion went directly to whether SAFe is agile enough to be called an agile framework. The specific arguments are outside the scope of this blog. The main argument (my interpretation) against it though is that SAFe seems to put an iterative, but somewhat sequential and hierarchical structure around accepted agile methodologies like Scrum and Kanban. The main proponent (my interpretation again) for this framework feels that the current corporate structure and external environmental factors like regulation is a real hard constraint just like any other technical constraint that agilists have to work with. SAFe is a very attractive way of building enterprise agility with what we have. This is very interesting to me as I am currently consulting for a big bank in Canada. Does this provide a structured way to finally bring large enterprises across the chasm over to mainstream agile community?

Incidentally, I got involved in a passionate online debate on LinkedIn group Lean and Agile Software Development – “QA organization with agile teams”. The discussion started with a question on whether you need a QA Director role in an agile software development organization of decent size. The replies converged nicely into a qualified “no”. While the managerial oversight is an overkill, the need of a voice in support of quality and quality practices is there. The need to nurture and care for QA professionals in the team also exists. Whether these responsibilities become job of one or more and whether they have a title of Director are implementation details. However, the discussion spiraled into whether a flat network system is better than hierarchical structure. While no one favours a deep hierarchy, people with senior management responsibilities, myself included, think there is a scalability issue with network systems within enterprises or public companies.   Sooner or later, hierarchy will emerge to manage external interaction and inter-dependencies between networks.

This brings me back to the original discussion on SAFe. If the premise of flat network systems can become a reality than SAFe may not be the most agile framework that can be applied. However, is it possible for an enterprise to undergo such a drastic transformation even if you have years or decades? How far would you go to move your or your large enterprise client’s organization towards that direction?

Alphali Inc

#TDD-is-alive … and well …

I attended the Agile Canada Camp Canada (#ACCCA14) 2014 over the last weekend. It was really invigorating and lively. I was able to pick up several new concept and ideas and helped a few people along the way. I will post some important thoughts in a series of blogs on these topic. This is my first one as it is closely related to my previous post on the outrageous claim by DHH on TDD.

I was actually going to talk about this topic in the Open Space session, but two people already grabbed that spot, so I eagerly joined the conversation. While opinions from both sides were presented, consensus was that TDD is a good coding practice that should not be buried by one hash tag. We all understand where Hansson was coming from and some people even thought Hansson was only referring to rails world and not other languages. Nevertheless, the more important message we want to send though is that TDD is a really useful tool for people starting out with the unit testing and programming in general. It is like a training wheels on a bike. The technique will encourage developers to think about how to design the application in a testable manner which turns out that encourages loose coupling. Over doing it will lead to falls, just like a fast turn on the bike with training wheel will do, but once you have the habit of thinking about coding in this way, then the wheels can come off.

So here is my contribution to this cause. TDD is alive and well, in fact, it is maturing. Like any good idea, challenges to established “truth” are healthy, but present the full picture, not a sensational hash tag (the power and downside of Twitter), so that we don’t mislead people.

power of moderatioin

Does #TDD really need to be dead?

Test Driven Development (TDD) has always been a hard sell in many companies I have worked with. The resistance would come from all possible directions. Developers worry about extra overhead, managers worry about delayed project schedule, and finance worry about increased cost. However, since David Heinemeier Hansson posted his controvertial post “TDD is dead. Long live testing” more than a month ago, an already heated topic just got a lot hotter. For example, it has sparked agile heavyweights Kent Beck, Martin Fowler and David to have a series of hangout sessions to discuss and understand each other’s view. These are great discussions with lots of history and knowledge embedded. I highly encourage everyone to at least read up on the summaries of each discussion whether you agree TDD is dead or not!

Throughout these conversations, I detected a common theme though.  Whether the discussion is around test isolation or design damage; over testing or unit test runtime limit, the objections seem to be derived from observations or experiences of strict, or better dogmatic, execution of a believe. This reminds me how my son would insist something as universal truth because he has seen it once. It is very hard to convince someone otherwise of what they have seen or done. As someone specilized in software development methodologies and best practices, I have learned and still often remind myself the principles of Tai Chi. Water, one of the softest things on earth, will always erode away any type of rock in their way. Anything used in extreme can be harmful, be it a technique, methodology, medicine, or ideology.

This is also why I not only preach agile development, but also try to live an agile life. Life is a journey, we, as driver of our own lives, have to steer us out of trouble in order to reach the next stop. So, can  we declare TDD as dead but still be an advocate for proper testing? Yes. Do we need to declare it as dead in order to make that point? I don’t think so. What’s your thought?

What does the origin of DevOps teach us?

Recently, I read an interesting story on how the name DevOps came about.

True enough, as an “afanaciado” of DevOps movement, I didn’t know this story. It even had a reference to the city I work in – Toronto. It is home to one of the two people that caused this name to appear! I am feeling proud, now:-) As I read the story, I was intrigued with Patrick Debrois and Andrew Shafer’s tenacity to pursue what they believe is important for two years in the face of seemingly indifference from the fellow community members. In addition, Patrick’s courage to organize the first #DevOps conference in 2009 on a brand new topic further proved that if you believe in something, follow it up with action, then you will be rewarded. That reward may not be success as in this case, but the experience itself will be reward enough.

That is one of the fundamental pillars of agile. Don’t wait for everything to be ready. Don’t expect all the documentations will be complete. Don’t even think you can spec out all the possible edge cases before building the software. Do execute on what you already know. Get feedback quickly and then build on top of that. You will be rewarded with information/knowledge that otherwise will just be a design/estimate/assumption/risk. As an added bonus, you might just get a working product that’s right for the customer. Isn’t that nice 🙂

Is agile reinventing-the-wheel?

I was training a fairly successful multi-national software company on agile estimation and planning several days ago and a question popped up at the start asking why is agile better than methodologies used in the manufacturing industry? Why is it better than mil-std-498, a standard that promoted iterative development and focuses on quality? Isn’t that what agile claims to do? In essence, why are we reinventing the wheel instead of taking what’s already establish for centuries in the manufacturing world through countless trial & error and proven to be effective? Those are really good questions and ones that I, myself, have asked before, and a lot of people are still asking.

Software is a relatively young industry compared to manufacturing. So, it is logical for software professionals to learn from the experiences and processes in manufacturing. In fact, many software PMLC and SDLC processes were adopted from manufacturing. When it came to agile, it is no different. Methodologies such as Kanban have deep ties to manufacturing world. However, there are many different processes in manufacturing and not all of them apply to software development. Some manufacturing processes focus on streamline the assembly and improve repeatability of producing the same widget over and over again. That doesn’t map well to software development as every application is different from the one before. Software development is an innovation and research process, so software development  is more similar to scientific research or design in the manufacturing world. In addition, agile is more than developing software iteratively. Iterative Waterfall is also iterative development, and the one encouraged by standards like Mil-Std-498, that addresses one important aspect of agile which is doing things in smaller pieces. But it still emphasizes on documentation over working software; contract over collaboration; fights against instead of embraces change; and most importantly, processes and tools over people. So, YES, we should learn from other industries like manufacturing so not to reinvent-the-wheel AND adopt them to software development’s unique characteristics to make the wheel better for our needs.

An Estimate is an Estimate … or is it?

Since the dawn of software projects, we software professional have been struggling with estimates. They are sometimes too large, mostly too small, and rarely bang on. We have tried to employee myriads of processes, controls, templates to it, but it doesn’t seem to get any better. When we say “an estimate is an estimate”, everyone seems to agree, but what gets acted on can be very different. I have observed this same phenomenon in a few different companies.

Development teams are asked to produce an estimate, sometime detailed, once all estimates are entered into a project schedule, it is now a “commitment”. It is frozen and any changes to it must go to a stringent change request process along with many stern stares, among other things. Does that sound familiar to you? Well, it is more common than I thought possible. Some of those companies are also agile shops. How can that be, you ask? The answer is, the management, and by extension the business, needs to plan out expenditure, resource allocation, etc. These all take time. So, companies always budget into the future. Shareholders and the board will make sure the executives are running the company in fiscally responsible manner. That means, when estimate percolates up to certain level, it will be seen as a promise. That is the reality of running a business.

Instead of getting into an argument over how to change the financial models of the world, being an agilist, I think about how we can adapt to it. Software organizations and teams must change their frame of mind when we are asked to provide detailed estimates, or in Scrum terms – set sprint plan, that we ARE making a commitment, rightly or wrongly. What’s important, POs and SMs take notice, is to control the scope and distraction so that the team have the best chance of adhering to it. That is also why, as a team, we should keep an eye on the sprint progress during every standup and an eye on the release/project progress during every retrospective!

The first benefit a decent agile implementation provides to any organization is visibility. The first hill to conquer though, IMHO,  should be predictability.

Patrick Li