Call yourself Agile, will you?

Recently I was part of a project team that was using elements from the SCRUM methodology and therefor called itself Agile. Now not to long before that, I read quite a bit about Agile and SCRUM myself, the pitfalls and how to avoid them so I think it is safe to say that what this team was doing, could not formally be considered Agile or SCRUM. Still, it worked and as I remained silent longer I could see how picking certain elements from SCRUM worked well for this team but also why the things left out weren’t hurting all that much.

The things taken from SCRUM were mostly about lingo, we worked in a sprint, we had standups, a backlog, deployments were demo’d to the Product Owner, the last day of a cycle were spent on the demo, documentation and what to do during the next cycle. Nothing here is a big surprise and it worked well.

The things left out were more stunning. In the two months I was there, I never seen the product owner, she was called once every week. A sprint was formally planned to take 3 months but within the sprint we had small one-week iterations where we would decide what to take on for that week from the huge “formal” sprint backlog. Nothing could be added to this three month change bin without taking something out, so please plan your CR’s!

Now why did it work rather well? First the notion we were working agile appealed to many parties within the organization. There had been complaints for the past year that the IT department was taking to long with changes and was way to rigid to ever get anything done. Something had to be done! Agile was introduced and there was much rejoicing.

Second was the involvement of the Product Owner. Hearing her every week and allowing her to give input on next week’s planning was more than the business ever had, even if a sprint was formally three months, once something was in, there was a lot possible in having IT work on this first and then this.

Third factor of success for SCRUM was the insight into the current CR’s under development; we would have clear on what we were working during a week and over the month and would communicate this to the CR owner. So instead of being a black box, the team was a glass box.

Most of these things make sense and it wouldn’t need to be called SCRUM to work, but in the time I have been working in SCRUM teams I learned that it is often not the methods itself doing the work but the perception of these methods by those implementing it and those reaping the benefits of a revitalized team.

Agile development up close

My most recent assignment is as a Test Specialist in an Agile environment, prompting me to delve into the works about “Testing in an Agile Environment” and I must say that a lot has been said and written about this. A few of the most interesting findings will be listed here.

“Agile development will make a QA department obsolete.”
This is an often repeated mantra and my opinion differs somewhat. In the linked article, it is argued that Unit testing will solve most QA problems and that with proper unit testing in place, there is no need for a complete QA cycle before offering the product to the business for UAT.
First it is good to realize the article was written by Eli Lopian, the CEO of Typemock, the Unit Testing Company. He is not unbiased as are not many of the crew that wants to kick out all testers so watch your sources.
Second, the SCRUM guide says that everybody in a SCRUM team is a developer, no matter their specialism. Following that, someone who was formally a tester will be called a developer with testing as a specialty. By moving the testers from a centralized fortress with all kinds of stringent and delaying requirements into the development team, you are cutting down the cost of a QA department. The cost of developers will go up, as the QA work itself is shifted to the teams.

“Testing should be automated.”
For some reason, when searching for test automation the most results are not “How to automate” but “What should I automate?” When considering normal testing, one or two runs and the product can go live, manual testing is vastly preferred as manual testers are cheaper faster and easier found. However, as soon as you start talking about regression tests, automation starts to make sense. It becomes possible to test build that have no external changes to be retested really fast and often. Not many sane things are written about this, I could only find a few.

“Developers do the real work, testers create overhead.”
This came from many developers in many projects and are met with “if you made no mistakes, then I would have nothing to test.” In SCRUM this is worked around by calling everyone a developer, in practice it is all about trust and value. Today I was talking with a software architect about this and he simply said that everyone makes mistakes, testers and developers alike. It is the tester’s job to point out mistakes that makes him impopular.

“Waterfall is dead, Agile is the bomb!”
As with any “X is dead, Y is the new black”, agile development will shine and keep on shining for a while. It is a lean method giving fast results and acknowledging that the results are often missing target a bit or quite a bit, but not before changing directing is really expensive. But it is also a trend, a fad if you will. Many teams adopt Agile without knowing what they get themselves into and falling into the “partial implementation trap” where the interesting parts are used like short cycles, high level requirements and product demo’s but the hard parts are left out like having an involved product owner, daily stand-ups and self-managing teams. Working without somebody being the boss is the hardest thing ever.

Next time I will talk a bit more about test automation, especially on how to make it easier to implement and to keep up to date for future releases.
After that, business will be back as previous.

Surprises, a fact of life.

Some really great comments came in last weeks about User Acceptance Testing, Robin Goldsmith wrote an extensive article about it which can be found here.

The article you are reading now will talk about surprises, where they come from and how to deal with them. Often when testing the request comes up that a tester should “remove all surprises” when going live. The meaning of course is that vendor and client find it acceptable to go to production with some defects as long as the nature and impact of these defects is known, even better with a work around in place. So what is a surprise for business and how can we find them all?

A surprise is (among other meanings) defined as “To encounter suddenly or unexpectedly; take or catch unawares.” This can be translated to software as some operation that is not expected, out of the ordinary and possibly suddenly. A surprise is not even necessary a bug or deviation from specification, it can be a function that makes the user’s life easier or performs in a way that was previously unthought-of. However, the good surprises are caught as well in a proper testing process as they deviate enough to be unexpected.

As a surprise is unexpected in its appearance, it is good to look back at this quote from Donald Rumsfeld, secretary of defense at the time and talking about the possible presence of WoMD in Iraq:

There are known knowns; there are things we know that we know.
There are known unknowns; that is to say there are things that, we now know we don’t know.
But there are also unknown unknowns – there are things we do not know we don’t know.

 

Let’s look at software testing with this quote in hand:

• The known knowns, business users and project sponsors are normally accepting these as it can worked around when moving to production.
• The known unknows can be seen as an acceptable risk if the area that is not thoroughly tested is known, for example with using test prioritization based on risks.
• The unknown unknowns should be covered in your report as well: “We did not test area X and cannot report on possible defects or surprises that will occur there.” In the best world there will be no need to report this as there was enough time to do even a small coverage or the untested functions are not moved to production at all.

Summarizing, a surprise is an unexpected and often undesired deviation from expected operations. To minimize surprises, it is important not only to cover as much ground as possible but accept that there are things still unknown and to communicate clear on these unknown. Be honest in this, your business will thank you for it. If not now, then later.

User Acceptance Testing

User Acceptance Testing (UAT) is a big part of most software realization and implementation projects. The goal of this test can be diverse and ranging from the wide “finding everything that is wrong with this software” to the narrow “have a look and see it is fine”. These extremes are both ineffective in reaching the true goal of the UAT, which is to have the business end users test and ideally accept the implementation as it is planned to move to production.

In theory, the core purpose of UAT should be to validate the business processes against the software implementation using business data, business processes and all roles and responsibilities set as intended. Often when I am hired to organize the UAT for a software implementation project the purpose turns out to be either “get them to accept it” when dealing with IT centered  management or “we need to find everything that is wrong with the software” when the vendor is distrusted. While both have their merits, they have pitfalls that can impact project results when not addressed early in the planning stage.

Before doing anything else, I sit down with my employer and talk about the added value and limitations of User Acceptance Testing. The added value is clear: the end users have a hands-on first experience with the implementation and can signal where changes need to be made for a smooth transition. But the limitations are just as important and very easily overlooked. Here I will list a few and discuss how to mitigate the risks.

Users are not testers, users are users.
The UAT is done with the actual users of the product, they are taken from their day to day work and are now looking at unfamiliar screens, colors and display fields. Some discomfort is to be expected and when this is the first time the business has a look, your users can be quite skeptic. Some may have their own agenda, some will be clueless why they are there, some are happy to be away from the daily grind or worried what will be waiting for them when they come back and others are motivated to enter all exceptions they can think of to prove the system wrong. The main thing to remember is that your end-users will most likely not look at the designs and stay within the designed path but are more prone to divert from their tracks, move back and forth and enter very unlikely data. Nothing of this is planned much and most is ad hoc so reproduction of defects will be a nightmare so you are to make sure there is a robust system in place that is of high maturity and almost ready for go-live.

To many defects: out with confidence.
As seen above, a low product maturity will cause many defects when entering the UAT. Test early and often, absolutely, but use test professionals for this. They have a thick skin and are used to working with unfinished products. When asking your day to day business people to test a product that is not ready for UAT, they will wonder why they are interrupting their daily activities for something that is not ready. But worse, the word will spread your product is horrible and will ruin all efficiency that people worked so hard to implement. For a smooth transition and business changes, people need to be willing to work with the new product. Starting UAT to early will undermine the willingness to cooperate.

A big overhaul will cause massive delays
The biggest limitation on UAT is that no big changes are possible anymore if the deadline is to be made. Small display changes, report fine tuning and role and responsibilities can be adjusted easily but bigger changes to internal logic, backend communication or data processing cannot be made without going through a full design, build and test cycle. Planning your UAT four weeks before go-live, which is not uncommon, prevents any full cycles to be completed especially if big data conversions or interfaces are involved.

Testing on production data can be expensive or impossible
While the test professionals can create their own data based on production samples, the users will want to see the actual data as it will be after go-live. Every project I have been involved in and where IT was not able to deliver production data in the UAT environment ran into defects that were complaining about the testdata. It is understandable to the project manager and the realization team that the test environment is not suited for the huge dataset that will be used in production but always try to include at least a subset of data and communicate clearly to your users what this subset is. Sometimes laws and regulations or the fact that there is no production data at all prevent the project from using actual data. Then strive to have the data completed as much as possible, nicely rounded off cases and as real as possible. Rather use your company’s phone registry to have a bunch of names than use “testcustomer1” and “testcustomer2”, this will engage your users more.

When conducting the User Acceptance Test it all comes down to managing expectations. Manage the project’s expectation with regard to what the UAT will deliver, manage the users’ expectation with the new product can and cannot do and manage the expectation what can be changed and what cannot be changed. Keep in mind that the UAT is one of the first opportunities for the end-users to see the product in action, make sure it is an impression that counts.

What is Business Acceptance all about?

Welcome dear reader, you arrived at the first post of this blog and here I will explain a bit about what Business Acceptance is and how it can ease your working life.

I have been working on Test and Acceptance management for nearly ten years now and found time after time that the technical realization of a product to support your business flows is nearly under control. Sure, it goes over time and over budget but when the product arrives it is more or less what was promised by the vendor. So when testing the functionality of the product, nothing out of the ordinary is found, it meets specifications and requirements.

So it receives a formal go from the projects steering committee and is moved to the operational side of the business. People are trained, maybe even a User Acceptance Test is executed and an implementation party is held. The project is called a success, the project team is dissolved and your operations get back to their daily life. Where it turns out that where possible, nobody uses the carefully implemented product and those who do hate it. Efficiency is down, customers are complaining about faulty deliveries and employee morale is plummeting. Your people are complaining about the new tools a lot but there is a new mandatory process to be followed so they use it, reluctantly and confused.

What happened here? How could a seemingly fine project delivery and implementation end up in a rut like this is?

Essential is realizing that most projects that have their origin in the IT organization or have an IT product are just that: IT projects. The goal of an IT project is simply to deliver an IT product, an automated tool for the business, for the day to day operations people, to enable and ease their work. Some IT projects realize that the result of the activities are used in actual practice and will put forward valiant effort to instruct or otherwise get the end users up to speed on the product but most see this as an activity out of scope for the realization project.

The core of Business Acceptance is enabling the smooth flow from your IT project to your day to day business operations. The problems that stem from a technical oriented product should be removed with the tools and methods I describe in this blog.

The first is simple: “Realize that an IT project will give you IT, no less and certainly not more.”