issue 2 - Agile Record

The Magazine for Agile Developers and Agile Testers

April 2010

www.agilerecord.com © photocase.com/emanoo

free digital version

made in Germany

issue 2

Es gibt nur einen Weg zum Glück.

Editorial Dear readers, We are very proud to be able to now present the Agile Record in a printed Version. It was not an easy decision due to the low number of advertising, but we hope that the companies will see the added value of it and insert their ads in the following issues. We will continue investing money. We are done with the program for the Agile Testing Days in Berlin. I hope you like it and hope to see you there. We do have 10 tutorials, 40 talks and two nice evenings to chill out. On October 3rd we will celebrate 20 years of the reunification of Germany, the conference starts on the 4th. We have put together a weekend hotel package for those of you who want to join the big party in Berlin. Don’t miss it. The success of the Agile Testing Days in Berlin encouraged us to plan other conferences worldwide, either alone or with local partners. In November this year we will host the OZ Agile Days in Sydney. The call for Papers has just started. In January2011 we plan to have one more in Tel-Aviv (Israel Agile Days). We are also planning the AsiAgile Days, the USAgile Days, the Brazil Agile Days and last but not least the Arabian Agile Days. These will all run in 2011. For those involved in the Finance industry, we would like to inform you about the “Testing & Finance”, the conference for the testing professionals in Finance. It takes place in Frankfurt, Germany. Have a look on the website www.testingfinance.com for further details. We did have a very successful tutorial with Michael Bolton in Berlin. We went out for a dinner and spent over 4 hours having a nice conversation about the business, the future of testing, the people and last but not least about the loved topic “certification”. It was a real pleasure meeting Michael, he is a very knowledgeable person and I look forward to meeting him again. My home town is running for the Candidacy of European Culture City 2016. Please have a look at http://www.candidatecities.com/ and if you like it, please vote for Las Palmas de Gran Canaria (Spain). I would appreciate it. Last but not least I would like to thank all the authors, advertisers, supporters and colleagues involved in this issue. It took a lot of hard work to produce the first printed issue. Best regards and enjoy reading!

Yours sincerely

José Manuel Díaz Delgado

www.agilerecord.com

3

Contents Editorial /3 Software Process Improvement Using Scrum, RUP and CMMi: Three Reasons why this is a Bad Title /6 by Remi-Armand Collaris & Eef Dekker Ten Years After /10 by Miroslav Divis Implementing TDD: The developing tester’s role /13 by Andreas Faes Meaningful Certification? /16 by Markus Gaertner Scenario based automatic testing – a visual programming approach /19 by Nathan Iarovich Agile Methods – an Interview /30 with Gerhard Hastreiter AGILITY TESTING in Business Rule Engine /32 by Krishnachander Kaliyaperumal Thinking about Agile adoption /36 by Sara Medhat Sailing towards democracy on the Scrum boat /38 by Allmas Mullah Performance testing in an agile environment /42 by Juergen Pilz The Agile Organization /46 by Henrik Sternberg Agile at first sight /49 by Paweł Ziemba Financial Analysis Methods and Their Application in SCRUM /52 by Paul Battisson Agile is scary! /54 by Ruud Hochstenbach Mistake-Proofing /56 by Mary & Tom Poppendieck Can scrum prevent defect ping pong? /61 by Patrice Willemot The Agile Bible /65 by Simon Romans Masthead /74 Index Of Advertisers /74

4

www.agilerecord.com

© iStockphoto/Yuri_Arcurs

online training english & german (Foundation)

ISTQB® Certified Tester Foundation Level ISTQB® Certified Tester Advanced Level Test Manager

Our company saves up to

60% of training costs by online training. The obtained knowledge and the savings ensure the competitiveness of our company.

www.te-trainings-shop.com

© Kheng Guan Toh – Fotolia.com

Software Process Improvement Using Scrum, RUP and CMMi: Three Reasons why this is a Bad Title by Remi-Armand Collaris & Eef Dekker

Under current economic circumstances, “software process improvement” is hot again. It’s all about changing the way we work, and doing things right for a change. To decide what’s right, people turn to popular software development methodologies like Scrum and RUP or to a maturity model like CMMi. Software process improvement is often seen as a tool to shorten time-to-market, increase quality and increase customer satisfaction. In order to reach these objectives, however, it is not enough to look only at software development. In order to reach the three objectives mentioned, we have to look beyond the software development process and get the business involved in the creation of software that will deliver value for them. So, the first problem with the title lies in the word “software”. Another problem is that the process is not the first thing that should be changed, but the mindset that led to the current process. For any improvement initiative to be effective, we need to involve the people that execute the process. So, the second problem with the title lies in the word “process”. Initially, the implementation of a methodology or the attainment of a certain level of maturity may work fine as a means to reach our objectives. Once this is formulated, we often see a loss of focus: the means become the end, and everybody strives to implement the methodology or reach the maturity level, losing sight of the objectives. So the third problem lies in the words “using Scrum, RUP and CMMi”. In this article we will show how the mindset needs to be changed, in order to successfully improve on realizing value for the business. Furthermore we will show how this change in mindset can be initiated and consolidated using Agile and iterative practices from Scrum and RUP. Once such a mindset takes root, the organization’s maturity will increase. Finally, we propose a way to keep ourselves focused on the objectives rather than on the means to reach them.

6

www.agilerecord.com

Creating a change of mindset You need people to really feel responsible for the product and for the process used to make that product – responsible not only on a personal basis but as a team. This is true for the software development team, but should also include stakeholders. This means you need to find stakeholders who really hold a stake in the process improvement and involve them. You need to lower the fences between customer and supplier, so that they can at least look at each other instead of looking at the fence between them. This is even more true in outsourcing and offshoring situations, in which the contract should stimulate cooperation. You also need commitment from stakeholders so that the time the stakeholders need to invest will indeed be available. These are delicate points in most organizations, but they are crucial for success. Below, we present a number of Agile practices that can help to reach this mindset. Start iterating so that people can build trust on the basis of results. This point is so important that we will discuss several aspects of the iterative way of working: • Get subject matter experts from the business involved within the development team. Our experience is that once they have been actively involved in a software development effort, they become enthusiastic and will demand to always be involved in this way. • Put a relentless bias on timeboxing, but do it in combination with a ‘neutral’, non-punishing culture. Once you start slipping in time, the relevance of timeboxing will be diminished. It is counterproductive to ‘punish’ individuals for not having x, y or z ready at the end of the timebox. It is far better to assume each individual has done everything in their power to contribute to the success of that iteration. At the iteration assessment, measures to avoid downsizing or even failure of delivery can be discussed and implemented in the next iteration.

• Shorten the time between detailing requirements and having software delivered. This will greatly improve business involvement. The subject matter experts and other stakeholders will quickly see their direct influence on the result.

Create a contract that fosters cooperation.

Create effective communication channels and facilities. An important ingredient for successful cooperation is communication. The project setting should foster mutual understanding of objectives, needs and backgrounds. This leads to a feeling of team responsibility, where every member knows what is needed It’s all about changing and how he can contribute.

and doing things

Practices involved: • User stories or use cases as a means for communicating requirements • One project team room • A planning meeting at the start of each iteration, in which the goals and priorities are clarified and in which the team decides on its commitment for the upcoming iteration • Daily stand-up meetings • Working in pairs (not only the developers) • An evaluation session (retrospective or assessment) at the end of each iteration Create transparency in work products and their status. This ensures that everyone interested has access to the same status information. You may use the following tools to create this transparency: • Daily visual updates on progress (task board and burn-down chart) • A visible list of issues that block progress or slow the team down (impediments list) • A demo of completed working software at the end of each iteration It is no use to hide a problem. You foster trust by showing your problems as well as your ability to solve them. Even if you can’t solve it, it is better to clearly say so, and why. This stimulates people from outside the team to get involved in helping the team out. Doing all this will establish a transparent reporting structure with a minimal amount of work. Look for ‘in between’ responsibilities and solve them. These are responsibilities which nobody takes up because they are not perceived to be ‘our’ responsibility. If two parties think of a particular responsibility as ‘not ours’ and there is no check whether it is covered, we have orphan responsibilities. For example, imagine a service call, in a complex service-oriented environment which involves various departments, does not return an answer. For the past two weeks, several persons have looked into the problem and found ‘their’ part to be okay, whereas the company spirit should inspire them to form a small taskforce committed to cooperate until the problem is solved.

Contracts between supply and demand organizations often focus on the scope of the system to be built and do not explicitly address cooperation. As a result, conflicting interests arise, and it is hard to accommodate change. To create value for the business, however, accommodating change is crucial. Therefore focus your contracts on cooperation and include a reward system, which gives clear signals that loyalty to the project is the same as loyalty to the individual organizations.1

The same point is true at the level the way we work, of individuals: let the interests of right for a change. individuals coincide with the interests of the project. If a software developer will be evaluated not only on the quality of the code, but also on the usability of the product, this will enhance his awareness of responsibility for that product. If a tester will be evaluated not on the amount of bugs found but on the quality of the product, his behavior will change. Make sure that responsibility for the product as a whole is in line with personal concerns for every role involved. Keep focus on the objectives While implementing a methodology or trying to reach a higher maturity level, people involved in the improvement initiative often lose sight of the objectives they were trying to reach. An improvement plan will help keeping focus on these objectives. Elements of an improvement plan format, that we have had good results with, are:2 • • • • • •

description of the current process problem statement stakeholders stakeholder needs a prioritized list of improvement actions planning

First, you need to know what the problem is. Why does an organization want to improve their software development process? They want to improve, because they have encountered problems in their current process. Investigate these problems. Start with a high-level inventory in something called a problem statement. The problem statement includes the persons or groups affected by that problem, its impact and the solution as perceived by the stakeholders. Stating the problem in this way automatically leads to a clear view on who the stakeholders are: they are the ones that have these problems. Then, stakeholders hold a stake. What is that stake, or in other words, what is their interest in the software process improvement? Stakeholders have needs. What are they, and how should they be dealt with? What improvement actions do you propose to meet these needs? For every suggested improvement action, the plan should describe:

www.agilerecord.com

7

• • • • •

why you want to change something; when the action should be executed; what the desired result is; who is waiting for that result and willing to pay for achieving it; how you can measure whether that result has been reached.

Once you have all this, time has come to think about how the improvement actions can be planned in time. Having such a plan helps to avoid an over-enthusiast introduction of ‘the new golden hammer method’. It is good to have a clear idea of what the problem is, and why you think this problem can be diminished or solved by the proposed method. What does your proposal contribute to obtaining the right mindset? We want to make your software improvement effort as successful as possible. This requires a little thought at the outset. Make it clear that you’re about to make a good investment. Conclusion Software improvement efforts should be guided by a very downto-earth practical plan, and should have a very clear focus on stimulating a mindset rather than on introducing a new process, however promising that new process may be. We have given a quick glance at elements that help create the right mindset of responsibility and cooperation: • • • • •

Start iterating so that people can build trust on the basis of results. Create transparency in work products and their status. Create effective communication channels and facilities. Look for ‘in-between’ responsibilities and solve them. Create a contract that fosters cooperation.

If this mindset takes root, increased customer satisfaction, increased quality and a shorter time-to-market will come along. ■

[1] See the excellent book of Mary and Tom Poppendieck, Implementing Lean Software Development: From Concept to Cash, Addison Wesley 2008, p. 216-217. [2] You can download the template at www.scrumup.eu/downloads.html.

8

www.agilerecord.com

> About the authors Remi-Armand Collaris is a consultant at Ordina, based in The Netherlands. He has worked for a number of financial, insurance and semi-government institutions. In recent years, his focus shifted from project management to coaching organizations in adopting Agile using RUP, Scrum. An important part of his work at Ordina is contributing to the company's Agile RUP development case and giving presentations and workshops on RUP, Agile and project management. With co-author Eef Dekker, he wrote the Dutch book RUP op Maat: Een praktische handleiding voor IT-projecten, translated as RUP Tailored: A Practical Guide to IT Projects, second revised edition published in 2008 (see www.rupopmaat.nl). They are now working on a new book: ScrumUP, Agile Software Development with Scrum and RUP (see www.scrumup.eu). Eef Dekker is a consultant at Ordina, based in The Netherlands. He mainly coaches organizations in implementing RUP in an Agile way. Furthermore, he gives presentations and workshops on RUP, Use Case Modeling and software estimation with Use Case Points. With co-author Remi-Armand Collaris, he wrote the Dutch book RUP op Maat: Een praktische handleiding voor ITprojecten, translated as RUP Tailored: A Practical Guide to IT Projects, second revised edition published in 2008 (see www.rupopmaat.nl). They are now working on a new book: ScrumUP, Agile Software Development with Scrum and RUP (see www.scrumup.eu).

© iStockphoto.com/Petrovich9

Managing the Testing Process Training Course by

Rex Black

June 09 to 11, 2010 in Bad Homburg

d limite places

Test managers must take a potentially infinite job—testing a computer system—and accomplish it within tight time and resource restraints. It’s a tall order, but successful test managers have found proven ways to handle the challenges. This course will give attendees the tools they need to succeed as test managers. We’ll look at quality risk analysis, test estimation, and test planning. We’ll discuss developing high-quality test systems—test cases, test data, test tools, even automated test systems— that improve over time. We’ll talk about tracking bugs and test cases. We’ll discuss ways to derive and present metrics, charts, and graphs from the test results. We’ll also cover the human side of test management. We’ll look at ways to measure and manage the skills testers need. We’ll discuss hiring testers. We’ll talk about education and certification for testers. We’ll examine some ways to motivate and reward testers— and some ways not to! We’ll cover working effectively within the project organization, which is especially challenging when you’re the bearer of bad news. We’ll also look at the context of testing. We’ll discuss system development lifecycles and how they affect testing. We’ll cover testing as an investment. We’ll finish up by discussing test labs, test environments, and hardware issues. The materials presented in the course follow Rex Black’s book, Managing the Testing Process, which is the distillation of over two decades of software, hardware, and systems experience. www.agilerecord.com/knowledge_transfer.html

1200,- €

(plus VAT)

3

The Magazine for Professional Testers

www.testingexperience.com

© Orlando Florin Rosu – Fotolia.com

Ten Years After by Miroslav Divis

It’s been ten years since the Agile Manifesto has come and changed our IT world. The Agile Manifesto has pointed us in the right direction – accented the role of people, goal orientation; it reminded us that less (managing) means more (effect). I am taking the upcoming anniversary as an opportunity to express a few remarks – more for software development than for Agile methodologies – I have never worked in a team that pronounced itself as being agile. I use the well- known Twelve Agile Principles1 as a warp for my comments.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale.

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

4. Business people and developers must work together daily throughout the project.

Unfortunately, this is not true. With small exceptions (when you bought a software for you own use and you – as an owner - really want the software to be functional), we all are equal employees or suppliers and our highest priority is to have the job not just today, but also tomorrow, the day after tomorrow, as long as possible. It means in fact, that the software we produce and hand over is the worst possible, only just acceptable for the customer. So we may remake, improve and maintain the software … for ever. By the way, why we all think, that making and using software is generally rewarding activity?

The cases we routinely meet in practice:

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage. There are people involved with developed software, who do not wish for any changes (their salary in their current state is satifactory for them). And we others? Of course, we are ready for changes, but “give us additional resources and postpone the milestones”! Changes everywhere! But someone will ask in the end –were you not able to do it right at first attempt? And what’s more - they will not ask you and think of your answer. They will ask at the steering committee three levels up, and they will answer as well, which causes you a budget cut for the next year. Because then it is them, who are the golden boy/girl. 10

www.agilerecord.com

I have met projects producing 4, 6, 8, 12 releases in a year, with patches not counted, and nobody said they were agile projects. I think this is standard nowadays; if this is an Agile Manifesto merit – let it be said loudly. Perhaps we should realize that Agile Manifesto was born relatively long ago, and it is not necessary to delimit it to the worst Waterfall excesses.

• There are lots of business people, and they are fully devoted to their daily jobs (in any case, they never confess that they may work for your project for months without any impact to their routine – they are irreplaceable, are they not?) • Interests of particular business people are in conflict • There is a conflict between the interests of business people and the project: - Business people lost their jobs - Business people lost their subordinates (with all consequences) - Business peoples’ work will be explicitly described (with all consequences) - Business people will have to change their routines - Business people are specialists, but they are not able to describe or explain their jobs - Business people are not specialists at all 5. Build projects around motivated individuals.  Give them the environment and support their needs, and trust them to get the job done. Developer’s paradise. How perfectly do you have to isolate you agile team from the non-agile rest of the company to get this situation? However, is the close cooperation between these two parts

supposed to happen? And what about the team’s inner conflicts? 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. No, no. We may generally admit that one-to-many interactive seminar is a very effective method of information broadcast. This is the first part of any effective communication; the second part – fixing of what was said – must follow! We are not able to remember all, or to remember it exactly. Why overfill our heads with garbage? If the conversation with colleagues is the only way to get information, I often end up in this situation: • I spend lot of time looking for the right person to ask • The addressed person is interrupted from his/her job and is probably bored from answering the same questions again and again • Such communication is a synchronous process – we both have to have time for it. I know that agile people do not like being suspected of refusing documentation. Not to refuse is not enough. Documentation is very an important tool of effective development. True is, that the only tool you really need for development is – a compiler (for projects of seminar paper size). The interface between man and computer through the compiler (IDE) is fascinating. Here the two irreplaceable parts meet. Unfortunately, we are not able to support other professions (analytics, testers) with such tools. Only programmers are fully equipped, and it means that they are not the right persons to dictate tempo and requirements to the whole team. Those must be set by the weakest. 7. Working software is the primary measure of progress. It sounds obvious. The question is, why has the world been accepting this situation when 2/3 (=67%) of software development projects failed. The onset of Agile Methodologies did not bring any change. Possible explanations are: • We suppliers have superiority and the world must accept it. Actually, this is true from the start of computers and programming. • The characteristics of our branch are not respected, there are inappropriate solutions from foreign branches introduced. • Our production is unrepeatable, so we have to plan using estimations, but we treat them inappropriately: - We do not respect the statistical nature of estimations. - We do not correct estimations during project elaboration, when new information becomes available. The broadly accepted solution is – ask as much as you can and prepare you alibi. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.  Nice wish.

9. Continuous attention to technical excellence and good design enhances agility. It is important and significant, how this point is explained using terms like refactoring, test-driven approach and coding guidelines. My suspicion that Agile methodologies are very code-centric is confirmed. Actually, why not? We will just respect it and consider it as part of the definition within the scope of using effective Agile Methodologies. 10. Simplicity – the art of maximizing the amount of work not done – is essential. On the contrary, as said in point 1 above, our priority is to make everything as complicated as possible (i.e. no one could uncover it), because we need to be needed! Become a unique knowledge bearer and keep the information in your head. Please note the fact that such an attitude is universal - because it is natural. 11. The best architectures, requirements, and designs emerge from self-organizing teams. Sorry, but why? And if so, how do they do this? I should feel very good in such a team, but I am a strong and mature personality. According to1 , where is written: “This is one of the most radical principles of the agile movement, one which I would love to see researched thoroughly by the academic community”, we put it on our wish list. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. OK, and we fire 1/3 of staff for redundancy and they will be happy, because everyone must to go through it. Conclusion One of biggest sins of slighted Waterfall is that it pretended to be a general-purpose methodology. I am afraid that Agile methodologies fell in the same trap – I have never met a limit for using Agile methodologies with one exception – they are recommended for projects with frequently changing requirements. My opinion - we can add this: Agile Methodologies are good for smaller, shorter projects, with a low number of stakeholders, oriented more technologically, with predominance of coding. Off-the-shelf software, utilities, web? But this definition is a job for agile evangelists. Let me introduce the following paragraph with a quote: “The basic problem of programming is the management of complexity “2 . I was asked a few times to introduce testing into companies having common properties: stabilized, successful for years, small (staff of 20-35 employees), without any testing methodology. Their concerns were shorter projects in a limited number of regions. I deduced that the corner stone of their success was this: their projects never crossed critical limits of complexity, so debugging by clicking through the application was enough. If you cross the limit, the specialist, their teams, problems with communication www.agilerecord.com

11

and management will multiply. Also the potential impact of faulty software will increase. This is why for bigger and more complicated projects, the Model Domain (inserted between early directly connected domains Problem and Solution) comes to play a fundamental role. So we are out of Agile methodologies, aren’t we? From articles written by colleagues, who have tested under Agile methodologies, I can sense a certain distance: we did it, but it is not our cup of tea. Is there any special kind of testing - an agile one – which can help? I do not think so – there is only one testing. Those who say “sure, automated unit tests is the solution” are ignorant. And this is another confirmation of Agile Methodologies “field of use” definition (see above). I see naive positivism in the Agile Manifesto – Make software, not war, there are no conflicts, making software is good and we love each other. I judge that the root cause of the contemporary disconsolate state of software development (fruitfulness of development about 30%) lies deeper than in an inappropriate methodology -– it is in the dominant paradigm, oriented around coding using imperative languages. The future is different – use of multi-agent systems, business rules engines or expert system driven applications. ■

[1] Ambler, Scott W.: http://www.ambysoft.com/essays/agileManifesto.html [2] Chandy, K. M.; Misra, J.: Parallel Program Design – A Foundation

Subscribe at www.agilerecord.com

12

www.agilerecord.com

> About the author Miroslav Divis, mathematician by education, man of many crafts - software developer, civil engineering manager, carpenter,… For the last four years engaged in software testing, Prague Accenture ex-Testing Capability Lead, nowadays working for a Czech Insurance Company. If you want to discuss with me, please use my address [email protected].

© mario beauregard – Fotolia.com

Implementing TDD: The developing tester’s role by Andreas Faes

There are two things that need to be said about Test Driven Development (TDD). It is a good thing (especially from a tester’s perspective), but it is only a good thing if it is done properly. The key to success in agile processes is a strong reliance on experience, and on the fact that this is passed on. One of the roles in such a team is dedicated to testing, and this article wants to put a case for having technical testers for agile start-ups in order to ensure that TDD is done properly. When using an agile methodology, testing is never far behind. Yet testing is something that does not come naturally to a developer. It is a skill that has to be triggered, nourished and sustained. A more traditional developer will show eagerness to start his coding, while an agile developer will typically start with thinking his software design over and spending an adequate amount of time on his functional and technical test design. Only then will he start coding. It is this difference that an agile start-up should try to eliminate as fast as possible, and here lies the opportunity for skilled software testers. Dubinsky and Hazzan (2004) break a good agile team down to members with dedicated roles (aside from coding): customer, tracker, coach, tester, etc. Specifically on the testing role, the authors stress that it should be attributed to a developer. His responsibility essentially lies in guiding team members by example. I would like to turn this argument around: a software tester with a sound background in development will have a much bigger impact on an agile team than a developer with a testing role. The developing tester Of course, this seems like a discussion on mere semantics: the developing tester versus the testing developer. The main difference between these two is in their mindsets. While a developer sees parameters in methods, an experienced software tester sees opportunity for invalid input. While a developer envisions his objects to be in certain logical states, a tester will do everything humanly possible to get them in erroneous states.

Even though both groups have the same basic goal – creating fit for purpose quality software – they go about it with different attitudes. And it is in this light that the difference between the two roles should be seen: a testing developer will still be foremost a developer, but with a strong emphasis on testing, while a developing tester is a tester with programming background. Testers in this ‘development’ role will not interfere with software design problems (such as determining the appropriate design pattern to solve the problem at hand). Instead, they will shed their light on the functional aspects of testing through meticulous analysis of the (global) requirements. This can be done, as suggested by Parkin by means of reviews and walkthroughs, either formal or informal (Parkin, 1997). Why in agile projects? Why should it be specifically a tester with a lot of technical baggage for agile projects? In more traditional software development methodologies, an experienced tester will suffice. These projects mainly rely on black box testing, and there is no need for knowledge of the internal workings of software components. But what are the opportunities for a professional software tester with a development background in an agile environment? Take unit testing, the foundation of TDD. A lot – if not everything – has been written about the syntax of a good unit test: it should be readable, speedy, easily maintainable, it should only hold one assert (which is up for debate), test one single concept, etc. (Martin, 2008; Shore, Warden, 2008). Yet not one author provided a clear assessment of what a good unit test suite should look like. It seems a given that all the tests should test everything that needs testing, but the how is another matter. Likewise, it is established by most authors on TDD that unit tests are solely the developer’s responsibility (Beck, 2003; Parkin, 1997). At that point, the developer can fall into some traps, such as using code coverage metrics to prove the aptness of the unit (as illustrated www.agilerecord.com

13

in Bielicki, 2008), or in quickly asserting that the written line of code does what it should do (Parkin, 1997). Other authors offer more useful advice, in describing formal test design techniques for developers (X, 1997). Without going into depth, these examples are sufficient for dealing with simple problem domains, but they stress the importance of experience as the most important trait when designing good tests to deal with the complex realities of software projects. And this is exactly the added value a professional tester will provide: with an understanding of how the unit code – and the production code as a consequence – operates, he is able to pinpoint problem areas by means of code reviews (another important practice in TDD). This close scrutinizing of the test code serves as an extra code review: testing the logic of the testing logic makes for better unit tests.

Last, there is the long-term effect of such a collaboration. Gradually developers in the project will develop a more keen sense of how a tester's mind works, and they will safeguard their software against the tester’s attacks. And at that point, it is entirely the tester's responsibility to steer clear of the pesticide paradox by implementing more variation through other test techniques and methodologies. Likewise, the tester will have absorbed the principles of the agile way of thinking and working, improving his usability in further projects. In total, this will only benefit the entire project: better defect prevention, better and more streamlined co-operation between development and testing, and more time for extensive system testing rather than tripping over easily avoidable defects. By example

Of course, when the unit test process described by both Beck (2003) and Newkirk & Vorontsov is followed, a test list is available: a plain language list of tests needed to test all aspects of a given unit under test. However, for reasons of convenience and speed, this list is often substituted by a list ‘in the programmer’s head’, which makes reviewing impossible. Thus code reviews on the unit tests are often the only possibility for a software tester to contribute to the unit tests.

An example. My previous project fell under the broad umbrella of agile development. Verification received all necessary documentation per iteration, and diligently made the appropriate test cases to ensure the desired coverage – which was near everything humanly possible. For one particularly crucial use case, I discovered through means of formal test design techniques thirteen meaningful paths through this particular use case, and made my test scripts.

This implicitly excludes non-technical testers. For them to be able to contribute, the test list should be carefully constructed and should always be kept up to date; changing something in the code means changing the documentation. This of course creates a lot of overhead for the developer’s sake: time spent on documenting things that do not need to be documented. After all, the unit tests should be self-documenting (Martin, 2008).

Then the software arrived, and I started my tests. One or two failed because the developer had not thought of those particular alternatives, and one or two failed because each of us had interpreted the documentation differently. Looking back, I missed a lot of opportunities to add value to the project.

Immediate & long term benefits It is in this plea for inter-disciplinarity rather than multi-disciplinarity (Van Berkel, 2009) that I see the biggest advantages for this cross-over role. First, there is the immediate gain of such a tester. He is able to give instantaneous feedback on all testing matters, especially those aspects most unfamiliar to developers. Second, in traditional (manual) testing cycles, any defect will be reproduced – and hence resolved – quicker, as familiarity with the programming platform allows them to understand and interpret technical information such as unhandled exceptions. Instead of merely describing the steps-to-reproduce, they might provide early insight by suggesting possible root causes. Next, it might be opportune to provide active hands-on coaching in traditional test design techniques. When developers deploy their test efforts, it is up to the testers to show them where the areas for improvement are: either in terms of efficiency, or in terms of requirements coverage. Then there is the social aspect. Introducing a tester into the development team accustoms the team to dealing with testing practices. Often testing is regarded as a destructive process, and it is in these situations that testing can prove its productive value. It is only through cooperation that error-prone software can be created. 14

www.agilerecord.com

What should/could I have done? First, I should have gone to the developer while he was developing, and informally gone over all of the possible paths (walkthrough). That way, we were sure he had covered them all in his unit tests, and we would have been on the same terms as to what the post conditions for all paths would look like. Second, I should have asked him to show me his unit tests when they were finished; this would enable me to find omissions or new assumptions (review). In the end, this would have cleared time for us both later on in the project, contributed to the overall (social) atmosphere and the quality of the end product. Conclusion Development teams considering the implementation of Test Driven Development methodologies should consider inter-disciplinarity rather than multi-disciplinarity. Adding an experienced software tester with a technical background to the development team can quickly produce added value. While aiding development in their testing tasks, the development team will absorb the necessary skill sets needed to ensure better overall testing, both by development and quality assurance. ■

References BECK, K., Test-Driven Development: By Example, Addison-Wesley, 2003. DUBISNKY, HAZZAN, Roles in Agile Software Development Teams, in: Extreme Programming and Agile Processes in Software Engineering, Springer, 2005. JIMMINK, E., Testing in an organisation going agile, in: Agile Record, 1, 2010. MARTIN, R.C., Clean Code: A handbook of Agile Software Craftsmanship, Prentice-Hall, 2008. NEWKIRK, VORONTSOV, Test-Driven Development in Microsoft .NET, Microsoft Press, 2004. PARKIN, R., Software Unit Testing, 1997. http://www.ivvaust. com.au/UnitTesting.pdf SHORE, WARDEN, The Art of Agile Development, O’Reilly, 2008. VAN BERKEL, J., Is there such a thing as an Agile tester, in: Agile Record, 1, 2010. x, Designing Unit Test Cases, IPL, 1997. http://www.ipl.com/pdf/ p0829.pdf

> About the author Andreas Faes has a background in history and linguistics, and is currently reinventing himself as a ‘developing tester’ by graduating in computer sciences. He has been employed at CTG Belgium for two years, working as an ISTQB certified test consultant in several projects in a number of sectors (healthcare, automotive sector, HR). Andreas has a strong interest in test automation, test tooling, agile and more traditional development methodologies.

Subscribe at www.agilerecord.com

www.agilerecord.com

15

© DXfoto.com – Fotolia.com

Meaningful Certification? by Markus Gaertner

I hereby confess, I’m guilty of being certified. This may seem hard, but it’s a relict from my life before software. Having spent multiple years on swimming sports, ‑ a bit more than a decade ‑ I finished my trainer’s license certification back in 1999. Before that, back in 1996, I also made my certification to become a referee helper on swimming competitions. This article shows the lessons I learned while getting certified as trainer and as referee helper, and how all this relates to most certifications in place in the fields of software, development, testing, and Agile methodologies. Finally, I will propose a meaningful way for certifications, and how to view them in the larger software culture. The problem The problem arises from the common body of knowledge. A while back, Anne‑Marie Charrett wrote a blog entry about her dream of software testing1 . Instead of arguing about the best way to test software, she pleaded to reach common ground and understanding. Instead of arguing about the best way to do software testing, the professionals should rather sit together to reach an agreedupon common body of knowledge. Software testers argue that the time spent on a project not testing, i.e. documenting, test reporting, mission clarification, is all wasted time. Paradoxically, we keep on wasting our time arguing about how to test software. So, the core problem of certifications lies in the agreement on what constitutes knowledge. Certification programs mention one or more of the following: theory and definitions (including history), techniques (principles and models) and tools. Among the several certification programs, each addresses another body of knowledge, which may be a combination picked from this list. And of course, all of them claim that they are based on the knowledge that is the most meaningful. Unfortunately, none of them actually says something about skills for the real world. Janet Gregory pointed this out in her blog entry “About Learning”.2 Certifications in testing, as they stand today, do not test skills nor solve problems. What they are targetted at is memoriza16

www.agilerecord.com

tion of the body of knowledge assumed to be meaningful according to the certification vendor. Neither do the certifications help the testers translate that knowledge from memorization to application, nor do they help testers to transform what they learned in their particular work place. So, what we end up with are testers, that can prove to have known something at a specific time, i.e. when the certification exam needed to be filled out. Recently, Ron Jeffries wrote on the plans of the Scrum Alliance for certifications on programmers and professionals.3 Ron spent time on these new plans to certify Scrum practitioners. According to him, the plans on the Certified Scrum Professional include to be working in a Scrum environment as well as “an acceptable report convincing people that you have a clue”. Despite the classroom certificates common in the software industry, this supposedly will raise the bar on certifications. Though, this raised profile may be necessary, it does not automatically mean that it might be sufficient in itself, either. Reflecting back, I remember one example from my past, where this “certification” model proved to be just paper. We had a father at our swimming club who was willing to help us out as a trainer. Since he had no past experience with swimming and training, he volunteered to take a trainer licensing course. Such a course consists of seven weekends spent learning about swimming and the underlying principles. When he finished his trainer license, he got in touch with us and asked how he could help us. So, we invited him to our regular “management meeting”, which we held once each month. During this evening we interviewed him, on his experience from the course, and on what he would like to do in our local swimming club. He told us that a good trainer needs to wfeature the good ones and get rid of the bad ones. Since this was such a dramatic gap between his viewpoints and ours, we argued the whole evening. In the end, we eventually lost that father and his two kids during that evening, but our underlying ethics were meaningful enough to let them go.

Certifications in sports Taking a closer look on certification in sports, there are two basic types of certifications I came across: referees and trainers. The learning mainly divides up into theoretical principles and rules, practical exercises, and tests or exams. Last, but not least, there is a prolongation period to renew each license. Principles and rules For swimming referees there are clear rules which have to be followed. Faced with a difficult situation at the edge of the pool, the sapient referee judges properly based on the knowledge at the time according to those rules. The situation is the same for other sports, too. There are a thousand different viewpoints in front of the television in major sports games, yet the referee has to decide based on what he saw and what he learned before the game. Similarly, a swimming trainer gets some body of knowledge during the educational courses. Yet, to come up with successful training, trainers need to adapt to the actual training situation and the particular student in the water. During the courses, the trainer learns mainly the underlying principles, like if the degree of lactate in the muscle is too high during training, then the student will get muscle ache the next day; or when this even might be the case. On this knowledge basis, the aspiring trainer learns techniques and practices to choose in order to achieve a certain goal of in training. In the end, the trainer is left alone after some tests and teaches people how they can master their sport. The classroom material of the certification programs in place in software development build the set of rules and principles that your Agile methodology or testing course mainly focuses on. These ingredients are essential, since they include lessons on the overall picture. They help frame the discussion to a particular context. Unfortunately, though principles and rules are essential, they are not sufficient, as they may leave out relevant details which appear in the real world. Practical exercises In the swimming courses, we went through difficult exercises using practical learning parts. We got to know exercises by doing them ourselves in the water. We ate our own food, thereby getting to know whether or not an eight‑year old may or may not have the power and the stamina for a particular exercise. Additionally, we were taught, that any new training exercise should be tried by the trainer himself before the kids do them. Thereby, the trainer knows how hard or easy the exercise in question is. During the referee courses, we also tackled practical evaluations. We practiced the business rules that we learned before‑hand with direct exercises. These exercises referred directly to practical lessons from actual situations at swimming competitions. Therefore, they were meaningful and authentic. Similarly, the course material from good certification programs today is filled with practical exercises on how to apply software test design, how to lead a daily Scrum or how the product backlog

gets managed. By going through artificial software development projects, or dealing with a simple and easy to learn software, people are guided through their transition process to transform the theoretical knowledge into practical experience. The exercises take the application out of the particular context, and into a safe environment, where making mistakes does not blow up the next power plant (hopefully). For the learning of the techniques these practical exercises are necessary to cross the gap between the right and the left brain, thereby transitioning the new knowledge into the long‑term memory. In addition, for swimming courses there were some group exercises, where each individual group was asked to create a training program for the other participants of the course. After leading through the session, these groups got direct feedback from the other aspiring trainers. Also, group exercises resulted in an exchange of experiences in other clubs. This was a vital part of the knowledge that I was able to take home with me. A meaningful certification course has to include these exercises for collaboration, since the professional exchange is essential ‑ for both sports and software development. Do you remember the most significant part of your last conference visit? Most likely this was the expert exchange between the sessions. The same can hold for certification programs. Tests and exams For the trainer license we went through multiple tests after conducting the theoretical and practical parts of the course. First, we needed to fill out a multiple choice test. The test also included questions which asked for fully written answers. After that I had to plan and exercise a training session with a given goal. At the beginning of the practical test, I got a tiny piece of paper with a goal for a training session. Then I had ten minutes to prepare that session. In another twenty minutes, I lead other participants through the training program I had just prepared. After the session I received direct feedback on my performance from both, the course leaders as well as the community of the aspiring swimming trainers who participated. If one of the aspirants failed either or both of the tests, he/she was asked to conduct a verbal interrogation afterwards with the course leads. Similarly, new referees have an evaluation period during which they are escorted to real swimming competitions. Each new referee and referee helper thereby gets practical lessons at a swimming event. The new aspirants get an experienced referee or helper at their side, learning directly hands on from the more experienced colleagues. Open questions that may arise and that were not raised during the theoretical courses get clarified directly. Additionally, the new referee receives direct feedback on his actions and learns how to steer difficult decisions. For software certifications we have just theoretical tests. You need to fill out some piece of paper with the knowledge you might have gained from the course itself or by self education from books, online material, or the course material itself. The theoretical tests simply exercise your memorization skills of that knowledge. Thus far, there are no exams showing that the course participants are www.agilerecord.com

17

capable of transforming the theoretical knowledge into practice. A test manager who can manage people on paper may not be able to lead real people. Similarly, a ScrumMaster may know the terms "iteration" and "product backlog", but this does not say anything about his ability to remove impediments. In contrast to this, practical tests are in wide use during job interviews. New testers get a simple application. In order to apply for the job position, they need to show their talent and their skills at learning a new program and showing relevant experience on their job. The same holds for Agile positions. Beyond certifications that the interviewees might already have, these practical exercises show the interviewer whether the applicant is suitable for the open position or not. Prolongation For my swimming licenses there is a fixed prolongation period during which I can renew them. For example, the trainer license can be prolonged within four years by attending a course over a single weekend. After four years, it expires and the renewal of the license requires more effort. This renewal process ensures that I still show interest as well as relevant practice in the field. Once a license was earned by spending the several weekends on the courses, it states something meaningful. But if you don't put this knowledge into practice over a longer period of time, it may diminish as you forget some of the lectures that you initially learned. This also holds for software certifications. Interestingly there are just few which call out for a prolongation. For referees there are yearly changes to the rules as the federations find out new things, e.g. on swimming clothes. During the prolongation session, participants are also informed about these changes. Interestingly, considering the flux of the software world, there is no comparable effort put into keeping certified software team members up to date on the latest changes in Scrum, XP, or even software testing. Meaningful certification A meaningful certification program should provide the theoretical principles and so far they all do. But it shouldn't be the only thing to take away from a certification course. Besides a theoretical test, a practical test provides the teacher with meaningful insights whether the participant learned the lessons and can start taking actions from the material directly or not. When failing one or both, a follow up verbal test may reveal further insights on whether or not to grant the certificate. Last, but not least, failing the course after having failed all three tests is a consequential outcome. As shown from the sports analogy, a meaningful certification course would include theoretical knowledge put directly into practical exercises. This includes running simulations of a project in the classroom, artificial training exercises on simple programs, which are easy to learn, as well as mini projects with mini iterations. In order to show that each participant has the ability to lead a Scrum or a test team, a written test alone is necessary, but not sufficient. Showing practical relevance to the teacher proves meaningfully that the knowledge has found its way into your brain 18

www.agilerecord.com

and that you can apply it at your work. In addition, escorting new or unsure participants during their first days back at work also helps to foster the knowledge and put it in practice in the right context. Last, but not least, a prolongation course may transfer meaningful insights from the last few years accompanied with the professional exchange from other participants and practioners. Finally, the corporate culture needs to get aware of the flaws of certifications. Certification alone means nothing. As the example from the misled father showed, you need to interview your certified tester or ScrumMaster and watch out if they fit into your working place. If they don't, either intensify their training or get rid of them. As shown in the example, this heavily relies on the support from the surrounding corporate culture. If there is no support, this may be the overall problem as Robert Sutton points out in his book "The No Asshole Rule".4 Maybe we need to create a software development culture, which is vitally suspect about certifications about certifications alone. Certification expresses that you may know something, but it does not express what you have made out of it and whether this fits into your workplace. ■

[1] Anne Marie Charrett, I have a dream, a software testing dream http://mavericktester.com/i have a dreama software testing dream [2] Janet Gregory, About learning http://janetgregory.blogspot.com/2010/02/ about learning.html [3] Ron Jeffries, Developer Quality! … and Certification? http://xprogramming. com/xpmag/developer quality and certification/ [4] Robert I. Sutton, The No Asshole Rule Little, Brown Book Group, 2007

> About the author Markus Gaertner is a software tester for Orga Systems in Paderborn, Germany. Personally committed to Agile methods, he believes in continuous improvement in software testing through skills. Markus co-founded the European Weekend Testing chapter, blogs at http://blog.shino.de and is a black-belt in the Miagi-Do school of software testing.

© Pascal Eisenschmidt – Fotolia.com

Scenario based automatic testing – a visual programming approach by Nathan Iarovich

This article discusses the concept of scenario based testing and shows how scenario based tests can be constructed from existent regular tests using a basic visual programing language and a specially developed GUI system which allows one to define scenarios as separate entities abstracted from the tests themselves. Advantages of this approach are discussed, as well as paths to improving the current solution.

which together with a specially developed GUI system, that will shortly be described, allow us to define and run scenario based tests by abstracting the scenarios from the tests themselves. In so doing, it is possible to create and run scenario based tests using any existent tests, written in any language, with the only condition being that each test is an isolated verification capsule.

Good automated tests are independent from other tests in the same tests suite; each verifies an isolated well-defined functioality. This allows ease of maintenance, maximization of re-use, minimization of tests-dependency failures and the possibility to run a suite of tests in any test-order. In fact, each test is an isolated verification capsule, which ideally sets its necessary environment, verifies a specific state, returns the environment to a pre-known state and potentially returns a test pass/failed value for later reporting.

An isolated verification capsule has the following properties:

Scenario based testing is defined in the context of this document as creating dependencies between individual tests with the intent to run the tests in a meaningful context. Scenario testing is economic because it provides us with the means to group and run certain tests together in meaningful chunks of verifications, while skipping the need to perform other tests if specific conditions are not met. For example, a verification scenario related to a databasevalidation set of tests might indicate to run the individual functional tests only if a first database connectivity test passes successfully. This is economic compared to running the entire suite of tests and have it all fail because of a broken database connection to start with.

1. Non-greedy – i.e. verifies a single functionality, a single state – doesn’t attempt to verify multiple things in the same context 2. It is not code dependent on any other test – i.e. it does not use variables, handles, etc. defined or opened in another context than its own – note that an isolated verification capsule might still be semantically dependent on another – i.e. it might be meaningful to run the present test only if another test returns an expected result. 3. In all its execution paths – success, failure, abort –the tested environment is returned to the pre-known state before the execution of the test began 4. Returns a value when its execution finishes – pass/fail/ abort etc. – the need for this property will be discussed shortly. Our GUI system allows wrapping any existent test system using the following approach.

Different test frameworks employ different methods to define related-tests groups. These range from special harness-recognizable attributes defined in code (C#), global definition flags/markers, or different initialization files read-in by the test harness at run-time. Our approach to scenario definitions in this article is different. We present a specially developed visual programming language, Figure 1: Overview of general concept www.agilerecord.com

19

Each test system is characterized by the following generic architecture.

the canvas. Our GUI system represents the underlying test system as a tree – this information being available in the xml mediation layer (see Figure 5). Control objects has the same functionality of the if conAn if condition dition object widely used in programming flowcharts – specifically it has a single input which it evaluates and two outcome paths – one outcome will be executed if the input is a positive value (in this case a “Passed” value); the other path of execution will be selected if the input is a false value (in our case a test which returned a “failed” or an “aborted” value). This why we need an isolated verification capsule to return a value when its execution finishes; as discussed in a previous paragraph.

Figure 2: Test system structure

Usually a test system characterizes a single test department/ organization/products family. This is usually mandated by the unique shared services layer inside the test system, which addresses configurations for a single product set. The Xml mediation layers are predefined xml files that are used to expose the lower system to our GUI based tool. These files have to be constructed according to a predefined format and for each underlying test system, have to be maintained as the test system evolves. The xml mediation layer fulfills the following roles: • Maps the whole tests and suites of the lower test system so that the GUI system will know them • Indicates what harness is used to run the tests of the lower test system and where is its location so the GUI system will be able to invoke it at run time • Is used to store scenario diagrams, serialize and deserialize them as needed The role of the adaptors in figure 1 will be explained shortly. They are originally not part of the test systems, and have to be developed subsequently in order to allow interfacing between the lower test systems and the GUI system at run time.

is a control object with a user-adjustable inteA loop object ger counter which indicates the number of iterations a specific path has to be re-run. Scenario based tests definition and representation Armed with the info above we can construct our first scenario – see Figure 3 below. The scenario below is constructed dragging control objects and execution tokens (tests) on the canvas.

Our visual programming language Our simple basic visual programming language is created using a flow chart control which allows us to define scenario delimiters, execution tokens and control objects. Scenario delimiters delimiter or a de­li­miter. A scenario delimiter is a A start delimiter marks the start point of a scenario’s execution. An end delimiter marks the end point of a scenario’s execution path. Execution tokens also known as an isolated An execution token is a test, verification capsule. A test is created by dragging a test node on 20

www.agilerecord.com

Figure 3: Our first scenario

The scenario is completely abstracted from the underlying testing system in the sense that the underlying system has no knowledge of the flow diagram we created. Yet, the GUI system permits visualizing the underlying tests and grouping them in scenarios. We have created a logic dependency flow. And now we want to run this scenario. In order to do so, it is necessary to translate this abstraction in a set of commands for the lower system to be invoked according to the logic above. In our application this is achieved by a 2-pass translation process with both passes being performed at run time. In the 1st pass code inside the GUI system walks the diagram nodes and translates the scenario above into pseudo code. In the 2nd pass we translate this pseudo code into real code using the specialized adaptors (see Figure 1). The

Figure 4: Two consecutive runs of the scenario - results of test are reflected visually in the diagram

real code generated by the adaptor matches the underlying automated system’s code. In addition, the adaptor may be used to communicate with the GUI system and feedback the GUI system on the progress of the running tests – in such a way visual feedback is possible in the scenario diagrams showing the user the status of the running tests. This communication can be achieved by sockets based IPC. For example here is the same diagram after the above scenario has run with visual feedback on the status of the tests (i.e. “Yellow” here indicates that the test execution has been “Aborted” and “Red” means that the test has returned a “Failed” result). From the point of view of the “if”-condition object, execution is considered Failed for both an “Aborted” as well as for a “Failed” test outcome. Detailed logging of the execution steps in each test is the responsibility of the underlying

automated test system. Here we have run the scenario twice, after initially observing an environment problem which caused the connectivity test to be aborted and the ping test to fail. In the second run of our scenario the simple query test fails and needs investigation (see figure 4). Advantages of visual test scenarios The creation of visual test scenarios has several advantages. Abstraction, serialization and portability Visual scenario diagrams are abstracted from the underlying test system and so can be serialized and ported to be run on

Figure 5: Scenarios visible as thumbnails stored in the xml

www.agilerecord.com

21

Figure 6: Pattern template extraction from a given scenario

other test machines that host the original tests or to be run above other test implementations provided the name of the tests remain identical and a suitable adaptor for real code translation is provided. In our system, the xml file is a place keeper for the scenarios diagrams themselves. Scenarios are represented as thumbnails once the xml file is loaded by the GUI system (see figure 5). Each scenario can be opened and run against the underlying system. Test pattern templates for new scenarios generation Another advantage is the ability to abstract the graph flow from the actual content of the execution capsules and create templates upon which different scenarios can be constructed. For

Figure 7: Patterns library

22

www.agilerecord.com

example the scenario described in figure 3 above can be thought of as a frequently occurring verification pattern: “test A – if test A passes than test B and test C else if A fails – verify D”. This test pattern may be employed not only for testing the above database functionality, but can also be used for testing various things in other contexts where a first key test case result is the determinant of how to continue the subsequent test effort. Our GUI system allows abstracting this pattern into an empty flow graph, which can be subsequently filled in with different other test cases that semantically should be run in the same manner (see figure 6). Once a pattern has been extracted it can be stored in a patterns library and can be used to generate new scenarios by specifying new content for their execution capsules (i.e. other tests to run).

Figure 10: Possible use of the scenario diagram as a bug attachment in a defect management system Figure 8: New scenario generated from the same template pattern but run-

More complex scenarios

Chaining of multiple scenarios

In addition, employing additional control objects – i.e. loops – allow us to define more involved scenarios.

ning different tests

Multiple scenarios can be chained together and executed conditionally in order. This capability allows for more intelligent grouping of tests. For example we might want to perform a more involved diagnostics scenario if a first functionality scenario fails. In our example the functionality scenario is the one in figure 3. The diagnostics scenario can be chained to the functionality one in case this last one fails (see figure 9). Scenario diagrams as attachments in defect management systems Since scenarios are diagram based objects they can be saved to image files and can be used as bug attachments. We can add a note object to provide meaningful comments for the scenario and help drive the bug investigation.

Figure 11: Adding a loop object to repeat a certain scenario part several times

Figure 9: Chaining of multiple scenarios to achieve more meaningful testing

www.agilerecord.com

23

Future developments

Exposing global configurable test parameters at diagram level

Additional control objects for enriching the scenarios

Another possibility of enriching visually programmed test-based scenarios is to expose global configuration parameters of underlying individual tests (if available in the underlying system) and have them configured on the fly by manipulating their values from the scenario diagram in order to help reconstruct real world issues. For example, the following describes a real world failure scenario: “If we run test A with a frame length set to 64bytes followed by a ping test and then rerun test A with a frame length of 128 bytes we experiment a consistent failure” (presumably during the above flow, state builds up in the system under test, is not cleaned up properly, and causes test A to fail when we increase the frame size). This can be translated in a scenario diagram, saved and played back accordingly in order to reproduce the issue (see figure 13).

Enriching of scenario based tests can be improved in several ways. One way is to use more control objects. For example there might be test scenarios in which an external process is required to be activated in the background to monitor status or interact with our tests. This can be represented by a configurable object that can be setup to run a specific executable in the background, and can be inserted and activated in sequence inside our scenario diagram. The process object and its properties are xml serializable along with the entire scenario diagram so that the scenario may be ported to run from machine to machine.

APIs to create and run scenarios The system presented so far allows creation of scenarios by manually dragging objects on a canvas. Scenarios are validated so that invalid flows cannot be created (i.e. there are associated rules for each diagram object as well as for the flow itself). Future enhancements to this solution can be made by developing an API to automatically generate valid scenario based tests programmatically, as well as run them automatically in bulk, and thus allowing the system to scale up and be able to run a large set of scenarios per single run. In addition, the system can be extended to employ visual debugging using the scenario diagrams themselves. ■ Figure 12: Process object extends the scenario by having a needed executable triggered in the background at the right time

Figure 13: Advanced scenario created by exposing global configuration parameters to affect test behavior

24

www.agilerecord.com

Final words This article has presented a visual programming approach to scenario based testing. Our aim was to show the underlying possibilities in defining, running and extending scenario based tests visually. We have shown how scenarios can be constructed, saved, run, as well as potentially extended to cover more involved states and how they can be used in reconstruction of real world issues. We have also shown how scenarios can be abstracted from the underlying system, yet have the ability to communicate with it by means of an xml mediation layer and specially crafted adaptors, in order to be able to run the tests themselves according to the defined logic flow, as well as provide visual progress status of the run. In addition, we have discussed the possibility of integrating scenario diagrams in defect management systems as bug attachments, and in this way provide a useful visual aid in bug investigations.

> About the author Nathan Iarovich has been working in the Israeli test industry for the past 12 years where he filled different test management and architect roles with specialization in automatic testing. For 5 years (2002-2007) he has managed, designed and developed the Functional Testing System of Marvell Semiconductors Israel in the Switches and Routers Division - to this date still in use in the company. Since 2008, Nathan has been employed with the Microsoft Corporation where he continues with his passion in testing, working on different interesting projects. In parallel to his daily work, he also presents work at testing events, and has recently lectured on “Agile Automation Testing” during the December 2009 event of the SIGiST Israeli forum (Specialist Group in Software Testing). Prior to becoming a full time Microsoft employee, Nathan has worked as a freelance test automation consultant for Microsoft Israel and in parallel to that he privately developed Windmill (the system presented in this article) and researched the topic of scenario based testing using visual programming.

Subscribe at www.agilerecord.com

www.agilerecord.com

25

Agile

TESTING DAYS

Agile Testing Days 2010 4–7 October 2010, Berlin

The Agile Testing Days is the European conference for the worldwide professionals involved in the agile world. We chose Berlin, one of the best-connected capitals in Europe to host this event due to the affordability/service ratio for hotels and flights. Berlin also offers a lot of fabulous sights to visit in your spare time. Please have a look at the program at

www.agiletestingdays.com

and enjoy the conference!

Agile

TESTING DAYS Berlin, Germany

October 4

Tutorials

Lisa Crispin

Janet Gregory

Elisabeth Hendrickson

Stuart Reid

Isabel Evans

Linda Rising

Michael Bolton

Jennitta Andrea

Anko Tijman

Eric Jimmink

Pekka Klärck

Juha Rantanen

Janne Härkönen

October 5

Conference

Day 1

Track 1

Track 2

Track 3

08:00

Opening

09:30

Keynote – Lisa Crispin Incremental Scenario Testing: Beyond Exploratory Testing

Experiences on test planning practices in Agile mode of operation

Matthias Ratert

Eveliina Vuolli

11:30 11:50

a Díaz & Hilterscheid Conference

Testing Web Applications in practice using Robot Framework, Selenium and Hudson

Mike Scott

Thomas Jaspers

Test Driven Migration of Complex Systems

Making GUI Testing Agile and Productive

Real World Test Driven Development

Dr. Martin Wagner and Thomas Scherm

Geoffrey Bache

Emanuele DelBono

Continuous Deployment and Agile Testing Alexander Grosse

Acceptance Test Driven Development using Robot Framework

Talk 5.2

Pekka Klärck

Lunch

14:20

Keynote – Linda Rising Test Center and agile testers – a contradiction? Dr. Erhardt Wunderlich

Requirements in the Agile World

Error-driven development

Genefa Murphy

Alexandra Imrie

16:20 16:40

Testability and Agility – “Get your quality for nothing and your checks for free” Break

12:50 15:20

Track 5 – Vendor Track

Registration

09:25 10:30

Track 4

Flexible testing environments in the cloud

Talk 5.3

Jonas Hermansson

Break Solving the puzzles of agile testing

Structures kill testing creativity

Matthew Steer

Rob Lambert

Integration Test in agile development

Waterfall to an Agile Testing Culture

Anne Kramer

Ray Arell

17:40

Keynote – Elisabeth Hendrickson

19:00

Chill Out/Event

Talk 5.4

Agile

TESTING DAYS Berlin, Germany

October 6

Conference

Day 2

Track 1

Track 2

Track 3

08:00

Track 4

Registration

09:25

Opening

09:30

Keynote – Michael Bolton

10:30

Top 10 reasons why teams fail with ATDD, and how to avoid them

Agile hits Formal – Development meets Testing

Gojko Adzic

Matthias Zieger

A lucky shot at agile? Zeger Van Hese

TDD vs BDD: from developers to customers

How can the use of the People Capability Maturity Model® (People CMM®) improve your agile test process?

Mitigating Agile Testing Pitfalls Anko Tijman

Maximizing Feedback – On the importance of Feedback in an Agile Life Cycle

12:50

Lunch

14:20

Keynote – Stuart Reid The secret sauce of Agile testing in distributed teams

Hitting a Moving Target – Fixing Quality on Unfixed Scope

Fully Integrated Efficiency Testing in Agile Environment

The Flexible Art of Managing a Multilayered Agile team

Cirilo Wortel

David Evans

Ralph van Roosmalen

Shoubhik Sanyal

Mastering Start-up Chaos: Implementing Agile Testing on the Fly

The coding dojo as a forum for teaching Test Driven Development

Implementing collective test ownership

Christiane Philipps

Emily Bache and Fredrik Wendt

16:20

Talk 5.2

Lior Friedman

Cecile Davis

16:40

Talk 5.1

Break

Alessandro Melchiori

15:20

The Leaner Tester: Providing true Quality Assurance Hemal Kuntawala

11:30 11:50

Track 5 – Vendor Track

Talk 5.3

Break

Eric Jimminik

Markus Gärtner

17:40

Keynote – Janet Gregory

19:00

Chill Out/Event

October 7

Open Space

Alternative paths for self-education in software testing

Exhibitors

Open Space hosted by Brett L. Schuchert. Day 3

Track

08:00

Registration

09:25

Opening

09:30

Keynote – Isabel Evans

10:30

Open Space

11:30

Break

11:50

Open Space

12:50

Lunch

14:20

Keynote – Jennitta Andrea

15:20

Open Space

17:20

Keynote – Tom Gilb

18:20

Closing Session

Agile

Record

Talk 5.4

Agile

Please fax this form to +49 (0)30 74 76 28 99 or go to http://www.agiletestingdays.com/onlinereg.html.

TESTING DAYS Berlin, Germany

Participant

Billing Address (if different from participant)

Company:

Company:

First Name:

First Name:

Last Name:

Last Name:

Street:

Street:

Post Code:

Post Code:

City, State:

City, State:

Country:

Country:

Phone/Fax:

Phone/Fax:

E-mail:

E-mail:

Remarks/Code:

Tutorial

(4 October 2010)

Making Test Automation Work on Agile Teams by Lisa Crispin

The Foundations of Agile Software Development by Jennitta Andrea

A Rapid Introduction to Rapid Software Testing by Michael Bolton

Testing2.0 – agile testing in practice by Anko Tijman & Eric Jimmink

A Tester’s Guide to Navigating an Iteration by Janet Gregory

Managing Testing in Agile Projects by Stuart Reid & Isabel Evans

Patterns for Improved Customer Interaction & Influence Strategies for Practitioners by Linda Rising

Agile Transitions by Elisabeth Hendrickson

Executable Requirements in Practice by Pekka Klärck, Juha Rantanen & Janne Härkönen

750,– EUR (plus VAT)

Early Bird – register until June 30, 2010 to get the discount

625,– EUR (plus VAT)

Conference

(5–7 October 2010)

1 day (first day)

550,– EUR

3 days

1.350,– EUR

1 day (second day)

550,– EUR

2 days

1.100,– EUR

1 day (third day)

350,– EUR

1.350,– EUR (plus VAT)

Early Bird – register until June 30, 2010 to get the discount 1 day (first day) 1 day (second day) 1 day (third day)

440,– EUR 440,– EUR 250,– EUR

3 days 2 days

1.080,– EUR 880,– EUR

Included in the package: The participation at the exhibition, the social event and the catering during the event.

Settlement Date Payment becomes due no later than at the start of the event.

Notice of Cancellation No fee is charged for cancellation up to 60 days prior to the event. Up to 15 days prior to the event a payment of 50 % of the course fee becomes due and after this a payment of 100 % of the course fee becomes due. An alternative participant can be designated at any time at no extra cost.

Liability Except in the event of premeditation or gross negligence, the course holders and Díaz & Hilterscheid GmbH reject any liability either for themselves or for those they employ. This particularly includes any damage which may occur as a result of computer viruses.

Date

1.080,– EUR (plus VAT)

Applicable Law and Place of Jurisdiction Berlin is considered to be the place of jurisdiction for exercising German law in all disputes arising from enrolling for or participating in events by Díaz & Hilterscheid GmbH.

Signature, Company Stamp

© beckmarkwith – Fotolia.com

Agile Methods – an Interview with Gerhard Hastreiter Allianz is one of the largest insurance and banking organizations and has multicultural teams applying several different development and quality assurance approaches. Which was the driver to introduce agile methods at Allianz in Germany?  In the end it is all about productivity and velocity. Doing the right things with the right quality, making the most efficient use of our resources and delivering as fast as possible in an ever-changing environment. This is of course the goal of any software development process, but we think that “agile” provides the best framework for achieving this goal.   What did you expect from introducing agile methods? Market leadership? A quicker time to market?  IT is core to our insurance business. In fact, selling and servicing insurances is nothing but processing information. Thus, in many cases IT massively influences our competitive position, and any method improving IT productivity directly contributes to our business. For me, however, the most important issue is balancing short-cycled prioritization with a stable environment for the people doing the real job, which is developing software. On the one hand, we live in a world of ever faster changing requests and immensely rising cost pressures. On the other hand, in order to be efficient and to produce high-quality results, you need solid ground. I think, “Agile” provides the best framework for balancing these needs and focusing on the real business needs. How did you introduce agile methods? What role did training and coaching take?  Simon Roberts, a coach we work with closely, calls our approach “Introducing Agile by Lightning Beacons”, and this describes it pretty well. We didn’t really start bottom up, but had strong support of senior management right from the start. Nevertheless, we did not make this a “brute force” top-down initiative. Instead, we 30

www.agilerecord.com

selected some “lightning beacon products” in order to pilot the agile approach. Of course, we selected products which had the preconditions to write success stories, for example proper team composition and modern technology. At the same time they had to be complex enough, providing the chance for us to learn from our experiences as well as not being regarded as simple showcases  from the outside perspective. Methodically we tried to use Scrum rather by the book, enriched with some elements of Extreme Programming, for example test-driven development or pair programming. Building on these beacons, we increased the scope towards bigger team sizes step-by-step (scrum of scrums), more complex platforms and mission-critical products. Finally, we even used the method to develop from scratch a web interface for the newly founded Allianz Bank within five months. Parallel to this step-bystep approach, we set up an enterprise transition team at senior management level,which is responsible for the implementation on the large scale. From my point of view, coaching was essential for the process. You can’t implement this kind of change based on reading textbooks. You need people  thinking and talking “agile” based on personal experience, skilled and passionate to overcome the obstacles which inevitably crop up. You should be careful in selecting these coaches, and I personally believe that they should also have a strong hands-on background in software development, architectures and tools in order to be accepted by the respective teams. Which were and are the biggest problems that you faced?   Frankly, I think the biggest problem still lies ahead. It will be making the change sustainable, making “agile” part of our DNA. I feel, we are in danger of losing momentum, some teams starting to cherry-pick only the elements they like, some product owners only following half-heartedly.

Would you recommend doing things differently if you could start again? I would start earlier to try and gain the commitment of business owners and business analysts. We started this basically as an IT initiative, supported by some early movers on the business side. Later in the process it showed that it is even more of a change for business owners and business analysts than it is for IT project managers or developers. There seems to be some fear of losing control by not producing huge and complete requirement documents prior to implementation or of losing autonomy by having to interact so consequently and regularly with the whole team. Of course these fears can be overcome in almost every case, but the role of the product owner is so key  that  there should be a very early commitment that “agile” is a substantial improvement of the process as a whole, not just the latest fad of these IT guys. Somewhat in the same category fall our  budgeting,  reporting and controlling procedures. In the early stages we tweaked them somehow to make them fit to the new method, for example map-

In fact you should find that reflected in ROI or customer satisfaction, but since we don’t sell software but insurance, these values are superimposed by many other factors.  Nonetheless, I feel very safe to claim that we now have in place some teams which produce a constant stream of  substantially higher value and higher quality compared to any of the more traditional approaches. What were the expectations of the teams involved and what did they get in the end? People working in the IT business are from my perspective intrinsically curious and  love to work near the cutting edge, also methodically.  Agile methods  also promise a more continuous stream of work and output, while we still suffer from huge “peak pressures”. We have not yet reached this state of flow in every case, but - again - I know some teams and products where we are pretty close. 

Dr. Gerhard Hastreiter, Division Manager D-IT Customers & Sales, Allianz Deutschland AG

ping “projects” onto the new product view. Now we are gradually moving towards the latter perspective, for example in our budgeting processes, which by the way substantially simplifies the management of budgets that are getting tighter. But, retrospectively, I think, we should have started earlier and more firmly to adapt these formal processes, as it is important to make change happen in an organization as large as ours. Did you measure the results? For example ROI, costs, quality, time to market and customer satisfaction?   What we do measure are velocities and - traditionally - of course the other performance indicators you stated. But that is not what most people - especially the more critically minded - would like to see. Most people would like to have a handy comparison of these values before the introduction of agile methods and now. We don’t have such a comparison. Even in an artificial setup, implementing the same requirements over a sufficiently long period of time, this would probably be difficult. But that is not the point. The point is that by using agile methods the final product should look very different from the one developed using more traditional methods.

On the other hand, if someone had expected that “agile” would be a great excuse for a lack of discipline or efficiency, I am convinced that everybody now knows that the opposite is true. The agile rule set is simple, but doesn’t provide much place to hide. Answering the questions “What did I achieve yesterday?” and “What am I going to achieve today?” on a daily basis creates remarkable pressure as well as transparency provided by your team’s daily burn-down chart. Is there a plan to introduce agile methods within other parts of Allianz worldwide? Yes, an international initiative is already on its way. What is coming next? Again, the top priority is to make this change sustainable by further increasing the number of products we develop agile, by confirming the commitment of other stakeholders, especially of business owners and business analysts, and by further adjusting our formal systems to these methods. Then, we will face the challenge to substantially offshore external capacities we used to use onsite. We are convinced that maybe the only way to do this successfully is by doing this agile. ■  www.agilerecord.com

31

by Krishnachander Kaliyaperumal

Abstract The purpose of this document is to describe the Agility Testing approach towards the Business Rule Engine. The agility testing strategy will establish the total testing time and cost effectiveness. From the business side, the Information Technology solution for business challenges shall be flexible with respect to changes in market needs. This sort of testing approach will reduce three fourth of the testing time. Business Challenges: The business organization engineered a centralized solution – a shared service that orchestrates processing and fulfilment from both web self-service and call centre channels. The system provided the speed to market and technical capability of the Service Oriented Architecture approach – now adopted by the bank as an enterprise standard. To launch a new product, work and intelligence have to be built into all the applications which require lot of time and effort. Changes are expensive, as they require a system development. It is difficult to launch a new set of products using Java or .Net platforms in a short time span. Business Rule Engine takes the intelligence part for an application and reduces the turnaround time of launching the new products in applications. The business organization had to bring new products to the market in a timelier manner. The bottleneck problem in this process is each application’s ability to handle and reflect the required changes.

Moving towards Service Oriented Architecture means processing business rules through a specialized engine, which is flexible and easy for product introductions, leaving the business-critical applications to handle the request and response parts. Business-critical applications call the Business Rule Engine to display the lists of offers available to the customer, where the offer eligibility rules are configured in the Business Rule Engine. Configuration changes to the Business Rule Engine can be deployed independently of application releases. Agility is defined as the ability to change without being dependent on a release based on code changes. How does it do this? • Application calls the Business Rule Engine to show the offers available to the customer. • The rules are configured in the Business Rule Engine, not hard-coded in the application. • System application is ignorant of eligibility changes made in the Business Rule Engine. • The application GUI should not require updates to absorb the Business Rule Engine’s changes. • The Business Rule Engine configuration can drive the application display. Agility Testing Approach: The new business requirement is common for all the applications with few differentiations in the scope of testing. The impact is relevant to all the applications which are in the scope. The scenarios are framed as rule sets (explained in part 5).

A Solution to the Business Challenges: Configuration System: The Business Rule Engine has the ability to introduce new products independent of the releases by configuring the new offer in the Business Rule Engine rather than hard-code the product in desktop applications and web applications. 32

www.agilerecord.com

To achieve this, the testing team organizes a meeting to come out with a common scenario to be tested by all the respective teams towards their application. A spreadsheet is introduced to

© Surrender – Fotolia.com

AGILITY TESTING in Business Rule Engine

Figure 5.1 – Schematic representation of Rule Groups and Rule Sets

map the scenarios to the requirements. This common configuration system is configured in the Business Rule Engine as the “rule set”. What is a rule set? A rule set is a configuration which has been aligned with the scenarios framed for testing. What is a rule group? One or more rule sets consolidated in a file, which is known as rule group.

For example, let us take three environments, say Environment 1, Environment 2, and Environment 3, which are as follows:

Table 6.1 – Environments available

Rule Groups for Agility: (Fig 5.1) • Rule Groups allow to group and to turn on specific rule sets in the Business Rule Engine • Allow testing teams to group and target specific capabilities and scenarios. Example • Rule Set 1 has the production configuration • Rule Set 2 overrides the production configuration of business requirement 1. • Rule Set 3 overrides the production configuration of business requirement 2. • Rule Set 4 overrides the production configuration of business requirement 3. • Rule Group 1 turns on Rule Set 2 and Rule Set 3 • Rule Group 2 turns on Rule Set 4 Overriding the production configuration occurs on a priority ranking basis. The production configuration is given less priority when rule set 2 is given higher priority to turn on. Testing Environments: In the real world, we have different types of applications and XML interfaces to support applications. The new product shall be implemented in all the applications, where the scope of testing differs from a functionality perspective.

Figure 6.2 – Environments and Rule Groups

In figure 6.2, Rule Group 1 has been turned ON in Environment 1, which is known as agile mode. Testing team 1 will test the rule group 1 scenarios. At the same time, Rule Group 2 will be turned ON in Environment 2 for testing team 2 to test their part. Similarly, Rule Group 3 will be turned ON in Environment 3 for testing team 3 to execute their test cases. This is a simultaneous process which happens in three different environments. These rule groups will be turned ON only for certain time. After the agility testing is done, it will revert back to its normal mode. Rule groups will be swapped accordingly in all three environments to complete the testing cycle within a limited time. Note: Environment 1 will be used by different testing teams (desktop application and Web application) Advantages: This sort of testing reduces the total testing time/cost effort to release the new offers and business changes in timely manner. www.agilerecord.com

33

Figure 7.1 – Agility Configuration Testing Strategy

Agility Configuration Test Strategy: (figure 7.1) • Allow testers to specify the scenarios to be tested – Rule Groups: - The testing will be completed which is in the scope against the configured rule groups. • Rule Sets contain the configuration of business requirements in the Business rule engine • Pre-configure Rule Groups – Pre-configuration of rule groups happens in the Business Rule Engine. • Deploy Rule Groups at test teams’ request – Based on testing teams’ request, the rule groups will be deployed in their corresponding environment for testing. • Communicate agility testing mode & progress – Intimation will be given to all about the agility mode turned ON by the testing co-ordination team. Agility Testing Process: (Fig 8.1) Define a common set of test scenarios that meet agility test scenarios cross applications/stream • Determine the agile components for each system application/ stream, which are controlled by the business rule engine • Generate generic test scenarios based on allowed permutations of the components controlled by the Business rule engine Configure the test scenarios into a standard ‘Agility Rule Set’ • Combine system applications’ test scenarios into a single set • Document scenarios and indicate per scenario which test teams it applies to • Business rule engine owners will configure these rules into an ‘Agility rule-set’ Deploy this as part of a build into all environments • The agility rule set is deployed into all environments, but not as the leading rule set. 34

www.agilerecord.com

Fig 8.1 Agility Testing Process

Switch individual environments into Agility mode • Testing co-ordination will co-ordinate switching to the agility ruleset during agility testing periods • All users are notified as part of the normal environment change notification processes • Test teams will test scenarios that apply to them Defect Process • Agility defects are raised indicating agility testing and test case number • Affected environments will be put into Agility mode to test defect fix in testing cycle 2 • Depending upon the severity, the rule groups will be turned on in parallel environments to address and close the defects. Test Management Framework: Test Management Framework provides testers and test managers with an application and environment for creating test plans, performing tests, and monitoring progress on applications being

tested and deployed. The Test Management Framework allows managers to define, assign, and monitor test segments, scenarios, and suites. Tests can be based on use cases from the Application Profiler, test cases from Automated Unit Testing, or can be built from scratch. Users can also attach files and/or effort estimates to tests. The Test Management Framework generates and monitors a test plan based on tests and estimates.  The framework then reports on results against the plan, quality, and completeness during testing.  The Test Management Framework supports testers by presenting and managing assignments. Users perform tests and report results using the framework. Test Management Framework features include: • Construction of test segments, scenarios, and suites for multiple applications • Creation of work assignments for testers • Inclusion of use cases and test cases from Application Profiler and Automated Unit Testing • Generation of comprehensive documentation including test plans, progress reports, quality results, and coverage details • Compliance with automated and user-run tests Test Management Framework benefits include: • Complete coverage from details to entire applications • Improved testing with managed processes • Consistent testing throughout projects • Visibility into quality and completeness, including management reports

lution design should be based on ‘easy to maintain’ way. As a result, corporations can adapt to market needs rapidly and efficiently. Integration: The Business Rule Engine can be integrated with other technologies like mainframe and web services (Request and Response methodology). Tool: Test Harness (tool) is put into practice to test the Business Rule Engine, which mimics the integration part. Conclusion: There is a huge learning curve required in order to face the technical challenges The Business Rule engine adds good value to the business (centralize and maintenance). No new release is required for a new product to release in market. Adjust to changing market needs. ■ Agile Results/Benefits: • 100% increase in the number of product releases per year • Increased customer satisfaction by including minor features in maintenance releases • Increased product quality and team productivity through enhanced morale

Significance to Information Technology business: References In today’s modern Information Technology industry, where coding for an application is firmly coupled with delivering to the business in a timely manner, the Business Rule Engine process captures the significance in IT. Challenges: Analyzing the present business rules and designing the business solution is a big challenge. Strategic Advantage: The idea is that all business rules are centralized and maintainable in one location. The Business Rule Engine supports any kind of business rule that is configurable, whether it is simple or complex. This kind of technology will move the CODING part outside and compose the business optimally with excellent performance. From an Enterprise level, if one product is successful using the Business Rule Engine, the entire corporate will make use of it. The ideal principle is to unify and streamline in order to create profit to Corporate. Organization Needs / Solution: An IT solution should be flexible and change as per market needs and business competition, adjust with the competitor. The so-

http://pega.com/industries/financial-services/ wholesale-banking.asp http://www.infoq.com/articles/Agile-Business-Rules-Taylor http://www.gartner.com/ DisplayDocument?ref=g_search&id=491111

> About the author Krishnachander Kaliyaperumal presently working for HCL Technologies, India and has over 5.5 years of experience in software testing. I have worked for Telecom, Aerospace, Banking domains. I am currently responsible for Test planning, preparation, execution and defect management. I hold a master’s degree in Computer Applications. My areas of expertise include Software Testing and I am a certified ISTQB professio­ nal. [email protected] www.agilerecord.com

35

by Sara Medhat

IT companies nowadays think about the decision of moving to Agile. Is it the right step for us? Will it succeed? What is the risk? Is Agile suitable for us as a small company/big company? Many questions like this are flying around in every company thinking about Agile adoption. When it is announced in a company that “Our company is going AGILE” these magical words make everyone in the room spinning and looking in each other’s eyes wondering: What does it mean? We are good this way, why move to Agile? What do we need to learn? What will be my role in an Agile team? What if we FAIL? People are always afraid of the unknown, we are afraid of the dark because we cannot see in it. It is the same situation here, people are afraid of the transition to Agile, because they are afraid of how things will be. It is the role of the decision-makers in the company to convey the message in a positive way, so that employees feel positive about it, and become eager to learn all about Agile. Let’s agree that transition to Agile is a cultural challenge. Getting people out from their comfort zone and introducing new roles, new responsibilities, and a new culture needs patience, and needs the company to empower the team not to lose their identity in the transition phase. Making the company goals clear and share the vision with the employees will help getting the desired results. Introducing Agile into a company is a huge step that will determine whether the transition phase will be successful or whether people will resist the idea. The introduction phase of Agile is like introducing a new product in the market; it requires a well-planned marketing campaign to succeed and to make people enthusiastic enough to know, learn, and practice Agile. Moving to Agile Software Development Teams will be motivated to move to Agile if they were able to see, feel, and understand the value added from practicing Agile. Moving immediately from traditional methodologies to Agile may be risky. 36

www.agilerecord.com

The best practice is to move gradually and to start with a pilot project and invest time with its team. Only then should one reflect if moving to Agile is the right step for the company or not. Applying basics of Scrum, Lean, XP, or whatever approach you want to apply will motivate people to learn more about it and when they get the value from it, they will spread the spirit all over the place. We are good the way we are • Almost 45% of delivered features in software are unusable. In Agile we do the highest-value work first and stop when we deliver the required business value. • Most of the time we have low customer satisfaction after delivering the agreed product. 60-75% of the customers do not work again with the same company, as they did not get what they expected. In Agile it is the highest priority to satisfy the customer, which will guarantee great reputation in the market and potential projects with the same customer • Requirements are not a Holy Book. The customer who is providing us with the requirement is the one who asks for changes. People hate change, because it means rework and introducing chance for new changes. In Agile we are open to change and feedback, because we are working iteratively and changes will be easily managed. • Developers have to read the requirements book (sorry, requirements document) before starting implementation with no time to understand the business and get what the customer really wants. In Agile, business people and developers work together, which enables the developers to develop better solutions, and to deliver what is expected from the client’s side. • Developers, designers, or architects can’t resist complex designs and abstract meta-models, but what is the business value from such a complex design? In Agile, technical excellence is about using the right frameworks and supporting tools to be able to deliver higher quality software.

© macroman – Fotolia.com

Thinking about Agile adoption

What to Learn Agile is a mindset. You may need to think differently to cope with an Agile team. Agile teams consist of value-driven members, who are collaborative and understand the business very well. It is a team. in which everyone shares the same vision and looks for the same results for their project. Learn to be a team player; Agile is not a one-man show. Agile is about working, delivering, taking responsibility of our project as a team not as individuals, and whether we won or lost we will also be the team’s result. Agility is the methodology not the technology, it won’t tell you how to be a team member or how to do your job right, but it facilitates communication between team members, and between team and customer.

> About the author Sara Medhat is a Senior Software Test Engineer, ISTQB (International Software Testing and Qualifications Board) Certified, and Software Testing Trainer with 5 Years of experience in quality control, and in the software testing field. For the last 2 years, she has been working with agile teams as an Agile Tester and Project Coordinator at Santeon Inc.’s Egypt office.

What will be my role in an Agile team? Collaboration is required between all team members to deliver business value iteratively. Also, engaging the customer from the beginning is necessary, so early feedback can put the team on the right track of development, in order to deliver a highly valuable product. People will resist if they feel that transition to Agile will threaten their positions or jobs. So the change needs to be designed, and have an identifiable start point and endpoint. Moving with people in an iterative, time-boxed manner, you may get less resistance, and even grow with them. At the beginning explaining each team member’s role and identifying what is expected from him will help people to do their parts correctly. An Agile team member may have a specific role, or may play more than one role; it depends on the work agreement between the team and the criticality of the time and budget for a specific team. What is your role in an Agile team? It actually does not matter as long as you are doing it right and everyone is gaining value from it. What if we fail It is always recommended before you start a new technology or take a new approach, that you have a clear understanding of it first. If you did not, then do not be surprised if you fail. Before deploying Agile into your company, you should first know what the problems are that are to be solved with Agile. Identifying your problems may save you time, effort, and money in your transition to Agile. You have equal chances to fail or to succeed, so do not think about it much and start now. If you succeed, you can move forward with it and even transit to a higher level with your company. If you fail, well at least you then know that Agile does not fit your company at the moment. Whatever is the outcome, do not leave it to chance. Remember, it is your choice! You know what suits you and what would work for your company. The world is gaining speed around us, and we need to cope with it or we will be left far behind. So, do not leave let a chance for gaining an experience pass you by without taking it into consideration. ■ www.agilerecord.com

37

© João Freitas – Fotolia.com

Sailing towards democracy ... on the Scrum boat by Allmas Mullah

We, the people We, the people are social creatures. We, the people share interpersonal relationships with other people, some personal, some professional. The relationship between us, the people working in the information technology industry could be described as a ‘facultative symbiotic relationship’. Wikipedia describes it as a relationship that is beneficial but not essential for survival of the organisms.1   Traditionally, the leadership style in any industry including the information technology industry has been autocratic.   The leaders must have been inspired by F.W.Taylor who said, “In the past the man has been first; in the future the system must be first... The first object of any good system must be that of developing first class men.” Frederick Winslow Taylor was one of the first management consultants and is regarded as the father of scientific management.2 Most us of have experienced bosses who take all the important decisions, hide information, give commands and closely monitor our work. Some even give you a phone call at midnight to ask if you have completed your task. Others may ask you to come back to the office to solve an emergency issue, while you are enjoying the ride back home after a long day at work. Expectations about you having to work late nights, on weekends and public holidays is a given. After all, you have to stretch, they say. Such leaders were commonplace when the traditional waterfall model for software development was used within the companies. And the Battle of Waterloo continues! A typical software development project would comprise of a project manager responsible primarily for scheduling, resourcing and product delivery. 38

www.agilerecord.com

The project would have one or more teams depending on the size and complexity of the product under development. Each team would have a team lead, programmers and testers. The manager and the team lead would invariably have a technical, programming background. The manager would prepare the high-level plan with project milestones using some popular project management software program like Microsoft Project or simply Microsoft Powerpoint. He would allocate work chunks to the teams and ask the team leads to divide and distribute that chunk of work between their members. So now the team had a plan. They didn’t have to think about when to do what. They could come into office and ‘just do it’. It was simple, thought the leads, people just have to do what they are told to do, within the time that they are given. But this did not happen. ‘Slippage’ became the buzz word. The manager would ask the team leads, “How much slippage?”, leads reply “We lag by 5 days only”. Manager says, “Oh ok .... carry on”. After a few days, “How much slippage?”, leads reply “We lag by 15 days only”. Manager says, “Hmmm ... buck up guys”.   After some more days, “How much slippage?”, leads reply “We lag by 25 days only”. Manager says, “What? … you have to make your team stretch”. On the day of the proposed milestone, “Have we reached the milestone?”, leads reply “Not yet, we’re 70% done.” Manager, in high pitch, “I want the first milestone achieved on the 10th day from today.”, bangs fist on table and leaves. The team leads go back to their cubicles and announce aggressively, “Guys, no leaves or weekends for next 10 days. We have to achieve the milestone. You just have to do it!” We, the people who create software, lead interesting lives. They tell us that something that we know takes a month to be developed is must be coded and tested in 10 days. The manager has been generous enough to allow the leads to take in more ‘resources’. It’s like the math problems at school, if 5 men can build a wall in 10 days, how many men can build it in 1 day. So two programmers are added to the team. (It’s always the programmers and not testers that are so critical for delivery.)

So the team works for 10 days, they stretch as much as they can and finish off the work. The manager beams on day 11. He is happy that he would not have to change the project plan and also face an irate customer. The customer wants to see what is built so far. It is sent to him. At the customer site, the users try out the product. They are appalled by its poor quality. There are bugs they would never like to see. They raise an alarm. The manager is called and given a feedback. The fuming manager returns to the office and calls a team meeting. He points his finger towards the testers and says, “What have you been doing? How did the bugs escape?” The most senior tester gathers courage and speaks up on behalf of her team, “The testers did not get sufficient time to test during the last few days before release.” Of course, the manager wasn’t going to accept this explanation. The same story repeats itself at each milestone meeting. The testers maligned and de-moralized, blame the programmers for their woes. The programmers retort by saying that testers efforts are not up to the mark. This causes hostility and animosity amongst working partners. There are teams within teams, the tester team and the programmer team, who are constantly at loggerheads. It’s no fun to work in such a place or to work with such people. It’s like the Battle of Waterloo, with just one difference, this one doesn’t end!   The New Beginnings The management knows quite well that their ship is rocking on rough waters. Either they have to quickly sail to calmer waters or don life-jackets. Just as women go label-shopping when they are sad, management decides to shop for the latest trend in development, “Agile”. They decide to bring in Scrum to their world. For this transition, they hire an expert coach, a Scrum guru. A software development process coach is revered as a modern-day saint with an invisible halo. All the people are assembled in the conference room and our halo-guy gives his first discourse. “Welcome to a new world.”, he says, “A world where you rule.” “Wow, this sounds good.”, we think. Ears perked we eagerly await what he has to say next. 3 Scrum, he says, is an agile software development framework. We shall work in time-boxed iterations, these are called “sprints” and are typically two to four weeks long. We shall begin each sprint with the planning meeting. This will be attended by all members of the team where they are required to put forth their view and make a commitment about work that they are confident of completing in the sprint. Because the output of every sprint is a “potentially shippable product”, simply put, a piece of code that works as per requirement! In most cases, this also means that the piece of code has passed through the hands of the business analyst, the programmer and the tester, who collectively have “developed” it. “So far so good.”, we think! “The team shall have a daily meeting,” says the coach. “Oh no, not another string of meetings,” we brood.

“The daily scrum meeting or the stand-up meeting is a short 15 minutes meeting where each member of the team answers just three questions: “What did I do from the last stand-up till now?” “What will I do from now till the next stand-up?” “What are my obstacles?” The objective of this short meeting is to create transparency and to gauge progress. The added value of this meeting for team spirit is that, if a member is facing issues then the others can pitch-in to resolve these issue, rather than the person with trouble having to approach individuals for help and wasting time in doing so. This statement felt like a breath of fresh air to us. The coach continued, “On the last day of sprint, there shall be a sprint review meeting. The team shall give a demo of their “potentially shippable product” to the stakeholder. The team shall also retrospect on what went right in the sprint and what areas need improvement. This so that the mistakes are not carried forward to the next sprint. Similar to the Plan-Do-Check-Act cycle (PDCA). 4 Coming to roles, there shall be a Product Owner, a Scrum Master and the rest are simply called the team. The team is supposed to be self-organized. You shall pick the task from a prioritized list of customer requirements called the sprint backlog. “What? We get to choose the work? Is this for real?”, we look at him wide-eyed with disbelief. The Product Owner shall be responsible for the sprint backlog, he is the acting agent of the customer. Typically, a business analyst plays this role The Scrum Master is someone who shall bring order from chaos, if there is any, typically he will not be a working team member, but rather a neutral observer, so that his decisions are not emotionbased#. He shall endeavor to remove obstacles, technical or otherwise, and ensure smooth functioning of the sprint. “How shall we see the progress?”, someone asks. The coach replies, “With a burn-down chart.” But how shall we measure the progress?” is the next question. “With complexity points,” says coach. “In the planning meeting, when you take up a task, you shall also give the task complexity points, typically any number from the fibonacci series like 1,2,3,5,8,13.5 The team shall debate about the number, some may think the task isn’t too complex, while others may think that it would require substantial time. Once agreed upon by all, the number is finalized. This estimation exercise is called ‘planning poker’”. “So this is Scrum. Let’s start using it,” the coach ends smilingly. We wonder aloud, “Is that all? Is it that simple? Does it work?” The coach looks serene as he replies, “Good is usually simple!”

b. Image credit

www.agilerecord.com

39

I’m lovin’ it! We are now 5 sprints old and loving it. The synergy and output have been amazing. Of course, there have been hiccups, and we’ve stumbled a few times. But we’ve learnt from the mistakes and have only improved. The team camaraderie is one to be seen to be believed. We’ve been telling our friends from other workplaces, “If you are not already using Scrum, ask your manager to consider it.”

> About the author Allmas Mullah is a black box software tester based in India.  Having worked in the software development field for nearly a decade, she has experienced different approaches of software development, from Waterfall to the latest Agile-Scrum. She blogs at “http:// allmasmullah.blogspot.

What Scrum gives to an individual: • • • •

Self-discipline Decision-making power Pride Importance (all member feel they are adding value, hence feel important) • Equality (all are treated equal: programmers, testers, business analysts) • Not having to face bullying or finger-pointing • Sense of ownership and responsibility. This is achieved through: • Frequent, transparent communication. • Knowing what customer wants and how customer feels on a regular basis. • Teamwork (in the truest sense)  • Doing important things first. • Measurable results in a short time span • Chance to recognize and act upon mistakes early. Every coin has two sides, every process has its share of pros and cons, Scrum is not an exception to this rule.But the pros side of the Scrum coin does shine bright! ■

[1] http://en.wikipedia.org/wiki/Symbiosis [2] http://www.quotesdaddy.com/author/Frederick+W.+Taylor [3] http://www.scrumalliance.org/learn_about_scrum [4] http://en.wikipedia.org/wiki/PDCA [5] http://en.wikipedia.org/wiki/Fibonacci_number

Image credits: [a] boat image:  http://www.bizarrebids.com/mechanical/ ugly-wooden-boat-with-tempermental-engine-06/ [b] Scrum process: http://www.scrumalliance.org/ learn_about_scrum

40

www.agilerecord.com

com/” and tweets @Allmas

Sydney 2010

November 22

November 23

Tutorials

November 24

Conference

Conference

Call for Papers Dear experts, agile testers, presenters … please send your proposal for the next

OZ Agile Days 2010

the Australian conference for the professionals involved in the agile world using our template at www.ozagile.com until May 30, 2010 to [email protected].

© Sebastian Kaulitzki – Fotolia.com

Performance testing in an agile environment by Juergen Pilz

Performance validation is an integral part of every software development project. But in agile projects, it takes on a higher role. It makes it possible to deliver a potentially shippable product or application increment. This means that, if desired, the work done in iteration (a sprint) has gone through enough checks and balances (including meeting performance objectives) that the application could be deployed or shipped to meet business needs. The decision to actually deploy or ship an application increment is influenced by many other factors, such as the incremental value added to the application in one sprint, the effect of an update on a company’s operations, and the effect of frequent updates on customers or end users. Therefore, performance is not the only issue—but it’s an important issue today.

While good application performance is an expected outcome of any release, its foundation is put in place during the release planning stage. Considerations include user stories that describe performance requirements and functional user stories that can potentially affect the performance of an application.

Incorporating performance testing in agile processes Performance testing is an integral part of agile processes; it can help to develop higher quality software in less time while reducing development costs. The goal is to test performance early and often in the development effort, and to test functionality and performance in the same sprint. That’s because the longer you wait to conduct performance tests, the more expensive it will become to incorporate changes. Performance testing becomes more important for Web 2.0 applications because mashups that incorporate content from multiple websites and databases into customized Web pages bring in a whole new set of performance challenges. The Web 2.0 applications are not only more dynamic, but also have many small components that could affect performance and the end-user experience. With Web 2.0 applications, there are a lot more moving parts and everything is interwoven—from client scripts to network traffic, from Web services to server scripts.

Performance testing can help you answer questions such as these: • Does the application architecture need to be modified to meet the performance guidelines? • Will the use of newer technologies such as AJAX degrade the performance of the application? • Does the IT infrastructure of the testing and production sites need to be upgraded? • Will the user interface designs degrade the performance of the application? • Can an expected increase in usage impact application performance? • Will performance suffer when the application is made available in new geographies?

42

www.agilerecord.com

Picture 1: Agile performance validation as it should be

But what does it take to incorporate performance testing into your agile development processes? It is not something that happens after an application is pretty much complete. Per-

formance testing that takes place at the end of the software development lifecycle (SDLC) in a waterfall process moves to the beginning in an agile process, along with analysis, design, and coding. Performance of an application is directly proportional to its design and hence should be addressed at an early stage of the development lifecycle. Performance testing is taken into into consideration throughout the agile SDLC—from the release planning stage onward. One way of ensuring that performance testing is included early in the SDLC process is by having a clear definition of “done” and its adoption in the agile implementation so that the application is tested for functionality, performance and security by the end of each sprint and hence is available for incremental deployment, if desired. In an agile project, the definition of “done” should include the completion of performance testing within a sprint. It is only when performance testing has been completed that a successful application can be delivered confidently to the end users. The best practices for performance testing within a sprint are:

response times, numbers of simultaneously supported users, and other factors that affect the end-user experience. These requirements, which become part of the application backlog, must be met before an application can go live in a production environment. Application owners and stakeholders should be interested in the performance aspects of the application right from the start. They determine the priorities for the SLO requirements. For example, those requirements that have the greatest impact on the application—and by extension, the business—are given higher priority in the development process. For example, an application owner might offer this user story: “As a user, I want all the pages of our website to be refreshed within two seconds so that I can spend my time more productively on our website.” Here’s another example story: “As an application owner, I want the application to support as many as 100,000 users at a time without degrading the performance of the application, so that I can make the application available globally to all employees of my company.” These examples set the requirements for the application, which in turn will define the acceptance criteria and drive the test scripts.

• Gather all performance-related requirements and address them during system architecture discussions and planning. - Work closely with end users and stakeholders to define acceptance criteria for each performance story. - Involve performance testers early in the project, in the planning and infrastructure stages. - Make performance testers part of the development (sprint) team. - Ensure that the performance testers work on test cases and test data preparation while developers are coding for those user stories. - Get performance testers to create stubs for any external web services. - Deliver each relevant user story to performance testers as soon as it is signed off by the functional testers. - Provide continuous feedback to developers, architects, and system analysts. Share performance test assets across projects and versions. - Schedule performance tests for off-hours to increase the utilization of time within the Picture 2: Performance SLA results screen showing transaction response times and goal definitions (HP Perfor­mance Center) sprint. Key considerations in agile performance testing

Focused performance testing

To gain a closer understanding of how performance testing is integrated into agile processes, let us look at some of the key considerations in performance testing.

SLOs spell out the expected behavior of applications. But in a practical sense, development, test teams, and the business don’t always know the exact requirements for an application until they see how it performs in a production environment.

Service-level objectives Service-level objectives (SLOs) drive the planning of performance requirements in agile environments. Through SLOs, business and IT managers agree on requirements for application throughput,

For this reason, agile processes rely on the production availability lifecycle (PAL) to see how an application actually performs in the real world. The feedback from the production environment helps developers and testers focus on specific problem areas of the www.agilerecord.com

43

application, thereby increasing the resources during short sprint cycles. This is an important ongoing step in the agile development process. It provides developers and testers with actual metrics and usage trends. They can then take those findings into account when they work on the next version of the application. Real-world metrics can drive better tests—and better software. Test data preparation In an agile environment, sprints are short— just two to four weeks typically—so testing Picture 3: Overview of available timeslots and resource availability for performance testing (HP Perfortimes are short too. This makes it all the more mance Center important to bring test data preparation into the planning stage, so that there is enough time for performance virtual development team, including contractors working on an testing. outsourced basis, access to the same test assets. Test data preparation may be done under the guidance of a line of business (LOB) or some other area of the business that has a Among other benefits, the repository provides efficiencies that stake in the protection and privacy of the data that will be tested. come with “follow-the-sun” testing. Test scripts can be created For instance, a stakeholder might want to mask a data set to over the course of a day in one geography and then made availprotect the privacy of a confidential customer or employee inforable to testers in another geography, who will run the test during mation. their business day. The earlier test data preparation takes place, the more time you have for testing. So performance testers should work with stakeholders in the planning stage to prepare tests and test data. This kind of collaboration is one of the keys to getting a lot of work done in a relatively short sprint. Different types of work must happen simultaneously. In this sense, agile software development is a bit like just-in-time (JIT) manufacturing. You plan carefully for your needs, and then bring in the resources just when you need them. Trending In an agile environment, it’s important for application owners to see continual improvement in an application over the course of successive sprints. They want to see a favorable trend, where each iteration of the application is better than the last. This makes it all the more important to monitor application performance trends in terms of SLO requirements. Trending reports allow you to give stakeholders regular snapshots of performance, which should ideally show that performance is getting steadily better or at least is not degrading. In addition, by looking at trending reports you do not necessarily have to study the analysis on every test run. Reusable and shared testing assets To accelerate testing and development work, agile processes should make use of a repository of reusable and shared testing assets. This repository of test scripts gives everyone on the 44

www.agilerecord.com

The ability to reuse and share test assets becomes more important with agile development when the testing cycle is limited. It allows more work to get done within the available time window. Automated testing The use of automated testing tools can speed up the process of performance testing. With the right software in place, you can create a script, make it reusable, and then schedule a test to run in the off hours, when developers are not changing the code you’re testing. This makes it possible to achieve higher levels of software quality in less time. Automated testing helps you meet your regression and performance testing objectives within the tight timeframes of a two-to four-week sprint. This becomes even more important when you consider that developers often hold onto their work till 60 percent of the time has passed in a sprint, before handing off the build for testing. That doesn’t leave a lot of time for testing. Continual analysis In agile processes, continual analysis is important. Both contributors (“pigs” in scrum terminology) and stakeholders (“chickens” in scrum terminology) need to keep a close eye on the progress of the project, especially when it comes to application functionality and performance. To give them the view they need, performance analysis should be both continual and comprehensive. This ongoing analysis helps pinpoint any problem areas in the application. Analysis takes place all the way down to the daily scrums that include IT infrastructure and performance testers as contributors and application stakeholders. Contributors are active members

of the sprint team who participate in daily scrums, which give all stakeholders visibility into the current state of the development effort. When all interested team members know the performance of each sprint, they are in a better position to keep the quality of the entire application high. Component testing In an agile environment, there is no end-to-end application in every sprint. This makes it important to be able to performance test only a portion or a component of an application. Stubbing provides a way to test components. Stubbing simulates parts of an application that are either not written or not available. If an application uses outside, third-party data sources—a common occurrence in a Web 2.0 world—then performance testers and quality assurance (QA) specialists are going to need stubs because they cannot add load to third-party production servers. By performance testing components inside each sprint, you will help enable the development effort yields of a high-quality application that performs well from end-to-end. One last check! A hardening sprint is an optional last step in the agile process. This sprint is dedicated to final checks, such as a last round of testing, end-to-end performance checks, and tests to verify compliance with SLOs.

While it can highlight problems that need to be addressed, the hardening sprint is really there to increase everyone’s confidence in the application before it goes into production. If you’ve done things right—and made performance testing part the entire agile SDLC—you should have no surprises. The hardening sprint should not be the first time that performance testing is being carried out. ■

> About the author Juergen Pilz, Presales Manager at HP in Germany, has more than 20 years of   experience in software de­velopment and testing. He started his career as a programmer, worked as an applications engineer at Applix and managed the presales team for Applications at Mercury Interactive in Central and Eastern Europe. In his current role he is Manager Presales and Sales Lead Germany for the Applications Portfolio of HP Software.

A day in the life of an agile performance user story • Application Sales lead generation application offered as softwareas-a-service (SaaS) • Current user base 30,000 subscribers • Vision Driven by the business: “Grow total subscribers to 150,000 in the next 12 months by upgrading the application’s user interface to more modern Web 2.0 for more interactive and engaging process flows and notifications.” • Roadmap requirement High-level and not so specific: “As an application owner of the sales leads application, I want the application to scale and perform well to as many as 150,000 users so that new and existing subscribers are able to interact with the application with no perceived delays.” • Release and sprint requirement Specific “As an application owner of the sales lead application, I want the application to perform well to as many as 150,000 simultaneous users so that when subscribers change their search criteria, an updated count of leads available is refreshed with two seconds on the screen.” • Sprint execution Each sprint checks the two-second search requirement is maintained. Multiple sprints found that the two-second requirement was slipping. But each time it was fixed immediately. • Outcome The application is delivered on time with the two-second search requirement because the performance requirement was included in each sprint. If performance had been put off until the hardening sprint, the product would have been late or the requirement would not have been met.

www.agilerecord.com

45

by Henrik Sternberg

The agile principles and practices are widely accepted in the IT industry today. Experiences are mostly positive and optimistic, but experience also shows that the surrounding organization can have difficulties “absorbing” the agile movement. This means that there will be a conflict between the agile projects and the rest of the organizatio,n which challenges the utilization of the full potential of the agile movement. What is good system development? The agile movement is based on principles that we as system developers to a large degree regard as “common sense”, being hard to live up to, though. Principles as close contact to the customers, aiming for high customer satisfaction through frequent deliveries, obtaining quality through motivated, highly skilled teams and built-in feedback mechanisms. These principles, however, are not built on common sense alone. They are also built on solid system development theory. As system developers in a pragmatic world, we often forget that our profession IS indeed a profession, which apart from our indispensable experience includes well-founded theories about system development. And we often forget that understanding of the theory can help us when our experience fails. And finally, that theory can help us create a common language. One of the theories is described in “Professional System Development” [Andersen et.al 1986]. This theory is based on a description of the coherence between the system development elements. The main activities are the activities we ALWAYS carry out in a software project, regardless of the choice of methods: analysis, design, realization, planning, evaluation and regulation. “Analysis” refers to all activities aimed to understand the current situation, NOT a specific phase. The activities are mutually dependant and presuppose each other. This means that you cannot have a specific phase, say an analysis phase which includes all analysis 46

www.agilerecord.com

activities. We MUST carry out system development with regards to the mutual dependency, for instance through prototyping and iterative, incremental development. Principles over practices The agile methods and practices are built on a common philosophy and common principles for good system development, aligned with the theoretical base. These principles are described in the agile manifesto. When agile methods and practices are implemented, it’s often out of the wish for a more efficient development process, but equally often without knowing if the organizational conditions for such a renewal are present or wanted. Often, and typically in larger organizations, we see that agile attempts can result in resistance, isolation or even rejection of the attempt. Agile methods have typically in these situations revealed organizational conflicts. There might be conflicting approaches on how to talk to customers, or what to expect from them, for instance. The agile principles are about handling uncertainty. But this ability cannot be obtained independently from the surrounding organization. It is an ability the entire organization must have. Handling uncertainty is an overall theme in software development, and all project stakeholders must embrace this: developers, project managers, sponsors, product owners, requirement owners, users and so on. Agile principles build on assumptions of organizational behavior and self-perception. Courage, appreciation, respect and trust cannot thrive in isolation. Trust assumes trust. The conditions for the members to be able to show courage, appreciation and mutual respect requires an organization that supports this. An organization that expects software projects to anticipate and plan for all system development activities from sketches to solutions alone, and that understands deviations from the plan as

© tom – Fotolia.com

The Agile Organization

inappropriate, must have difficulties supporting the agile project manager’s need to make contracts without detailed descriptions. So, to obtain the desired effect of agile practices, we must seek the principles and values, that lie behind the organization. I don’t mean values written on the mouse pad, or on posters in the canteen. I mean the values that live in the organization. If a lived value in an organization is that the customers are important, it MUST imply that contract negotiation starts with handling the uncertainty that is inevitable in software development projects. In organizations that live by principles of hierarchical, knowledgebased management and measurements, and from rewarding personal achievements, agile projects will “stick out”, as they will promote principles of common responsibility. So agile principles cannot be isolated to the system development department. IF we want to be able to benefit from the improved process efficiency, the principles that govern system development must be in agreement with the principles that govern the general behavior in the organization. ■

> About the author Henrik Sternberg I have been working with software processes and methodologies for 15 years. As a teacher and as consultant. Now I work as an independant consultant, and our aim is to improve our use of processes by focusing on: - development of efficient teams - development of shared visions - use of sustainable practices Our methods is based on dialogue and appreciative approaches.

References Andersen, Kensing et.al. Professionel Systemudvikling, Teknisk Forlag 1986

Subscribe at www.agilerecord.com

www.agilerecord.com

47

©iStockPhoto.com/Inok

The Conference for Testing & Finance Professionals

June 7th and 8th, 2010 in Bad Homburg (near Frankfurt a. M., Germany)

Mo and re in for re www .te gist mati ra on sti ngfi tion nan a ce. t com

Keynotes by

Nitin Bhargava

BJ Rollison

Gernot Nolte

Barclays Bank, UK

Microsoft Corp

Fiducia

Sponsors

Exhibitors

Supporting Organisations

Prof. Dr. Hermann Schulte-Mattler Fachhochschule Dortmund

Rex Black RBCS

© Dron – Fotolia.com

Agile at first sight by Paweł Ziemba

There are many articles and books about Agile programming, about its strengths and advantages in comparison to the traditional – heavy - approach. It may appear that better cooperation with clients, emphasis on dialog, direct impact on process details etc. are simply good... There is no weakness? When I started my first agile project, I had the feeling that problems were hiding somewhere...

ferent roles – analysts, architects, testers, technical writers etc. In Agile, every single team member should do all that work on his own, so every single team member should have enough skills and knowledge.

Now I can identify some of the basic factors that distinguish agile from traditional heavy methods. Obviously, these factors have some commonly used meaning, but I would like to mention my three initial reflections, presenting my own definition of them. Responsibility One of the first factors, for someone who is involved in an agile project, is responsibility. Here both the single developer and the whole team are responsible for the project. It is one of the most important benefits for development – everyone eats the whole cake, not only the sweeties on top and the cream, but also the baked part. In the old approach, where everyone has their own task - when we forget to plan something, the situation will probably become mixed up, like in the Charles Osgood tale about Anybody, Everybody, Somebody and Nobody. It has sometimes been assumed that it is a reason for the daily stand-ups. Of course, no one wants to be ill at ease when he does actually nothing. But the purpose of the short meetings are different. It isn’t a confession time, it benefits from what teammates have learned by themselves and through the observation of each other. We can consider it as a kind of process. It’s amazing how it happens, but there is the other side of the coin. When we take the responsibility for the project, we should get all the equipment needed – skills and knowledge. The level of competence is the key. When we start an agile project, we need experienced team members, and then an experienced team. In a traditional project, like Prince2 or PMBoK, we have several dif-

According to a learning theory called the Learning Pyramid - reading or listening are improving knowledge less than discussing, practice and teaching. Fortunately, the Agile methodology puts interactions and collaboration over comprehensive documentation. This is the hidden, but I think the most powerful advantage of this modern way. Team members can increase their knowledge without much additional effort - simply by discussing, practising and immediate use, whilst in the traditional methodology they can only read the documentation and then remember 10% at most. Of course, we can’t give up reading. Learning is a process which gradually develops our knowledge from the lectures, through visual demonstration, to the bottom of the pyramid. We should often give each other the opportunity for discussing, practicing and teaching. But still stay careful – learn one thing at a time. Dealing with too many things at the same time carries a danger that nothing will be done well enough. Similar skills are in the classic theory of A. Maslow, divided into four stages. Let’s consider a situation when we learn some new things, e.g. SOLID Principles. Firstly, we don’t even know that something like SOLID exists – it’s Unconscious Incompetence. www.agilerecord.com

49

so on. In this meaning, applying a methodology like Scrum may be helpful, we can keep solid rules. But we can’t remove all changes. Changeability is in the nature of Agile, directly from the Manifesto. The key is to realize that everyone is a separate being. Everyone has a different character and habits. The client has his own goals, the team has their own, and probably each team member has his own personal goal as well.

Then we become aware that we are incompetent at these principles – we’re in the stage of Conscious Incompetence. We can now develop skills by thinking and using them – Conscious Competence. And at last we are applying SOLID without thinking about it – we are in the Unconscious Competence stage now. The most effort we must put in at the second and third stages, but again some Agile tools, such as as XP or Pair Programming, carry us through that process.

Following The Art of Project Management by Scott Berkun, we can split goals into three layers. The Vision – mostly developed by a client, Team Goals which the team should achieve, and Individual Goals. In this approach we may see that the resistance for introducing changes has different natures, depending on goals levels of changes and who introduces the changes.

Changes to vision depend on the client, whereas changes to individual goals are developed in the same process as in the theory of skills mentioned above – the middle stages consume most resistance. The most effort should be put on keeping people foTo sum up – we need experienced team members to stay Agile, cused on the same Team Goals. We must be sure that everyone but on the other hand we have still great abilities to develop ex in the team is heard. One of the basic techniques is a daily standperience on our own. Furthermore, up. During the cyclic meeting it is when we do have the experience, very important that all members “Everything should be made as simple then we can take the full respondon’t only generate reports, but as possible, but not one bit simpler.” sibility for the job with great conare involved in other matters as A. Einstein fidence. well. Changes Another important element that is particularly confusing is the permanent readiness for changes. It confuses all team members including even the client. It’s in human nature – a single, simple change isn’t really a problem. But if we have a continuous stream of them, we can reach a critical point. One thing is that a team knows that. But I think not everybody has enough ability to adopt it. In the traditional approach, the project manager can be a filter and take care of members. In Agile, there is more transparency, so all team members are thrown in at the deep end. So the first impression is that we should keep as much as we can without changes. Fix the length of iterations, fix hours of daily stand-ups and other meetings, and

Another technique, as introduced in the Agile Coaching, is to build an ownership for change. It is easier for people to change when it’s their idea. We may make a change as an experiment – try to use, and delay decision for a trial. But we shouldn’t conjure up the feeling that somebody is trying to frame us. Motivation There are so many theories about motivation. Some of them divide motivation into interior and exterior, others into financial and non-financial. But the key in Agile is that every member has real influence on the way how the project’s goals are being reached. Everyone has a chance to be heard, and their own experience may be used. The Agile way makes it worthwhile. One of the most important elements for increasing motivation, in this sense, is the retrospective meeting. We should save as much time as we need and not jump to conclusions too early. Each member of our team should be heard. At the end, the conclusions are transformed into some decisions. Two things are important. First, we shouldn’t make too many changes. Success in introducing a new value may be spoiled twice as much by another value which fails. And the second was mentioned – making the decision by the whole team, but having in mind who is the actual owner of it. Nonetheless we shouldn’t impose our opinions. In another way, when we get the feeling that our participation in a project makes the product significantly better, our motivation

50

www.agilerecord.com

increases. It’s happening gradually – project by project, iteration by iteration. Motivation grows slowly, but is very strong. Much stronger than occasional bonuses. We should rather find ways how to celebrate success of every finished project. It may solidify the whole team’s motivation. Of course responsibility, changeability and motivation are not the only distinct attributes that characterize the Agile way, but they come at the very top when we come from a classic to a modern approach. These attributes are based on other important factors. E.g. learning - which, as mentioned by Lisa Crispin in a comment for the article Be the worst, is the nature of each Agile project. We should seek and create learning opportunities – in order to stay Agile and take the full advantage of it. ■

Bibliography

> About the author Paweł Ziemba is a developer with over five years experience in IT project management. He has a master’s degree in computer science and a university diploma in project management. His main style of leading is coaching – it origins in belief that helping to resolve problems is more important than creating new ones. Currently he works for Espeo Software Ltd., a Polish IT outsourcing company, as a team leader and runs Agile projects.

Scott Berkun; The Art of Project Management Rachel Davies, Liz Sedley; Agile Coaching Dawn Cannan, Lisa Crispin; Agile Records #1, Be the Worst Charles Osgood; A Poem about Responsibility NTL Institute; The learning pyramid Abraham Maslow; Four stages of competence

Subscribe at www.agilerecord.com

www.agilerecord.com

51

© Michelle Albers – Fotolia.com

Financial Analysis Methods and Their Application in SCRUM by Paul Battisson

The agile principle “individuals and actions over processes and tools” has been almost universally interpreted to mean that the use of metrics to analyze a team’s performance is banned. It is usefu,l however, for a project manager or scrum master to have a technically consistent way of indicating how the team is performing and how it is expected to perform in future. In this article I will describe how we can think of our team’s use of resources as a share price and apply existing financial analytical methods to improve our predictive and with it our adaptive and reactive abilities in scrum. Firstly, it should be noted that the word “indicating” and not “measuring” was used. In software development (in particular with agile development) we are continuously concerned with what is going to happen. We write task cards and plan our workload, writing our code not only to function as required now, but also be expandable enough to handle any future requirements that may need to be added. At the same time, of course, we are always worried about whether or not things will be done “in time”. A metric will only ever tell us where we are at a particular time and provide no indication of where we are going. Every day we update a burndown chart that helps us to keep track of where we are (and in a limited sense) where we will be in the future. The burndown chart, however, is only useful over a relatively small defined window (namely 1 sprint), and even a release burndown can show us where we are on track, but it gives us no indication of where things will end up. It effectively provides a simple metric of “on track” or “off track”. By using our burndown chart, we can, however, obtain some interesting numbers that will allow us with relative ease to make some more detailed technical analysis as to where we are likely to go. Firstly, we can retrieve the expected daily burn of the team, simply by dividing the total work to be done by the number of days. We also add to the chart the actual daily burn of the team on a daily basis, revealing to us that day by day there is either a gain or a loss (we will optimistically say that burning as expected is having 0 loss). 52

www.agilerecord.com

If we now draw a graph with the x-axis as days and the y-axis as resources used (”burn”), we can plot an interesting graph. Set the initial point as your expected sprint burn and then add or remove daily the difference in burn from that day, which is analogous to providing a stock/share price chart. We can consider our team as a security, something that we wish to invest in, and its burn rate is the price which will help us decide whether or not that team is performing well or not. Updating this daily over an extended period of time will enable us to build up a good picture of how we are doing in our agile process over a long term view. We can also use this structure to implement some financial analytical methods, that will help to indicate whether we expect our burn rate (share price) to increase or decrease. Before we discuss this any further, we should clarify that we are doing this to provide ourselves with an indication of how the team is doing and where we are expecting our burn to go. The aim is not to measure the performance of the team, but to use this information to indicate how we expect our burn to go and so how our project aims will be achieved. The share price layout we have described allows us a number of advantages over our traditional burndown chart. As mentioned previously, our burndown chart only takes into account up to a sprint’s worth of data (i.e. yesterday’s weather), whereas our stock chart can show us any period of time we wish to see a trend over (as long as we have entered the data for it). We can also now apply any of the existing financial technical methods and trending techniques to our chart with the possibility that this will provide us with early indications of what may happen in our scrum. As an example, I will be discussing the use of the relative strength indicator on my blog (www. paulbattisson.com). I will not detail this here, but will instead detail another process from the financial world, which we can use to manage our scrum process. Every quarter our team will have gone through a number of sprints, after which we can assess how our chart looks. For example, say our team goes through four- week sprints and completes 3 sprints every quarter. After each quarter we will have made a

net gain or loss in overall burn. If we have made a net gain, that is large enough, we can pay out a dividend to the team’s stakeholders (shareholders). We can divide this net gain (say 3 points) by our number of sprints (so 3 points divided by 3 sprints equals 1 point per sprint) and reset our graph to our original level plus this bonus. This is because we have averaged an extra point per sprint and therefore our expected burn should change (i.e. our team’s time is more valuable, so the burn price increases). Similarly, if we have incurred a net loss of 3 points over the quarter of 3 sprints, we can lower our expected burn to the original minus 1 point. This can help us to highlight to the stakeholders that either extra resources in the team are required, or that we are going to need to lower the amount expected. Conclusion This discussion has been based around theoretical work I have done by analyzing the way in which a scrum team runs and the way in which its workload and flow are managed. In my future work I hope to further investigate how we can apply not only financial methods to scrum and other agile processes, but integrate work on game theory and graph theory into this discussion as well. It is my belief that through the application of high level mathematical processes, such as these to agile development, we can provide insight into how to improve our process continually as well as indicate early on how we need to inspect and adapt our methods to achieve maximum efficiency for both the business and the team. ■

> About the author Paul Battisson Paul Battisson is a force. com developer who has done work in a number of languages, but has a particular interest in agile development. He is a Certified Scrum Master and holds a Masters degree in mathematics. It is his view that through correct application of advanced mathematics to the young field of agile software development, we can greatly improve our predictive and adaptive abilities as well as improve the decisions we make. In this article he discusses how we can apply financial analysis to a scrum team.

Subscribe at www.agilerecord.com

www.agilerecord.com

53

© christopher townson – Fotolia.com

Agile is scary! by Ruud Hochstenbach

Since the emergence of Agile one would expect that we should know better now. But in most cases, we firmly stick to a waterfall approach, although we surely know its disadvantages by now. And in many cases, a perfect alternative approach is available, the Agile approach. What strikes me is that many organizations do not want to go change their approach of conducting projects to Agile, despite the obvious benefits. Doing projects according to the traditional waterfall method is apparently a matter of convenience. Everyone knows how it is done: gather extensive and detailed requirements, make a proposal, create a functional and technical design, build, test, deliver, and ready. Implicitly we take the following for granted: too many requirements (research has shown that nearly half of the requirements will never be used), changes in scope during the execution of the project and the effects on price and completion date, sometimes weeks to months later than initially planned. Finally, it turns out all too often that the customer actually had something different in mind than what ultimately is delivered. Why not opt for an alternative? The Agile approach addresses precisely those issues that usually go wrong with the traditional approach (i.e. waterfall). The Agile approach ensures delivery within time and budget and a very high customer satisfaction. Agile can cope with adjustments during the implementation of the project, and the final result closely comes up to the expectations. On top of that, projects using the Agile approach start much faster, because the initial requirements gathering phase is limited to the high level requirements. This high level inventory of the requirements is then processed into a project scope, that in a certain time, called the time-box, will be delivered. The duration of Agile projects is usually much shorter, because the focus is on features that bring real business value only. The implementation takes place in so-called iterations. Each iteration, with a typical duration of two to three weeks, provides a working solution that will be directly used and tested by the business. Each 54

www.agilerecord.com

iteration provides a richer application in terms of functionality. The application is ready when the time-box is consumed. Conditional for the Agile approach is that the business is actively involved in project implementation. This is certainly a critical success factor. By actively participating in the creation, the business can and should directly influence the final project outcome. In this way, change requests during execution can be inserted into the project backlog. Of course, the sacrificed features, which have a lower priority must wait for the next release. This Agile way of working ensures a project outcome that is very close to the business needs. However , this requires from the business that they take responsibility for the final result. Although this approach seems scary to some, Agile projects definitely deliver better quality. Moreover, with the use of an Agile development platform, combined with a supportive project environment, a significant time and cost reduction can be achieved! In sum, Agile is not scary, it is more a matter of fear for the unknown! ■ ing. Ruud Hochstenbach Technical Account/Partner Manager OutSystems

> About the author Ing. Ruud Hochstenbach is Technical Partner Manager at OutSystems Benelux B.V. based in Maarssen, NL. In this role, he is responsible for technical support  to  OutSystems’ customers and partners in the region Northern  Europe. In 1988, Ruud started to work in the ICT business as a support technician. After he joined EDS and CMG, he became  a consultant / team leader. After a short intermezzo, having served as a Service Manager for a healthcare company,  he joined OutSystems. At OutSystems he started out as an Engagement Manager focusing on WEB application development and   was then given the opportunity to increase his knowledge of Agile development. As of September 2009, he has been using his knowledge and skills to support customers and partners of OutSystems.   OutSystems provides the Agile Platform, the first unified solution based on Agile methodologies, to address the full lifecycle of delivering and managing web business applications.  Companies use the Agile Platform to integrate, assemble, deploy and manage applications on-time and on-budget—delivering business value within weeks versus months or years. Since it was first introduced in 2001, the OutSystems Agile Platform has been used in over 4000 IT environments across 138 countries. For additional information, please visit: http://www.outsystems.com or send an email to info@ outsystems.com

IREB Certified Professional for Requirements Engineering - Foundation Level http://training.diazhilterscheid.com/ [email protected]

© iStockphoto.com/ Palto

21.04.10-23.04.10 09.06.10-11.06.10

Berlin Berlin

© Arne Pastoor – Fotolia.com

Mistake-Proofing by Mary & Tom Poppendieck

This article is an excerpt from the forthcoming book: Leading Lean Software Development, By Mary and Tom Poppendieck, published by Addison Wesley, ISBN 0321620704. All software development methodologies have the same objective: Lower the cost of correcting mistakes by discovering them as soon as they are made and correcting them immediately. However, various methodologies differ dramatically in the way they frame this goal. The sequential life cycle frame is to create a complete, detailed design so as to find defects by inspection before any coding happens – because fixing design defects before coding is supposedly less expensive than fixing design defects later on. This theory has a fundamental problem: It separates design from implementation. As we discussed in Chapter 1, separating decision-making from implementation is one of the biggest causes of waste in product development. Good designs come from handson engineering, lots of prototypes, and frequent feedback cycles. The sequential development frame – as typically implemented – has no place for go-to-the-source feedback. But it gets worse. The sequential frame leaves testing until the end of the development process –exactly the wrong time to find defects. Every development frame we know of says that finding and fixing defects late in the development cycle is very expensive – including the sequential frame. And yet, sequential frames have brought about exactly the result they are trying to prevent – they delay testing until long after defects are injected into the code. In fact, people working in sequential frames expect to discover defects during final testing – often a third or more of a software release cycle is reserved to find and fix defects after coding is theoretically “complete.” The irony is that the sequential frames are designed to find and fix defects early, but they invariably end up finding and fixing defects much later than necessary.

56

www.agilerecord.com

Where did we get the idea that finding and fixing defects at the end of the development process is appropriate? Just about every quality program in existence teaches exactly the opposite. The rational approach is to mistake-proof the development process so that the moment a defect in injected into the code, it will be detected immediately and fixed on the spot. We know that it is infinitely easier to find defects that have just been inserted into defect-free code than it is to find defects buried layers deep in defect-ridden code. And yet we save testing until the end of the development process. Why do we do this? Part of the reason why we delay testing is because testing has historically been labor intensive and difficult to perform on an incomplete system. In lean development, we no longer accept this historical frame – we change it. In fact, changing the historical mindset regarding testing is probably the most important characteristic we have seen in successful agile development initiatives. You can put all of the agile project management practices you want in place, but if you ignore the disciplines of test-driven development and continuous integration, your chances of success are greatly diminished. The so-called “engineering disciplines” are not optional, and should not be left to chance. Test-Driven Development The challenge laid down by Edsger Dijkstra in 1972 was: “Those who want really reliable software will discover that they must find means of avoiding the majority of bugs to start with, and as a result the programming process will become cheaper. If you want more effective programmers, you will discover that they should not waste their time debugging – they should not introduce bugs to start with.” 1 So how do you keep from introducing defects in the first place? Dijkstra was clear on that; you have to construct programs “from the point of view of provability.” This means that you start by stating the conditions which must hold if the program is to perform correctly (in other words, start with the tests which must pass).

Testing & Finance 2010 June 7th and 8th, 2010 in Bad Homburg Day 1

Track 1

Track 2

Track 3

9:10

Opening Speech, José Díaz

9:15

Key Note Nitin Bhargava - Barclays Bank, UK

10:15 10:20

Break “Exploratory testing explained“ Alon Linetzki

“Allheilmittel Test-Automatisierung?” Thomas Lingenfelder

“Test Governance structure“ Holger Bonde & Thomas Axen

“TM2010 - Doppelpass zum Erfolg Unsere Erfahrungen mit TPI®, ISTQB®-Standards und deren Anwendung“ Tom Schütz & Dr. Peter Hübner

11:10 11:30

Track 4 - Finance

“Managing a Lean test process; is testing waste?“ Chris Schotanus

“Zweite MaRisk Novelle Auswirkungen auf die Banken IT” Timo Schuhmann

Coffee Break “SCRUM & Testing; Back to the Future“ Erik van Veenendaal

12:25

Key Note “How we test at Microsoft“ BJ Rollison - Microsoft Corp.

13:25

Lunch

“Post Merger Integration einer Hypothekenbank“ Volker Sobieroy & Dr. Jörn Rank & Patrick Esperstedt

14:40

“Random Testing in a Trading System“ Noah Höjeberg

“Testautomatisierung durch MBT“ Florian Prester

“Ausgelagerte Testautomatisierung in einem agilen Softwareprojekt“ Michael Böll

“Aktuelle Änderungen im Aufsichtsrecht“ Dr. Karl Dürselen

15:35

“Governing Testing of Systems of Systems“ Bernard Homès

“It won’t Work, it Won’t Work. It Worked“ Jamie Dobson

“Roles and responsibilities of the tester in a Scrum team“ Danny Kovatch

“Umsetzung neuer aufsichtsrechtlicher Anforderungen mit BAIS Erfahrungsbericht eines BAIS Anwenders” Cornelia Rickels-Moser

16:25 16:40

Coffee Break “Start and Exit Criteria for Testing“ Hans Schaefer

“Knowledge is Power: Do you know where your quality is tonight?” Alexandra Imrie & Hans-Joachim Brede

“BAIS – Optimale Transparenz im Meldewesen“ Christian Karner

“Erfahrungsbericht Testautomation im Landesbanken Rating“ Volker Töwe

“Konzeption – Entwicklung – Testen Überlegungen zu einer zukunftsträchtigen IT-Architektur für das Meldewesen eines Kreditinstituts“ Hosam Elsayed

17:35

Key Note “Wie manage ich die Softwareentwicklung einer großen Rechenzentrale mit rund 200 Projekten?“ Gernot Nolte - Fiducia

18:40

Social Event Dinner + Theatre and Chill out

Day 2 9:10 10:15

Track 1

Track 2

Track 3

“Test Process Improvement: Critical Success Factors“ Graham Bath

“Flexible testing environments in the cloud“ Jonas Hermansson

“TMMi – how are we doing as an industry?“ Geoff Thompson

“Risk based test planning“ Dr. Hartwig Schwier

11:05 11:25

Track 4 - Finance

Key Note “Das Basel II Puzzle: angemessene Eigenkapitalausstattung auf dem Prüfstand“, Prof. Dr. Hermann Schulte-Mattler - Fachhochschule Dortmund “Agile approach for testing in a high risk legacy environment“ Jan Rodenburg

“Verbriefung und Refinanzierung – Plattformstrategie in der Erstellung und Nutzung von Standardsoftware“ Klaus Leitner

Coffee Break “IT-Governance, notwendig oder überdimensioniert? Wie ich mich der Herausforderung einer effizienten IT stellen kann.” Arjan Brands & Oliver Rupnow

“Neue Europäische Bankenaufsicht Auswirkungen auf das deutsche Meldewesen “ Nicolas Jost

“Requirements Management: Effizient, integriert, risikobasiert” Thomas Köppner 12:20

Key Note “Satisfying Test Stakeholders “, Rex Black

13:20 14:30

Lunch “Improving quality and saving costs using static analysis: a case study“ Dr. Mike Bartley

“Testprozess bei der Volkswagen Financial Service AG Keine Theorie sondern gelebte Praxis“ Ingolf Gäbel & Yvonne Sternberg

“Qualität von Anfang an – Anforderungsbasiertes Testen mit Microsoft Visual Studio Test Professional“ Matthias Zieger “Requirements Engineering and Certification” Benjamin Timmermanns

15:20 15:25

16:20

“Ertragsorientiertes Liquiditätsrisikomanagement - Erhöhung der Erträge durch bessere Liquiditätsrisikoanalyse in Zeiten rückläufiger Ergebnisse“ Prof. Dr. Stefan Zeranski

Coffee Break “TMMi - Model Development and practical expierences“ Matthias Rasking

“SOA testen - der inkrementelle fachliche Integrationstest“ Ronald Grindle

“Agilität im Wasserfall - Scrum: nur Ganz oder Gar nicht?” Gernot Glawe

Closing Session, José Díaz

“Pros und Cons zur Einführung einer Leverage-Ratio als Ergänzung der risikosensitiven Basel II-Normen“ Dr. Uwe Gaumert

Then you write the code which makes the conditions come true (makes the tests pass). 2 Dijkstra called this structured programming, but somehow in all of the years of structured programming, his clear, simple concept got lost. Then in the late 1990’s, Kent Beck, Erich Gamma and others revived Dijkstra’s concept and called it Test-Driven Development (TDD). Beck and Gamma created a tool (J-Unit) which has been widely used and adapted to just about every language out there. In 2002 Ward Cunningham expanded the concept of TDD to include functional tests with FIT (Framework for Integrated Testing), a tool which has been extended by many others. 3 Continuous Integration Code does not exist in a vacuum – if it did, software development would be trivial. Code exists in a social setting, and arguably the biggest problem we have in software development is that code isn’t always a good neighbor. Thus integration is often the biggest problem with software, and when it is, leaving integration to the end of development is another example of a defect-injection process. Harlan Mills figured this out 1970 – and he found a solution. First he developed a skeleton of a system, and then he added small programs one at a time and made sure they each worked before adding the next program. He called this top-down programming, but somehow, the underlying idea got lost in translation and few people really understood what Mills meant by the term. Mills’ explained it this way: “My principle criterion for judging whether top down programming was actually used is [the] absence of any difficulty at integration. The proof of the pudding is in the eating! 4 Today our term for what Mills called top-down programming is “continuous integration,” and we use the same criteria for judging its effectiveness: If you have a big bang integration problem – or even a little bang integration problem – you are not effectively doing continuous integration. Where did we get the idea that we should wait until the end of development to put the pieces of the system together? This is just another example of saving up all of our hidden defects and not even trying to find them until the end of development. This violates every known good testing practice in the book – how can we possibly think it makes sense? Why do we do this? The explanation lies partly in the distorted logic of sequential frame, but mainly in the sheer difficulty of testing software after every tiny integration step. First of all, testing has been laborious, and secondly, testing partially done code has an annoying tendency to raise false negatives. In addition, as we raise the level of integration to larger system environments, the testing becomes all the more complex, because we have to test across a much wider span of programs. And finally, it is impossible, really, to find everything that could go wrong, and testing for only a few of the possibilities can be a very long process, and therefore one that cannot be undertaken very frequently. So it should come as no surprise that continuous integration is not easy, and as integration escalates up to higher system levels, 58

www.agilerecord.com

it becomes a huge challenge. But consider the benefits: Many companies set aside 30% or more of each release cycle for a series of integration tests and the associated repairs necessary when the tests uncover defects. On the other hand, the best companies budget no more than 10% of a release cycle for final verification, and they do this with increasingly short release cycles. They can do this because the defects have been uncovered earlier in the development process; if they find defects at final verification, they are surprised and work to find out how to keep that defect from escaping to the final verification step in the future. ■

[1] (Dijkstra, The Humble Programmer, 1972) [2] (Freeman, 1975) p 489 [3] See (Mugridge & Cunningham, 2005) [4] (Mills H. , 1988) p5

Bibliography Dijkstra, E. W. (1972). The Humble Programmer. Communications of the ACM , 15 (10), 859-866. Freeman, P. (1975). Software Systems Principles: A Survey. Science Reserach Aoosciates. Mills, H. (1988). Software Productivity. Dorset House. Mugridge, R., & Cunningham, W. (2005). Fit for Developing Software: Framework for Integrated Tests. Prentice Hall.

> About the authors Mary Poppendieck has been in the Information Technology industry for over thirty years. She has managed software development, supply chain management, manufacturing operations, and new product development. She spearheaded the implementation of a Just-in-Time system in a 3M video tape manufacturing plant and led new product development teams, commercializing products ranging from digital controllers to 3M Light Fiber™. Mary is a popular writer and speaker, and co-author of the book Lean Software Development, which was awarded the Software Development Productivity Award in 2004. A sequel, Implementing Lean Software Development, was published in 2006. A third book, Leading Lean Software Development, will be published in late 2009.

Wir suchen

Senior Consultants IT Management & Quality Services (m/w) www.diazhilterscheid.de

© iStockphoto.com/mariusFM77

Tom Poppendieck has 25 years of experience in computing including eight years of work with object technology. His modelling and mentoring skills are rooted in his experience as a physics professor. His early work was in IT infrastructure, product development, and manufacturing support, and evolved to consulting project assignments in healthcare, logistics, mortgage banking, and travel services. Tom holds a PhD in Physics and has taught physics for ten years. He is co-author of the book Lean Software Development, which was awarded the Software Development Productivity Award in 2004. A sequel, Implementing Lean Software Development, was published in 2006. A third book, Leading Lean Software Development, will be published in late 2009.

Wachsen Sie mit uns!

www.agilerecord.com

59

Testing & Finance 2010 June 7th and 8th, 2010 in Bad Homburg

1 Day

325,- €

2 Days

500,- €

(plus VAT)

(plus VAT)

Yes, I would like to join the social event.

Included in the package: The participation on the exhibition, at the social event and the catering in course of the event. Notice of Cancellation No fee is charged for cancellation up to 60 days prior to the beginning of the event. Up to 15 days prior to the event a payment of 50% becomes due and after this a payment of 100% of the becomes due. An alternative participant can be designated at any time and at no extra cost. Settlement Date Payment becomes due no later than the beginning of the event. Liability Except in the event of premeditation or gross negligence, the course holders and Díaz & Hilterscheid GmbH reject any liability either for themselves or for those they employ. This also particularly includes any damage which may occur as a result of computer viruses. Applicable Law and Place of Jurisdiction Berlin is considered to be the place of jurisdiction for exercising German law in all disputes arising from enrolling for or participating in events by Díaz & Hilterscheid GmbH.

Please fax this form to +49 (0)30 74 76 28 99 or send an e-mail to [email protected]. Participant Company: First Name: Last Name: Street: Post Code: City, State: Country: Phone/Fax: E-mail: Billing Address (if differs from the one above) Company: First Name: Last Name: Street: Post Code: City, State: Country: Phone/Fax: E-mail: Remarks:

Date

Signature, Company Stamp

© ESLINE – Fotolia.com

Can scrum prevent defect ping pong? by Patrice Willemot

The first time I heard the term ‘defect ping pong’, I was thinking of a new discipline for the Olympic Summer Games of London 2012. Unfortunately, this isn’t the truth. But if this should become a reality, then I am sure that I can put a team together and that we will win the gold medal. It will be a hard to get this medal, because we aren’t the only potential winners in the world. If ‘defect ping pong’ isn’t a new discipline for the Olympic games, then what is it? Let’s split the term into two known terms. ‘Defect’ and ‘ping pong’.

• Developer team lead: the lead of the developers that will assign the defect to a certain developer. developer and the tester without being fixed.

Defect?

1. Tester: creates a defect (during test phase or exploratory testing) and assigns it to a developer. 2. Developer: looks at issue, tries to reproduce it, finds that the defect is not reproducible on the development environment, marks the bug as “not reproducible” and assigns it back to tester. 3. Tester: finds the defect still exists on the test environment, assigns the defect back to the developer. 4. Developer: fixes 1 component of the problem, but the defect is still present, assigns it back to the tester stating that the defect has been “fixed”. 5. Tester: tries the defect, finds that it is slightly better, re-activates the defect noting the additional failures.  6. Developer: Fixes the additional components, marks the defect as fixed, assigns it to tester for confirmation. 7. Tester: Confirms the fix by going over the problems that originally caused the defect and closes the defect.

A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. Ping pong? A game resembling tennis, played on a table with small paddles (bats) and a hollow celluloid or plastic ball, where a ball is driven back and forth over a net between the players. This means ‘defect ping pong’: a process where a defect is driven back and forth between the players. Who are the players in the ‘defect ping pong’ process ? For the testing team: • Tester: the person who creates the defect found in the test environment. • Test team lead : the lead of the testing team who is responsible for organizing the test effort by his team. For the developing team: • Developer: the person who will be assigned to fix the defect that has been entered by the tester

At least one candidate from the testing team and from the developing team is needed for playing the ‘defect ping pong’ game. Defect ping pong process at high level How does it work?

While the ideal flow of defects probably should look something like this: Tester: finds a problem –> creates a defect –> assigns the defect to a developer. Developer: reproduces the defect on the developer environment -> fixes the defect –> assigns it back to the tester to be verified. Tester: receives a new build with the fix of the defect -> verifies the fix on the test environment –> closes the defect Everyone yells YeeHa! www.agilerecord.com

61

Defect ping pong process in detail The ‘defect ping pong’ process contains two major bases. A communication problem or hesitation between the tester and developer or vice versa is the first basis of the process. The communication between testing and developing is very hot when discussing defects. The developers often don’t agree that a problem is really a defect and that his work is the cause of the defect. The testers don’t want to go to the developers too often with questions about defects, remarks about defects or rejected defects. So a certain communication hesitation will grow between both players. The second basis in the ‘defect ping pong’ process is the difference between the environments used by the development team and the testing team. The problem is that they are not just different versions, but that there is also a difference between the global settings or global parameters of the application. The difference between the environments can change a reproducible defect on the test environment into a non- reproducible defect on the development environment. Is scrum able to prevent the ‘defect ping pong’ process? The core of the scrum process is the ‘sprint’. In the sprints developers create code for the ‘user stories’ defined in the sprint back log. When a user story is developed, it must be tested (the 62

www.agilerecord.com

result of the test must be OK), and all acceptance criteria of the user story must pass before the story can reach the status ‘done’. Testing of the user story will be done during the sprint on either the development or the test environment. Preferably, the test environment will be used to test the user story. The test environment will contain the latest build of the actual development sprint. Every day during a sprint, a ‘stand up meeting’ is held. In this very short meeting (+/-15 minutes), all scrum team members (product owner, tester, developer, scrum master, …) describe what did yesterday, what will be done today and what the actual problems are.So if we take the two major bases of the ‘defect ping pong’ process and put them in the scrum process ,we will come to the following conclusion. Due to the fact that testers and developers are part of the daily ‘stand up’ meetings, they will tell the other scrum team members if there are defects in the ‘defect ping pong’ process. After the ‘stand up’ meeting, the defects in the ‘defect ping pong’ process can be discussed in detail in order to get the defects out of the process. The delta between the developing and the testing environment is nothing more than some user stories of the actual sprint. This small delta between the environments will encourage that reproducible defects on the test environment cannot easily become non-reproducible defects on the development environment.

Conclusion The ‘defect ping pong’ process cannot exist in a scrum process, if the rules of the scrum process are correctly followed by the scrum team members and the scrum master. What is the reality? Is this the reality? Is defect ping pong just a myth, when we are doing scrum? Not really. If acceptance testing is organized after the sprint, then the ‘defect ping pong’ process can exist. Acceptance testing can be organized, because there are different scrum teams working on the same application, and a separate test team will test the integration of those different sprints. Or it could be that testing in the sprint is done by developers, and a separate team of testers will test the result of the whole build and not user story by user story.

> About the author Patrice Willemot is an experienced test engineer at CTG nv Belgium. He has been in the field of software testing since 2006 testing in different sectors, such as banking, healthcare and automotive. He started out as Winrunner test automation engineer and moved to test lead and test manager. For one year he has been working on an agile project, and he is very interested in how testing and agile development interact with each other.

The first impact of this acceptance test phase is that the testers will test on an environment in version X – 1 (X is the current sprint in development). The global settings will be different because the acceptance test environment will be very close to the production environment. The second impact of this acceptance test phase is in the communication between developers and testers. The developers and the acceptance testers are no longer part of the daily ‘stand up’ meeting, because acceptance testing is done after the sprint. Defects that are in the ‘defect ping pong’ process are not so quickly discussed as during a sprint. The communication hesitation between both teams can and will exist during the acceptance test phase. At the end we can say that scrum will minimize the number of defects that will be ping ponged between developers and testers. This is thanks to the daily stand up meetings, where defects can be quickly discussed, and thanks to testing the user story during the sprint on a test environment, where the delta with the development environment is very small. ■

Subscribe at www.agilerecord.com

www.agilerecord.com

63

© JingleT – photocase

The Agile Bible by Simon Romans

Agile Overview Agile Roles What is Agile? Agile is a set of project management, development and testing methodologies. Agile can replace the old ‘waterfall’ and ‘v-model’ approach to project management with an iterative release often approach. Agile project management, development and testing consist of a number of processes whose main purpose is to ‘make quality better’. Each process acts as a net to trap bugs in the product. Questions need to be asked when considering a process for a project. Is the process really required? Does the process meet all that is required from it? This ‘just enough’ approach can also be used for project documentation. Even with agile processes in place, bugs can still get into the product through the following ways: a) Not having a process in place­ b) Having a process which is incomplete or not relevant c) Not following a process

Product Manager

The product manager oversees the whole project, representing the customer, clarifying requirements and keeping the project running smoothly.

Developer

The developer uses pair programming and ‘test driven development’ techniques to write the code.

Tester

In agile testers work more closely with the developers, exploratory testing stories and creating automated regression tests.

The whole team is responsible for following the processes. For instance if a stand-up meeting has not taken place then any team member can gather the team for one. An agile project timeline contains pre project planning, one or more sprints and a post project retrospective. (As shown in diagram 1.0)

Diagram 1.0

www.agilerecord.com

65

Diagram 2.0

The stand-up meeting is best at the beginning of the day so as to not interrupt the day’s development and testing workflow. During the day developers take a story card from the storyboard, create unit tests and code the story. Developers then update the storyboard by moving the story card along the swim lanes into the ‘Done’ lane. Testing then take the ‘Done’ story card and verify the acceptance criteria have been met, create automated tests to regression test the story card in the next sprint. Once testing is complete the tester updates the storyboard by moving the story card into the ‘Done Done’ swim lane.

Diagram 3.0

2-Week Sprint Timeline There are many processes that can be applied to an agile project. The key is to pick the right ones for your project, and then constantly review them, throwing away ones that don’t work, refining ones that partially work and sticking rigidly to the ones that do. 2-Week Sprint An Agile sprint can be 5 to 30 days long, however a 2-week sprint, (As shown in diagram 2.0), at MIT has been found to be the optimum length. A 2-week sprint contains just over 7 days of development and testing. Day1 - contains a sprint retrospective for the previous sprint, a planning session to pick the story cards for the current sprint, re-estimate the complexity of stories for the current sprint, and tasking out any stories of complexity size 8 and above. Any time left over can be used for development and testing. Days 9 to 10 - contain demo preparation and the demo itself, bug fixing/verification of any bugs found during the demo and release preparation assuming the product manager has given the green light for a release. Sprint Day to Day Tasks During a 2-week sprint there are daily tasks that are carried out from the tail end of day 1 to day 8 of the sprint (diagram 2.0). 66

www.agilerecord.com

Stand-Up Meeting: a meeting held by the team to discuss what they did the day before, what they are going to do today and impediments holding up the project. Story Card: a card that describes an element of functionality to be developed for the product. Acceptance Criteria: are listed on the back of a story card and used to prove the requirements of the story have been met. Storyboard: the focally point for the project. A white board which displays information about the progress of the sprint. Swim Lane: a vertical lane on the storyboard to show progress of a story card. Not Started, In Progress, Done and Done Done are all swim lanes. Pre-Project Planning and Estimating Traceability Matrix The traceability matrix (as shown in diagram 4.0) is a way of keeping track of the story cards against the requirements given by the client. This is to ensure all required functionality has been covered by development and testing. A traceability matrix can contain the requirement ID and description, the story card ID and description, the story card complexity points, the story card state (Not Started, In Progress, Done, Done Done), and the sprint the story card was, or is too be, developed in. Pre-Project Planning In preparation for the project ahead some pre project planning is required. From the client requirements and/or technical documentation the initial story cards need to be created. The story cards need to have their complexity points estimated and prioritized. The projects estimated velocity needs to be worked out and the stories picked for each sprint. A traceability matrix and a project burndown chart should be created. The storyboard needs to be drawn up on a whiteboard. Ideally an overview of product functionality should be shown to the team using diagrams and workflows. This information should be put into a technical specification document.

Diagram 4.0

Sprint Story Selection (diagram 5.0) Once all the story cards have complexity points and the sprint velocity has been determined, the story cards for the each sprint can be selected. At the beginning of the project this would only be a rough guide and stories are likely to be moved around as the project goes on. When selecting stories bare in mind that a usable piece of software needs to be released at the end of each sprint. Think of the project as a cake and each sprint is a slice of cake. Select enough stories to roughly fill the estimated sprint velocity. Once

Diagram 5.0

68

www.agilerecord.com

no more story cards are left you have the number of releases and an estimated project completion date. Project Burndown Once all the story cards have been allocated to their initial sprint the product manager can create the project burndown chart (diagram 6.0). The project burndown chart is used to track the story points remaining against the number of sprints left in the project. Burndown charts will be covered in more depth in another chapter.

Sprint Velocity

be created (see diagram “Complexity Scale”) for each developer and tester. First the product manager reads a story card to the team, including the acceptance criteria. The developers and testers then discuss the work involved. Each developer/tester decides on a complexity score and places the card face down on the table. Once everyone has selected their complexity score the cards are turned face up together. If the complexities differ then the person with the highest and lowest complexity declare their reasons. At this point the team can usually compromise on a complexity score, however if this does not happen then find a previously scored story that is in the middle of the lower and upper scores. Ask the team if this story is more or less complex than the story they are currently estimating.

The next sprint is given a velocity estimate prior to starting. This number is used, during the sprint planning meeting, to pick the number of story cards to be worked on.

If development has a lower complexity estimate and testing has a higher estimate then use the testing estimate. The reason for this will be covered in the agile testing chapter.

Diagram 6.0

Story Cards Complexity Total = Sprint Velocity Estimating Roles To estimate sprint velocities look at previous sprint velocities and take the average. If it is the first sprint of a project then gauge the sprint velocity by looking at other projects. Remember it is only an estimate and will get more accurate as the project progresses.

Product Manager

The PM reads the story cards and their acceptance criteria to the agile team. Answers questions from developers and testers about the story cards and their acceptance criteria. Keeps the momentum of the meeting going.

Developer

The developer asks questions about the story card and acceptance criteria. Estimates the complexity of the development work.

Tester

The tester asks questions about the story card and acceptance criteria. Estimates the complexity of the testing work.

When estimating a sprint velocity take into account team member holidays, planning meetings, stand-ups, etc.

Complexity Points

Story Estimating The whole team should carry out estimating stories as each team member has a role to play (see panel to the right) however only developers and testers actually decide on the complexity of a story. Prior to the planning meeting complexity cards should

The complexity points range from zero to 21 with zero being the least complex and 21 being the most complex. A story card which is given a zero for complexity is classified as functionally complete (Done Done) on the other end of the scale a story card with 21 complexity points is deemed too complex to estimate and should be broken down into smaller stories. The developers should task out stories with a complexity of 8 or 13 as this gives a clear view of the stories functionality. It is usual for questions to arise when tasking out more complex stories. So it is important that the product manager is present during this process.

www.agilerecord.com

69

- the tool for test case design and test data generation

© Pitopia / Klaus-Peter Adler, 2007

www.casemaker.eu

Factor In Everything When estimating the complexity of a story card all work to be carried out should be factored in. For a developer this would include discussing the story, existing knowledge, creating unit tests, creating automated regression tests, coding and effects on other functionality. For the testers this would include, planning the testing, writing the documentation, expanding/verifying the automated regression tests, exploratory testing and raising/verifying bugs.

ification document. A whiteboard could be used and a photo taken of the diagram adding this to the document. 8. Story Cards and tasks are placed in the ‘Not Started;’ swim lane on the storyboard. To pick a priority the following should be considered: • The value delivered by the story to the business • The importance of the functionality to the client • The implementation in relation to other stories Refining Acceptance Criteria Once the stories have been prioritized the Product Manager reads out each story card in turn to the team. The team then discuss the story and if there is any ambiguity in the stories acceptance criteria or there are missing acceptance criteria the story card is updated.

Sprint Planning

Before each sprint begins a sprint planning meeting is held and attended by the whole team. The meeting allows the team to fine AGILE TEAM STATEMENT tune the work/estimating carried We commit to deliver working and fully out during the pre-planning stage. Re-selecting Stories

tested code that demonstrates some thing of value to the end user.

The product manager reviews the stories selected for the sprint and shuffles the stories around the future sprints based on any new information. E.g. the client didn’t require a piece of functionality till a later sprint but now requires it urgently, or a piece of functionality is found to give more value to the business and should be moved to the current sprint. Prioritizing Stories The product manager then prioritizes the current sprint stories. Giving each story a traffic light priority of red/yellow/green. Meeting Activity Order

Re-estimating Stories The team then decides if the preestimated story complexity still applies or if they should re-estimate the stories complexity.

Re-estimating the Sprint Velocity During the sprint planning meeting the velocity of the sprint should be re-assessed. This can be done by looking at the previous sprint(s) velocities and working out an average. Holidays, planning meetings, stand-ups, etc, need to be taken into account. At first the sprint velocity may be volatile but as the project goes on estimating a sprint velocity will get more accurate. If it doesn’t then the team needs to investigate the reason(s) why. • Perhaps the acceptance criteria were not clear enough to give the story card the correct complexity estimate • Perhaps other tasks keep holding up the development of the story cards • Perhaps the story cards are too big and need to be broke down into small ones Tasking Stories Story cards with a complexity of 8 or 13 should be tasked out on post-it® notes. Use the same color post-it for all tasks so it is visually easier to see tasks on the storyboard.

1. Product Manager re-orders the stories cards between sprints, if required. 2. Product Manager prioritizes the current sprints story cards. 3. The Product Manager reads out the Story cards for the current sprint. 4. The team rewrites ambiguous acceptance criteria and adds missing acceptance criteria, if required. 5. If needed the developers and testers re-estimate the current sprints story cards. 6. The developers and testers task out story cards with a complexity of 8 or 13. 7. The developers and testers create a diagram of the flow of functionality and add this to the pre project planning technical spec-

It is a good idea to draw up a diagram of functionality for the sprint and/or story card. Other developers and testers can use the diagrams to understand the functionality of the product. • Extra Tasks During a Sprint During a sprint it is not unusual to have extra tasks from the product manager or unforeseen tasks created by story cards. Write these tasks on post-its® and stick them to the storyboard.The tasks can then be accounted for in the sprint planning meeting. Extra tasks do not need complexity points but it is agood idea to put the hours spent on the extra www.agilerecord.com

71

• Dealing with Impediments Impediments can crop up from time to time during a sprint. Any impediments encountered during the day should be written on a post-it® and stuck in the impediment section of the storyboard. The impediment should then be talked about at thedaily stand-up meeting. It’s the product manager’s job to help identify, track, assess and remove impediments. If the impediment has not been removed by the end of the sprint then talk about the impediment in the sprint retrospective and decide on a plan of action. Performance Sprint(s)

Tasking Stories

task. Accounting for time spent working on tasks outside of the sprints story cards makes it easier to estimate the next sprints velocity.

Product performance is an important factor and can have a detrimental effect on user experience if not up to speed. Performance can affect the front end (GUI’s, mobile WAP applications) and the back end (database, daemons). Ways to integrate performance improvements: • Incorporate a performance sprint into the project • Build in performance from the start • A combination of the two

• Adding Stories to a Sprint The agile testing and development processes allow for changBuild performance in: By adding performance acceptance criteing requirements. New story ria to the story cards the developcards can be created for new ers build performance tools and requirements, a change in alverify the performance of prod“Product quality is not just the ultimate aim. Timeliness, product features, complexity ready developed functionality uct features as the project goes and cost also need to be considered.” and bugs (bug story cards are along. Performance sprint: Plan explained in another chapter). for a 2-week performance sprint These story cards should be when estimating the release treated in the same way as the original story cards. Add acdeadline. During a performance sprint the developers create perceptance criteria, give the story a complexity and select the formance tools and run performance checks against the clients initial sprint for the story to be worked on. Track new story performance KPI’s. Testers create a performance document and cards on the project burndown and traceability matrix. add the results to it. This is to keep a benchmark record if future upgrades are required to the product. • Prioritizing the Backlog The product manager should prioritize the backlog so that Creating a Global Story Card (Covered in another chapter) can if the current sprints stories have all been completed ‘Done help clarify and remind the team what performance targets are Done’ then stories can been taken from the backlog in order relevant to the project. of priority. Place a few high priority backlog stories onto the storyboards backlog section. Unfinished Stories • High Priority Bugs High priority bugs (P1/P2) found during a sprint should be fixed and verified by the end of the sprint. Write the bug on a colored post-it® and stick it in the ‘Not Started’ swim lane. A high priority bug would be deemed a bug that stops a story card from being approved by testing, a bug that stops testing from continuing, and a bug which stops a user from using some functionality of the system. High priority bugs found in the sprint demo can be fixed during day 9/10 of the 2-week sprint.

72

www.agilerecord.com

Sometimes story cards are not completed, or part completed, during the sprint. The complexity points for these cards should not be counted for the current sprint. The part completed functionality should not be shown to the client in the sprint demo. The story should be moved to the next or a future sprint and the full complexity points added to that sprints velocity.

Agile KPI’s

> About the author

Key Performance Indicators KPI’s are quantifiable indicators to measure the success of an agile project. They can be used to gauge if agile processes are effective, to name a few, in capturing bugs, allowing accurate estimation of release dates and customer satisfaction of the end product. A KPI should contain a heading, goal, purpose and description. Agile KPI’s can be: • Quantitative Indicators – presented as a number • Actionable Indicators – agile process recommendations • Financial Indicators – project operating costs

Simon Romans has worked in testing for 14 years, starting his career as a games tester at Electronic Arts and most recently working as Test Manager for Mobile Interactive Technology. At MIT Simon has set-up the testing department and was responsible for the company's transition to Agile Development and Testing Methodologies. Simon says "When it comes to testing it can't rain all the time!".

Example KPI’s KPI Heading: Product Quality KPI Goal: Reduce the number of bugs in the live environment without negatively affecting timeliness and product features. KPI One Description: No. of Outstanding Bugs vs No. of Fixed Bugs – Recorded prior to release – tracks the amount and severity of bugs left in a product prior to a live release. Measures the quality of the product prior to a live release. (Business view) KPI Purpose: Using the tracked bug statistics we can see if the agile processes allow time to fix and verify bugs in the product. KPI Two Description: No. of Bugs after Release to Live – Recorded after release (Ongoing) - tracks the amount, date found and severity of bugs found after a release is made to live. Measures the quality of the product in terms of customer satisfaction. (Customer view) KPI Purpose: Using the tracked bug statistics we can see if the agile processes are capturing the bugs in the product prior to a live release. We can also see if the amount of bugs found in the live release reduces over time.

Subscribe at www.agilerecord.com

www.agilerecord.com

73

Masthead EDITOR Díaz & Hilterscheid Unternehmensberatung GmbH Kurfürstendamm 179 10707 Berlin, Germany Phone: +49 (0)30 74 76 28-0

Fax: +49 (0)30 74 76 28-99

E-Mail: [email protected]

Díaz & Hilterscheid is a member of “Verband der Zeitschriftenverleger Berlin-Brandenburg e.V.” EDITORIAL José Díaz



LAYOUT & DESIGN Frenkelson Werbeagentur WEBSITE www.agilerecord.com ARTICLES & AUTHORS [email protected] ADVERTISEMENTS [email protected] PRICE online version: free of charge

In all publications Díaz & Hilterscheid Unternehmensberatung GmbH makes every effort to respect the copyright of graphics and texts used, to make use of its own graphics and texts and to utilise public domain graphics and texts. All brands and trademarks mentioned, where applicable, registered by third-parties are subject without restriction to the provisions of ruling labelling legislation and the rights of ownership of the registered owners. The mere mention of a trademark in no way allows the conclusion to be drawn that it is not protected by the rights of third parties. The copyright for published material created by Díaz & Hilterscheid Unternehmensberatung GmbH remains the author’s property. The duplication or use of such graphics or texts in other electronic or printed media is not permitted without the express consent of Díaz & Hilterscheid Unternehmensberatung GmbH. The opinions expressed within the articles and contents herein do not necessarily express those of the publisher. Only the authors are responsible for the content of their articles. No material in this publication may be reproduced in any form without permission. Reprints of individual articles available.

Index Of Advertisers

74

gebrauchtwagen.de

2

Díaz & Hilterscheid GmbH

5, 26, 27, 28, 29, 41, 48, 55, 57, 59, 60, 76

Agile Records

9

bitzen.net

64, 67

Cabildo de Gran Canaria

75

CaseMaker

70

www.agilerecord.com

Training with a View

also onsite training worldwide in German, English, Spanish, French at http://training.diazhilterscheid.com/ [email protected] “A casual lecture style by Mr. Lieblang, and dry, incisive comments in-between. My attention was correspondingly high. With this preparation the exam was easy.” Mirko Gossler, T-Systems Multimedia Solutions GmbH “Thanks for the entertaining introduction to a complex topic and the thorough preparation for the certification. Who would have thought that ravens and cockroaches can be so important in software testing” Gerlinde Suling, Siemens AG

Kurfürstendamm, Berlin © Katrin Schülke

- subject to modifications 19.04.10-21.04.10 21.04.10-23.04.10 03.05.10-07.05.10 03.05.10-07.05.10 03.05.10-05.05.10 10.05.10-12.05.10 17.05.10-21.05.10 07.06.10-09.06.10 09.06.10-11.06.10 14.06.10-18.06.10 21.06.10-24.06.10

Certified Tester Foundation Level - Kompaktkurs Certified Professional for Requirements Engineering - Foundation Level Certified Tester Advanced Level - TESTMANAGER Certified Tester - TECHNICAL TEST ANALYST Certified Tester Foundation Level - Kompaktkurs Certified Tester Foundation Level - Kompaktkurs Certified Tester Advanced Level - TEST ANALYST Certified Tester Foundation Level - Kompaktkurs Certified Professional for Requirements Engineering - Foundation Level Certified Tester Advanced Level - TESTMANAGER Certified Tester Foundation Level

Hamburg Berlin Frankfurt am Main Berlin München Berlin Berlin Hannover Berlin Berlin Dresden