Agile and Iterative Development - Nominet Trust

5 downloads 229 Views 137KB Size Report
Agile and. Iterative. Development. Evaluation Insights. Practical lessons from social tech ventures ... mixed-method and
Evaluation Insights Practical lessons from social tech ventures

Agile and Iterative Development

Evaluation Insights Agile and Iterative Development

2

RESEARCH APPROACH AND METHODOLOGY Nominet Trust forms an active and close relationship with all of its funded partners in the evaluation of their ventures. This enables Nominet Trust to assess each project’s social impact and identify the key insights gained throughout its early product development stages.

At the beginning of our funding, each project is required to produce a detailed evaluation plan that includes targeted outcomes, measures, reporting dates and evaluation deliverables. The plan also identifies key project assumptions (relating to the project’s theory of change), hypothesis and research questions. An active relationship is maintained between the funded organisation and the Trust’s Development Research team, with regular evaluation updates provided through the lifetime of the funding and culminating in a final end of project Evaluation Report. The findings in this paper are based on multiple reports submitted by a number of projects throughout their funding period. Data was collected, entered and analysed using mixed-method and qualitative research software, Nvivo, utilising an emerging coding framework.

The project-related information sources that this paper draws on includes:

Evaluation plans



Quarterly evaluation updates



End of project evaluation forms



Project blogs



External evaluation reports



Presentations



Meetings between Nominet Trust evaluation staff and funder partner project leads

Evaluation Insights Agile and Iterative Development

3

INTRODUCTION Nominet Trust aims to seek out, develop and support ventures that use digital technology for social good. To do this effectively we look to learn from all of our activities to better understand how this technology can transform the ways in which we address profound social challenges.

Evaluation Insights openly shares the insights gained and lessons learnt by a range of social tech projects funded by Nominet Trust and offers key recommendations for other projects looking to embark on a similar journey. In sharing the experience of what works (and doesn’t work), testing theories of change, identifying critical success factors, good practice, and validated learning, Nominet Trust aims to increase understanding and effective practice in the ‘social tech’ sector.

The first publication in this series, Agile and Iterative Development highlights the successes, failures and lessons learnt by a number of Nominet Trust funded projects in developing social tech solutions. It looks at how agile and iterative approaches, typically used in the technology start-up sector, differ from traditional project management and how they can be applied to developing socially focused products and services.

Evaluation Insights Agile and Iterative Development

4

WHAT IS ‘AGILE’ AND ‘ITERATIVE’ DEVELOPMENT? If terms such as ‘agile’, ‘iteration’, ‘minimum viable product’ and ‘paper prototype’ are familiar to you, then there is a good chance that you have been involved in a technology start up. The last 20 years have seen rapid and constant advances in technology, with seemingly limitless design features and configurations, the democratisation of production and the emergence of new and untested markets.

As a result, traditional approaches to project development and business planning aren’t always suitable and new approaches could and should be considered. This seems to hold especially true in the case of early stage startup ventures, where the myriad possibilities, technical challenges and unknown markets, render conventional approaches to long-term strategic planning inappropriate.

It [agile development] “promotes adaptive planning,

evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change.1



1 http://en.wikipedia.org/wiki/ agile_software_development

Agile development - a definition Agile development is an approach developed to meet the challenges and opportunities of software development based on principles of iterative and incremental development. The term ‘agile development’ covers a range of methods and practices including: scrums, lean development, user stories, sprints, timeboxing, a/b testing, etc. Underpinning these particular practices and the agile approach as a whole is the conviction that the best way to develop new solutions in complex situations is through small repeated cycles of development (iterations) involving frequent user feedback and continuous incremental improvements. Progress is ‘little and often’, learning and exploration, rather than the precise execution of detailed plans based on preconceived ideas.

Evaluation Insights Agile and Iterative Development

5

WHY TAKE THE AGILE APPROACH? Agile and iterative development is intended to be a flexible and adaptive approach. It allows quick design changes in response to user behaviour, feedback and new or changing circumstances such as stakeholder suggestions, new technology and market opportunities. While largely familiar to organisations with a technology background, it is a relatively new approach to third sector organisations and social enterprises.

Although the ventures Nominet Trust invests in are primarily concerned with achieving social outcomes, the way of achieving them is through the development of new technology-based products and services.

they have historically used in project management are not so effective in early-stage technology development. Conversely, many of the approaches developed by tech start-ups have helped create effective new solutions to the social challenges they seek to address.

Feedback to Nominet Trust from its funded partners indicates that many of the approaches

Projects that have found agile and iterative development effective A number of ventures and projects supported by Nominet Trust have identified agile and iterative development as a key factor in their success and their learning has helped to shape the findings of this publication. These projects include: Apps for Good, CDI Europe www.appsforgood.org Historypin, We Are What We Do www.historypin.com WeFarm, Cafedirect Producers’ Foundation wefarm.info

Patchwork, FutureGov patchworkhq.com Georgie, Screenreader www.georgiephone.com Mind of My Own, Sixteen25 mindofmyown.org.uk Innovation Labs, Comic Relief, Right Here, Paul Hamlyn Foundation, Mental Health Foundation www.innovationlabs.org.uk Sidekick School, Sidekick Studios sidekickschool.org

Evaluation Insights Agile and Iterative Development

6

CHARACTERISTICS OF AN EFFECTIVE AGILE AND ITERATIVE APPROACH An agile and iterative development approach will take on different forms for different projects. However there are a number of principal characteristics that are common to most approaches. These include but are not limited to:



A continuous and quick cycle of planning, testing, feedback and refinement.



Close cooperation and very frequent communication (face-to-face if possible) between developer and project team.



An emphasis on delivering a working product as quickly as possible in order to facilitate hands-on/live testing with potential users (often referred to as alpha and beta testing).

working prototype “is aAvital deliverable. We

knew this before we started, but we’ve had it very clearly validated - having a working, usable bit of software transforms conversations from hypothetical / what if / scenario stuff into action.



‘Stuff we’ve learnt in three weeks of hacking at Comic Relief’, Sidekick School, project blog from Sidekick Studios, October 2012.

Some early evidence of unique “value, represented by a working prototype, will generate the interest and investment needed to add more layers to the user experience and allow projects to build on solid, genuinely worthwhile foundations.



Historypin project evaluation from We Are What We Do, January 2013.



Focus on core product functionality and the specific needs and interests of the intended users (rather than the interests of the developers). Needs are often captured in ‘user stories’ that typically record who the intended users are, what function they want from the product, why they want it, and where and how they want to use it. User stories also help identify the various ways different users may interact with your product.



Products which are co-designed in collaboration with key stakeholders, including the ultimate intended users, from the outset and throughout the design process.

Evaluation Insights Agile and Iterative Development





7

A minimum viable product built on what you need and not spending vital resources developing features that the potential user may not want. A product’s feature sets should grow and shift based on actual use shown through testing. Development activity and goals organised around a series of short ‘time-boxed’ sprints (typically two weeks) with regular weekly or daily updates on activity and progress towards goals. This applies to both the development team and the developer’s relationship with their client.

[we hired] ...a development “agency that truly practiced agile

methodology – with weekly sprints instead of development cycles of four to six weeks as per the practice of our previous agency. Issues that were previously not picked up in five months of development were picked up much sooner as a result of using agile methodology.



‘Building a Platform - What I Learnt the Hard Way’, Apps for Good project blog, December 2012.

Steve Blank’s Customer Development Process

Customer Discovery

Customer Validation

The characteristics captured in Steve Blank’s lean start-up model is as applicable to social organisations as it is to commercial. This model highlights the importance of an agile and iterative development approach. In both social and commercial sectors, there is a risk of investing considerable resources into a product or service which nobody really wants or will use. Discovering

Customer Creation

Company Building

whether there is a demand for your product or service, and learning about your customers interests and behaviours, is a vital and continuous part of digital product development and organisational growth. For more information see: http://steveblank.com/2012/ 09/06/the-lean-launchpad-online

Evaluation Insights Agile and Iterative Development

8

KEY FINDINGS FROM NOMINET TRUST FUNDED PROJECT EVALUATIONS The following findings are drawn from the experiences of funded organisations in developing digital products and implementing agile and iterative approaches. It is hoped that this collective learning will help inform and increase the likely success of other organisations developing digital products and services in order to effect positive social change.

The four key learning points that emerged are:

Learn from a commercial approach Although agile and iterative approaches were initially developed in a commercial setting, the technical challenges they were designed to address are equally relevant to organisations (both new and established) seeking to develop new and untested technical solutions that address social issues. Agile and iterative approaches have been successfully adopted by many social tech organisations funded by Nominet Trust to produce new services and applications that successfully meet the needs of their intended beneficiaries.

Build a working prototype as soon as possible It is important to focus on producing a working prototype as soon as possible in order to facilitate stakeholder testing and ongoing iterative development. A minimum viable product represents the simplest (and cheapest) product you can initially build in order to start the processes of user and market testing. There are a number of ways to create minimum viable products, from using paper visuals to model the technology’s behaviour to creating lightweight websites.

DocReady Paper Prototype - FutureGov Workshop Paper-protoyping enables projects to test and design the functions and user experience of a new app/platform/ website before any programming takes place. There are a number of advantages to this approach, including the decreased likelihood of expending limited resources on a product or functions that do not meet the needs and behaviours of the intended users. DocReady paper prototype generated at FutureGov workshop: www.flickr.com/photos/ futuregov/9315855290/in/ photostream

Evaluation Insights Agile and Iterative Development

When building and testing a prototype it is important to:





Consider the core functionality of the product. It can be tempting for developers and project teams to keep adding new functions and design elements. However, the clear message from successful projects is to maintain focus on the core functionality of a new product, which in turn is clearly focused on the needs of the intended users. Any additional elements can always be added later. Begin user testing new products with potential users and other key stakeholders as soon as possible. Key stakeholders are important as ‘users’ and not only include end beneficiaries, but anyone who might be involved in delivering new product or service e.g. teachers, carers, key workers, etc. This testing can be done using paper prototypes and wireframes as well as early digital prototypes, but the aim is to minimise unnecessary build costs and time by responding to real use. Test the potential market at an early stage to develop a minimum viable product. Testing the potential demand for a product can also have the advantage of generating a community of enthusiastic early adopters.

9

Never build automated “features for human behaviour

you haven’t tested manually or understood completely. Codifying the assumed behaviour on the platform meant that when users didn’t behave in the way we had planned for, we had spent money and time on useless features.



‘Building a Platform - What I Learnt the Hard Way’ Apps for Good project blog, December 2012.

The first version of “Historypin should have been

built with 75% open source technology, with only around 25% representing unique features and therefore custom code. If we had combined iterative prototyping and harnessing open source tools we would be in a considerably stronger position than we are now.



Historypin project evaluation from We Are What We Do, January 2013.

Evaluation Insights Agile and Iterative Development





A prototype is not a finished product. It is important to remind early users and testers that they are still using an incomplete version which may still have bugs, but these can all be addressed with the right feedback. Consider using open source products where possible to help reduce time and development costs. There are many platforms that can be customised to meet your needs. An example of this is Twitter Bootstrap2, which can help speed up the process of building an initial prototype.

platform uses a “greatThedealWeFarm of customised coding

to function smoothly. However it also takes advantage of existing software (open source where possible) to perform certain aspects of operations, such as sending out real world SMS messages.

10

Have the right resources in place To take advantage of an agile and iterative development approach, there are certain resources and capacities that need to be in place. The resources needed will differ depending on each project’s specific budget, needs and objectives, and may include, but are not limited too:

A good product developer who has experience and evidence of successful ‘agile’ product development.



Capacity to maintain regular and honest interactions between the project team, developer and stakeholders.



In-house development capacity to facilitate responsive iterative development.



A ‘champion’ or product owner in the lead organisation who has clear ownership of, and responsibility for, the product’s current technical and future development.



Easy access to, and ideally a trusting relationship with, key stakeholders including potential users who will be vital in helping test and co-design the new product on a regular basis. This relationship can be direct or through key project partners.



WeFarm project evaluation from Cafedirect Producers’ Foundation, July 2013.

2 http://en.wikipedia.org/wiki/Bootstrap _(front-end_framework)

Evaluation Insights Agile and Iterative Development

an internal developer] “in the[withteam, we were then not

only able to launch the product but also able to collect data on what did and didn’t work on the platform, and implement small improvements in daily revision cycles.



‘Building a Platform - What I Learnt the Hard Way’ Apps for Good project blog, May 2013.

Understand the risks, costs and challenges Agile and iterative approaches are not without their particular challenges and considerations. There are a number of obstacles, as identified by Nominet Trust funded projects that should be considered when adapting this approach:

Developing in-house technical skills capacity comes at a cost, which can increase if key team members move on.



Getting stakeholder input can take time, especially if said stakeholders face barriers to participation, for example time, health, or mobility.



Obtaining user stories is a key part of the agile development process, but there are challenges associated with both collecting and using them. For example, how authentic and representative they are, and how they can be sorted, prioritised and tested.

11



Completing documentation is something that agile approaches in particular try to simplify as much as possible. However, this can generate issues around record keeping, sharing, transparency and accountability. It is advisable to agree at the start of the design process what documentation respective stakeholders require (and when). It is also important to ensure that the documentation captures the iterative and responsiveness nature of the design process.

a trainee “coderWetohadhelpintroduced work on the

software - sadly he moved to Cardiff and took a job there - so the learning that we had invested in him is now only available to us part time. We can manage with the freelancers we use but a key issue remains access to the right coders and the ability to iterate quickly and cost effectively.



Impact Assessment App project monitoring form from Podnosh, April 2013.

Evaluation Insights Agile and Iterative Development

12

IS AN AGILE AND ITERATIVE APPROACH APPROPRIATE? When deciding whether an agile and iterative approach is a suitable development method for a product you should first consider what stage of development that the product is at. This development approach is generally best suited to the earlier stages of product design, when core aspects of function and design are being determined.

Agile and iterative development is very much focused on innovation and the development of new, sometimes quite radical, ideas. The constant integration of feedback and stakeholder involvement means that it is an approach well suited to exploring complex challenges to which there may be a range of potential solutions. Agile and iterative approaches work best when ‘time-boxed’ i.e. when there are fixed deadlines, and a certain urgency around delivering results, such as a minimum viable product. That said, it is important to adopt a management approach that complements the project. There may be situations when an agile and iterative approach is NOT appropriate, for example:

If the product in development has a very clear specification.



If there are very clear and fixed expectations of how and when the product will be produced.



If it is not possible to engage stakeholders, including intended users, in the design process.



If the product can only be delivered in a single ‘chunk’ or iteration.



When incremental improvements are being made to a product that has already been through a number of design cycles and the core of which would remain largely unchanged.



When there is a need to rigorously assess the social impact of a product (for example, through trial), perhaps with a view to accessing significant additional investment.



When a product or service is ‘finished’ and the emphasis moves to promotion or conducting forms of rigorous evaluation that require consistency.

Evaluation Insights Agile and Iterative Development

13

SUMMARY AND CONCLUSION The approaches to agile and iterative tech development described in this publication are still relatively new in the commercial sector, and are only recently being adopted by socially focused ventures. As such there is still a great deal to test, learn and share.

Through Nominet Trust’s work in this area and the creative ingenuity of the organisations it funds, there is an emerging body of learning that other social tech ventures can learn from. In summary:

Agile and iterative development approaches have been successfully used by socially focused digital tech ventures to effectively utilise limited resources in the development of new products and services that are appropriate to the needs, interests and behaviours of their intended users.



Prototypes and minimum viable products are useful and cost-effective tools for both user testing and market testing of new digital products and services.



Specific resources are required to properly implement an iterative approach to digital product development, including in-house staff capacity, responsive and knowledgeable developers, and strong project leadership.



A number of risks, costs and challenges do exist. Iterative product development is not always easy, and can require a significant change in the ways that

organisations and developers are used to operating.

Agile and iterative approaches are not appropriate for all projects. It is important to apply them in the correct context, and to recognise when a product has evolved to the point that a more predetermined and structured approach is required.

New and emerging digital technology has the potential to affect significant positive social change. However, it does not just represent an Zadd-on’ to existing products and services, but a whole new approach to engaging with users and addressing social challenges. The flexible nature of digital technology requires responsive and adaptable approaches to project management and business planning. Following an iterative approach will not guarantee your project will be a success, nor will it create a market for your product if one does not exist. However, it will help ensure that limited time and resources are allocated effectively on products and services that have value for intended users and markets. It will also help projects to respond more quickly to developments and discoveries in what can be a very complex, dynamic and changing environment.

Evaluation Insights Agile and Iterative Development

14

CHECKLIST FOR DEVELOPING AN EARLY STAGE DIGITAL TECH PROJECT Below is a ‘checklist’ of questions that an organisation developing a social tech venture should consider. They reflect the experiences, frustrations, success and advice of a number of Nominet Trust funded partners and therefore may be helpful to any organisation operating in this context.

Checklist How much confidence do you have in your current spec/identified needs?

Have you identified your core users and do you have user stories?

Have you identified the ‘core functionality’ of your product? How does it address the main need you have identified?

Have you tested your idea with intended/targeted users?

Have you considered whether adopting an agile and iterative approach to developing your product would be appropriate? If appointing an external developer, what experience do they have and what is their approach? Are they committed to open and regular communication, prototyping and regular stakeholder testing?

If you have any feedback on the areas covered in this publication then please send your comments and suggestions to Nominet Trust at:

Do you have access to key stakeholders, including potential end users? Do you have sufficient ‘in-house’ capacity to undertake iterative development? How soon will you be able to develop a working prototype? Have you allowed sufficient time and capacity to undertake regular testing, etc?

[email protected]

Evaluation Insights Agile and Iterative Development

15

ADDITIONAL RESOURCES Nominet Trust

‘How to Measure a Successful Tech for Good Venture’ blog, Nominet Trust, September 2013. http://www.nominettrust.org.uk/knowledgecentre/blogs/how-to-measure-successfultech-good-venture



Further reading and videos

Nominet Trust funded projects



‘Historypin Evaluation’, Historypin project blog from We Are What We Do, May 2013. http://www.nominettrust.org.uk/knowledgecentre/evaluating-your-project/historypinevaluation ‘Building a platform: What I learned (the hard way)’, project blog from Apps For Good, CDI Europe, December 2012. http://www.nominettrust.org.uk/what-wesupport/blogs/building-platform-what-ilearned-hard-way



‘Paper Prototypes At The Ready’, Mind Of My Own project blog from Sixteen25, April 2013. http://www.nominettrust.org.uk/what-wesupport/blogs/paper-prototypes-ready



‘In a lean environment design matters more than ever’, Sidekick School project blog from Sidekick Studios, March 2013. http://www.sidekickstudios.net/ blog/2013/03/in-a-lean-environmentdesign-matters-more-than-ever



‘Stuff we have learnt in three weeks of hacking at Comic Relief’, Sidekick School project blog from Sidekick Studios, October 2012. http://www.sidekickstudios.net/ blog/2012/10/stuff-weve-learnt-in-threeweeks-of-hacking-at-comic-relief

‘Agile and Prince2: Confessions of an accredited practitioner’, blog from Futuregov, May 2013. http://wearefuturegov.com/2013/05/agileand-prince2-confessions-of-an-accreditedpractitioner



The Agile Manifesto http://agilemanifesto.org/



The Lean Start-Up http://theleanstartup.com/



‘How to Build a Start-Up’, video from Steve Blank https://www.udacity.com/course/ep245



‘Execution = Learning’, conference presentation from Tristan Kromer http://www.youtube.com/ watch?feature=player_ embedded&v=ADgc0jAWTA4&noredirect=1#!



‘The Curve: The Future of Business’, recorded interview with Nicholas Lovell http://www.thersa.org/events/audio-andpast-events/2013/the-curve-the-future-ofbusiness



‘Have a Lean Startup with User Experience’, slideshow from Margareta Löfvenholm http://www.slideshare.net/ MargaretaLfvenholm/have-a-lean-startup



Project Management and Documentation



Basecamp https://basecamp.com/ - An online project management and collaboration tool where you can manage your tasks, files, messages, schedules, and milestones Pivotal Tracker http://www.pivotaltracker.com/ - A user-friendly, agile project management tool

Digital technology offers a phenomenal opportunity to stimulate new forms of collaboration, to mobilise new communities of interest, and to unleash the imagination of millions of users in addressing specific local and global challenges. Nominet Trust believes in harnessing the power of digital technology to improve lives and communities. As one of the UK’s leading social tech funders, we bring together, invest in and support people committed to using digital technology to create social and economic value. Through our on-going research programme we identify specific areas of need and channel funding towards initiatives designed to make a significant difference to people’s lives. Since our inception in September 2008, Nominet Trust has invested more than £19 million in hundreds of ventures, providing business support as well as financial investment, seeking to use digital technology to tackle complex social problems. To find out more about our work or how you can apply for funding, please visit: www.nominettrust.org.uk @nominettrust

Nominet Trust Minerva House Edmund Halley Road Oxford Science Park Oxford OX4 4DQ t +44 (0)1865 334 000 f +44 (0)1865 332 314 [email protected] www.nominettrust.org.uk