Informing the Design of New Mobile Development Methods and Tools

Informing the Design of New Mobile Development Methods and Tools Abstract Michael Nebeling, Christoph Zimmerli and Moira C. Norrie Institute of Information Systems, ETH Zurich CH-8092 Zurich, Switzerland {nebeling,zimmerli,norrie}@inf.ethz.ch

Given the proliferation of new mobile devices and different technologies, it is becoming increasingly difficult to develop for mobile settings. To gain a better understanding of the engineering problem and how developers currently work, we conducted a 14-week experiment with four developers. Using a diary study, we analysed commonalities and differences between various native mobile and webbased approaches including the different kinds of tasks and the effort expended on them. We discuss implications for design and describe our current work towards better supporting multi-platform development based on our findings.

Author Keywords Multi-device engineering; development experience.

ACM Classification Keywords H.5.2 [User Interfaces]: Screen design; Input devices and strategies

Introduction Copyright is held by the author/owner(s). CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris, France. ACM 978-1-4503-1952-2/13/04.

The current proliferation of mobile devices requires a redesign of many applications traditionally consumed through desktop-like interfaces to access them in a mobile setting. Given the diversity of device characteristics, it is a major challenge for application developers to create user inter-

faces (UIs) for the current range of mobile technologies. The range of mobile platforms, not only requires a wide skill set and experience with different programming languages and software development kits, but also results in a duplication of design effort and increased maintenance costs. To show the dimension of this problem, The New York Times currently offers six separate applications for mobile devices , but still does not cover the whole range of existing devices. A recent debate on native mobile vs. web-based application development [2] is of particular interest to this work. Proponents of the first approach see benefits in terms of performance and interface design, while advocates of the latter argue for the reduced implementation and maintenance costs of a device-independent platform. Based on a study of four developers and current approaches, this pa- per aims to build a better understanding of the design and implementation effort when migrating desktop applications to mobile platforms. Our analysis includes a diary study to identify the main tasks and the effort expended on them for current mobile platforms as well as determining commonalities and differences in the approach and collecting feedback on the development experience. Our specific observations concern framework usability and productivity in terms of both technical and subjective criteria as a basis for identifying factors that contribute to a “better” development experience and platform reachability. We also contribute metrics and criteria for evaluating different implementation approaches to help developers and designers determine an appropriate mobile strategy for a particular application. After describing the application and experiment design, we present a detailed analysis of the developer diaries and

feedback based on a set of criteria and metrics for evaluating the mobile application development experience. In our ongoing work, we build on these findings to improve on existing multi-device solutions with the major goal of supporting interactive development on the mobile devices themselves.

Experiment Our experiment concerned the development of mobile client applications for an existing server system called Twitstore 1 , which offers a Twitter-like approach to personal notes consisting of free-form text with hashtags for metadata used to organise them. Similar to the desktop application, a mobile client would allow users to create and manage new items, retrieve existing ones and update, delete or duplicate items. It is also possible to send items to other users and retrieve a list of all hashtags. The Twitstore service supports ten operations exposed as a RESTful API, which need to be handled by processing HTTP requests in JSON format. We focussed on native mobile vs. web-based development, choosing the three main platforms and a popular web framework. Table 1 gives an overview of the development environments, where iOS and Windows Phone (WP7) specifically required the respective OS. All resulting applications were deployed and tested on mobile devices apart from iOS for which the emulator was used due to licence restrictions. Since web frameworks share many features, we decided to use the single vendor-independent open-source framework jQuery Mobile due to its high acceptance among developers. Other frameworks such as PhoneGapwere excluded at this stage since they are often hybrid solutions that generate native mobile applications from web-based specifications. 1 http://maven.globis.ethz.ch/projects/twitstore

Version Language IDE Development OS Tested Devices

Android iOS 2.3 5.0 Java Objective-C Eclipse Indigo Xcode 4.2.1 Ubuntu Linux, Windows 7, Mac OS X Mac OS X Samsung Galaxy SII —

Windows Phone 7.5 C# Visual Studio 2010 Windows 7 Nokia Lumia 800, HTC Surround

jQuery Mobile 1.1.0 JavaScript Text editors, In-browser tools Ubuntu Linux, Windows 7, Mac OS X Samsung Galaxy SII, Nokia Lumia 800, Apple iPhone 4

Table 1: Platform versions and development tools.

Stud. Andr. iOS WP7 Web OSX 1 1 1 0 2 2 2 1 0 3 1 0 3 1 0 3 2 0 4 3 0 0 2 0

Table 2: Student’s development experience before the experiment: 0 = none, 1 = some knowledge, 2 = good knowledge, 3 = very good knowledge with some applications developed previously.

Figure 1: Combined development times of all four students per platform.

To avoid bias towards certain technologies, we recruited four final-year Computer Science master’s students with significant development experience in general-purpose languages acquired through programming lab courses and company internships. They were each given the task of implementing Twitstore client applications on all four platforms during a semester and asked to document all work together with the hours, tasks and personal comments in a diary. Despite the limitations of diary studies [3], we felt it was overall the best method to collect relevant data that might reveal interesting insights into current problems of mobile application development. In a first meeting, the students were introduced to the experiment. The total time for development was nine weeks where one week was considered equivalent to ten hours of work. The first week was used to introduce the technologies by reading documentation, looking at examples, setting up the development environment and completing developer tutorials. The following eight weeks were split into four blocks of two weeks with each student developing the application for one platform in each block (20 hours). The assignments were controlled so that no two students worked on the same platform at the same time, resulting in four different orders. Also, each student started with a platform with which they had no prior experience. These decisions gave equal conditions for every participant and were made to reduce learning effects. The initial assessment of the students’ experience is shown in Table 2. There were weekly meetings of

students and supervisors to show and discuss progress as well as two final meetings to define criteria and metrics for rating and comparing the development experience. Finally, the students were asked to write a 15-page project report documenting major milestones as well as providing written feedback on the development experience. The report together with the diaries provided the basis for our analysis.

Results A structural analysis of the diaries showed that the common format was short to medium size entries with an average of 43 items (σ = 18), ranging from very long entries by one student with a total of 19 and the lowest frequency of around two entries per week to 62 entries in nine weeks. The work logged was divided into five categories: reading documentation; UI design and implementation; implementation of the application logic and client/server communication; testing and debugging, and “other” for activities that either did not add directly to the project or were marked as extra effort to improve the basic functionality. According to this classification, Figure 1 shows the time spent per platform for all students. Generally, the results look similar with the largest differences in the times spent on the documentation and UI parts and some interesting differences of Android and Web compared to iOS and WP7. iOS has by far the

largest documentation part, which can be explained by the fact that three out of four students had little experience with Mac OS X, Xcode and Objective-C. The documentation and UI parts are the smallest for WP7 where half of the students had extensive experience. Interestingly, although none of the students had a good knowledge of iOS, it comes very close to WP7 in the UI part.

Figure 2: Hours spent by the students on each platform per iteration. Pos. Neg. P/N

Andr.

iOS

19 26 0.73

20 26 0.77

WP7 Web 24 24 1

25 23 1.09

Table 3: Expressions of positive and negative sentiments per platform in the diaries. P/N is the ratio of positive to negative sentiments.

Figure 2 shows the time spent on each prototype. Even though the platforms differ in terms of IDE and programming language, the main concepts such as screens, HTTP requests and JSON parsers were generally the same. We therefore expected that a certain amount of knowledge transfer would be visible in terms of reduced development time over the course of the experiment. There were however two special cases worth noting: Student 3 was already very proficient with WP7 and so the third prototype was completed significantly faster than the last iOS prototype. Student 2 had the most experience with WP7 and hence benefited from this and the previous development experience for the last WP7 prototype. Most of the ten features of the Twitstore service were realised using separate screens with a main screen to create, list and search items; detail screens for the update, delete, duplicate and share actions, and additional screens sometimes with detail views for login, received items and the list of tags. On average, the applications consisted of five to seven screens indicating that features were often combined into one view. Most prototypes had two iterations—the first to realise the basic functionality and the second for a more advanced UI and additional features. In some cases, a third iteration was used to improve and extend the core features with platform-specific support such as local storage as well as a

more sophisticated UI with user feedback including loading screens and form validation. Generally, the students mainly adopted one of two approaches. In the “all-UI-first” approach, they completed the UI with all screens and the navigation between them first and only then produced the code for the core functionality. In the “screen-by-screen” approach, they staged development by screens, but still completed the UI first for each screen before the code. Overall, in only three out of 16 cases did the students follow a “code-first, UI later” approach. We collected developer feedback by interviewing the students in weekly meetings as well as analysing the diaries and project report. We scanned the diaries for expressions of sentiment and categorised them into weak or strong, positive or negative, statements. In total, we found 130 such statements of which 62 were positive and 68 negative. Giving the strong statements a double weight and summing up the positive and negative statements per platform provides us with the values shown in Table 3. The last line in the table is the ratio of positive to negative statements as a sentiment-based ranking of the platforms according to the developer diaries. The written statements about the four platforms are summarised below. Android The familiar Eclipse development environment was a plus. Manually editing config files, e.g. updating the manifest when a new activity is created, was perceived as a nuisance that should be automated. The interface builder and link between code and GUI needs improvement, as supported by the highest number of hours for the UI part in Figure 1. The API for HTTP requests is deemed simple to use and it is appreciated that it was very easy to deploy an app on a mobile device for testing. iOS All were impressed with Apple’s Xcode IDE, praising the auto-completion feature and the interface builder with

its XML-free storyboard approach. However, Objective-C was new to all students and received mostly negative comments, as supported by the numbers in Figure 1. Windows Phone Although the newest platform, it profited from the well-established support of Visual Studio as the IDE. However, the emulator is currently very slow and, while the linking between UI elements and source code was complimented, the UI designer was regarded inferior to others such as Xcode. Figure 3: Overall grading of the platforms (1=lowest, 6=highest grade). Grading Criteria Technical (Weight: 40%): Aspects of the development platform such as the programming language, IDE with interface builder and debugging tools (emulator) and available documentation. Subjective (30%): The developer’s feelings towards the platform, including criteria such as satisfaction with the resulting prototypes, the confidence in the platform and framework support as well as the ease of deployment and distribution of the application. Productivity (30%): The ratio of development output and the effort required to achieve it (e.g. number of platforms covered vs. the time spent coding).

Web The lack of a standard IDE with common debugging tools is the main negative point. Initially, solutions such as Aptana Studio and Eclipse were considered but not found suitable. General purpose text editors were therefore used with in-browser debugging using built-in tools or add-ons, e.g. Firebug. This gave the longest debugging time as shown in Figure 1. The rapid deployment of applications, the customisable look and feel and the ability to reach all platforms with a single app were all positive. After the implementations, the students were asked to grade their experiences with the four platforms. For this grading, a common set of criteria were devised and grouped into the three categories shown on the left. The associated weights reflect the importance factors as expressed by the students. Figure 3 shows the combined grades over the four students for each platform and criterion as well as the weighted overall grade. iOS got the best grades in the technical part with Windows Phone 7 a close second due to advanced and user-friendly development tools. According to the subjective criteria, Android is the clear winner due to familiarity with the environment and easy deployment. The productivity part is clearly dominated by the web-based solution as all platforms can be reached with a single application. The big lead in the productivity part ultimately helps the

web-based solution to be first in the weighted overall score even though it came last in terms of the other criteria. However, the overall ranking is very close and hence there is no clear-cut winner or loser according to the defined criteria. Note that the students’ weighted ranking confirms the sentiment-based diary analysis in Table 3. We can summarise the platform-specific observations into an “optimal development platform” by combining the positive features as follows: A powerful IDE with a good interface builder like Xcode, but with a more main-stream programming language; good link between interface builder and code as in Visual Studio; no need to edit configuration files manually; easy deployment of the application as in Android or the web-based approach; ability to reach all platforms with a single app.

Discussion and Future Work Overall, research has been dominated by comprehensive model-driven approaches such as PUC [7] and MARIA [8] for multi-device authoring. The vast majority of solutions require special tools and UI description languages that tend to follow a common model of abstraction layers [1], but use different vocabulary to specify the appearance of the UI in a device and modality-independent way. Often a top-down approach starting from abstract descriptions of the UI down to the concrete and final interfaces is promoted, but—as observed in our experiment—this may not seem practical for many developers and designers who typically tend to start with concrete UI prototypes. With the exception of design tools such as Damask [4] and DMacs [5] that aim to simplify multi-device authoring based on layered editing and tools for recording and replaying simple adaptations on different devices, more advanced approaches impose specific development methods and add to design complexity.

Acknoweledgements We thank Alexandre Blondin, Mikhail Chatillon, Christos Papastergiou and Hedi Radhouane for contributing to this experiment, as well as Ken Casada from Microsoft Switzerland for supporting this research with Windows Phone 7 devices.

Although device-specific authoring has serious limitations in terms of maintainability and scalability, it has recently become a popular method for the new generation of touchphones and tablet computers [2], also fostered by vendors through online app stores. The study presented here made several problems evident. There appears to be both a mismatch regarding framework usability and platform reachability. Also, it appears that the more platforms can be reached with a tool, the worse is the technical support and subjective feeling about the development experience. Our initial experiment was limited to a single application, selected platforms and a few participants only. Nevertheless, the rich qualitative data obtained from the diary study indicates the need for better integrated multi-device engineering solutions and a fresh approach. Based on the results reported here, we are currently working on new authoring tools specifically designed for multi-device user interfaces, investigating novel prototyping techniques based on our previous work on adaptivity metrics [6] and the idea of responsive design previews to test and refine device- specific adaptations. Another direction we follow is to explore interactive development techniques that help facilitate the authoring process directly on the target devices themselves as opposed to using emulators and simulating device behaviour. This includes both mobile devices and large interactive display surfaces. A particular challenge here is that the authoring environment itself must be context-aware and able to adapt to different viewing conditions and interaction modalities. At the same time, we are also investigating how end-user development and crowdsourcing techniques could be

combined and integrated into the authoring tools to leverage, rather than replace, existing methods and techniques for single-device authoring with the aim to further reduce design complexity and the effort on developers.

References [1] Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., and Vanderdonckt, J. A Unifying Reference Framework for Multi-Target User Interfaces. IWC 15 (2003). [2] Charland, A., and LeRoux, B. Mobile Application Development: Web vs. Native. CACM 54, 5 (2011). [3] Czerwinski, M., Horvitz, E., and Wilhite, S. A Diary Study of Task Switching and Interruptions. In Proc. CHI (2004). [4] Lin, J., and Landay, J. A. Employing Patterns and Layers for Early-Stage Design and Prototyping of Cross-Device User Interfaces. In Proc. CHI (2008). [5] Meskens, J., Luyten, K., and Coninx, K. DMacs: Building Multi-Device User Interfaces by Demonstrating, Sharing and Replaying Design Actions. In Proc. UIST (2010). [6] Nebeling, M., Speicher, M., and Norrie, M. C. W3Touch: Metrics-based Web Page Adaptation for Touch. In Proc. CHI (2013). [7] Nichols, J., Chau, D. H., and Myers, B. A. Demonstrating the Viability of Automatically Generated User Interfaces. In Proc. CHI (2007). [8] Patern`o, F., Santoro, C., and Spano, L. MARIA: A Universal, Declarative, Multiple Abstraction-Level Language for Service-Oriented Applications in Ubiquitous Environments. TOCHI 16, 4 (2009).