Renovate to Innovate

1 downloads 315 Views 2MB Size Report
COMPANIES TO EXPOSE IP, AND DELIVER FASTER INNOVATION CYCLES. White Paper ... Whether you're a large enterprise, an inde
DAITAN WHITE PAPER

Renovate to Innovate HOW MEDIATED APIS AND MICROSERVICES ARE ENABLING TECHNOLOGY COMPANIES TO EXPOSE IP, AND DELIVER FASTER INNOVATION CYCLES

We hear terminology from industry pundits like Digital Transformation, Digital Business, and Digital Economy.

White Paper Contents What’s driving the Renovate to Innovate movement How does renovation work Renovation architecture Benefits, challenges and best practices

And, as broad as it sounds, it encompasses a fundamental shift from business status quo, to businesses ‘transforming’ themselves in order to improve customer experience, and significantly accelerate how they innovate applications and products1. To help ground this in real terms, here is a definition that we believe encompasses the meaning: every company today that wants to reach employees and customers in a highly-connected ‘digital’ world, must invest in new technology, business models and engineering processes that result in rapid innovation cycles and create greater value. This is how the world competes today. As the speed of innovation accelerates, every technology executive works to keep pace with competition and increase business. This movement for faster innovation is driving fundamental changes in companies across all industries—from consumer-facing to B2B models. Whether you’re a large enterprise, an independent software vendor (ISV), or a startup, the demand is the same: Deliver faster cycles of innovation and enable experimentation, to connect with the outer world and at the same time, scale IT infrastructure on-demand to dynamically respond to your success while avoiding unnecessary costs. This is reality for every business’ survival. 1 DevOps & the Engineering-Led World of the Digital Enterprise, Daitan Group White Paper Enterprise APIs for Rapid Service Delivery, Daitan Group White Paper

White Paper: Renovate to Innovate

1 | © Daitan Group 2016 | Accelerating.TM

Every company today that wants to reach employees and customers in a highly-connected digital world, must invest in new technology, business models and engineering processes that result in rapid innovation cycles and create greater value. This is how the world competes today.

While the demands are the same for all companies, it bears mentioning that for technology organizations with existing applications and products, the challenges are more complex versus a start-up that carries no baggage from legacy systems or technical debt. Startups can move quicker and impact market share of an existing enterprise or ISV, so the urgency for established companies to ‘transform’ is exacerbated. That said, at the other end of the spectrum, large organizations like Amazon, Facebook, Netflix, eBay and others have already been “digitally transformed.” They led the charge by pioneering, launching and operating wholly new architectures capable of handling their super-connected world of customers. Now, the rest of the world learns by these examples and asks, “How can we adapt these strategies for our business?”

This White Paper The purpose of this whitepaper is to present approaches we have successfully deployed on multiple occasions for both mature ISVs and enterprise environments faced with the business imperative to accelerate innovation and leverage their existing assets without disrupting operations of an existing application. In short, we ‘Renovate to Innovate.’

Deliver faster cycles of innovation and enable experimentation, to connect with the outer world and at the same time, scale IT infrastructure on-demand to dynamically respond to your success while avoiding unnecessary costs. This is reality for every business’ survival.

White Paper: Renovate to Innovate

2 | © Daitan Group 2016 | Accelerating.TM

What’s Driving The Renovate to Innovate Movement? As today’s technology organizations embrace the inevitability of their digital future, and move toward becoming more fully digital businesses, they are recognizing the imperative to become more agile everywhere. Without agility, innovation is constrained. Organizations recognize they must adopt a customer-centered focus, and deliver software and product innovation faster, more frequently and with fewer errors. Agile methodology has moved from where it has been traditionally found — in engineering departments —  to all parts of the organization, from IT to marketing. And it has opened new paths to innovation. It is common to find enterprise I&O (Infrastructure & Operations) leaders are expected to provide the foundation for the agility and innovation throughout the entire organization. And when measured against the extremely high bar of digital-first businesses like Amazon and Netflix, enormous pressure is put on these leaders to adopt Agile in order to deliver value faster. Along with Agile there is DevOps, which extends the value of agility to the operational side. It has gained substantial momentum in recent years and resulted in an entirely new set of skills needed for organizations to effectively operate as a digital business. It is well documented2 that the DevOps culture has already proven its value, established clear ROI and accelerated delivery by enabling multi-functional teams to manage code and infrastructure together throughout the entire lifecycle. Successful DevOps environments are best characterized by organizations embracing new skills, technologies, tools and processes; as well as, a cultural mind set for experimentation and collaboration—all of which we like to think of as the new “digital stack.” 2 The DevOps Handbook: How To Create World-Class Agility, Reliability, & Security in Technology Organizations | by Gene Kim, Jez Humble, Patrick Debois, and Willis Willis, 2016

White Paper: Renovate to Innovate

3 | © Daitan Group 2016 | Accelerating.TM

What’s Driving The Renovate to Innovate Movement? Organizations are Compromised When They Let the Past Dictate Their Future When organizations choose to maintain legacy teams that stick to legacy methods and culture, they are unable to leverage the enormous potential of a modern digital business and are compromised in multiple ways: 1. Monolithic systems requiring legacy methods, infrastructure and teams remain isolated from digital business innovation. 2. The business risk grows for competitors and/or start-ups to accelerate past them, steal market share and revenue. 3. Valuable functionality and data from monolithic systems remain unavailable to new, potential business opportunities and broader audiences. 4. It becomes increasingly challenging to hire qualified, high-achieving engineers, or engineers early in their career— as they are generally more attracted to pure-play innovation roles rather than sustaining legacy, monolithic systems. Taken together, these challenges represent enormous risk to the company’s future. So, it becomes understandable why the very big concept of Digital Business is being driven by the executive level. Businesses can’t thrive in today’s increasingly competitive world unless they transform.

White Paper: Renovate to Innovate

Embracing the Digital Stack Can Start with Focused Initial Efforts That Capitalize on New Opportunities Today many organizations start— and are having early-stage success— by focusing on experimental and innovative new projects. These are green-field projects that can be developed independently from existing, monolithic legacy systems.  These teams typically have the right skills and experience to produce a modern architecture capable of rapid service delivery to digitally-connected customers and employees. So, even in environments where legacy systems exist, green-field development can be possible. Of course, it means finding the right technical talent, acknowledging the incremental cost of concurrent development, and recognizing that there are essentially two development teams co-existing bimodally. However, the real issue with this approach is it fails to take advantage of any existing value, whether it’s clever functionality or unique data, which is why we developed the model: “Renovate to Innovate”.

Renovate to Innovate Bridges the Gap Between ‘Existing’ and ‘New’ It’s a more practical approach. Renovating-to-innovate helps bridge the gap between two islands of development (and product). So, the business actually leverages the value of their existing IP from the legacy system onto the strategy for innovative ‘green-field’ development projects. This approach addresses the important challenges mentioned above as well as, facilitates the organizational and cultural transition to progress at a manageable pace. The importance of people, skills and culture should not be under estimated. And, one of the great attributes of Renovate to Innovate is it enables companies to begin — often gradually at first — to move development to Mode 2 approaches.   This brings the product’s value and the organization forward together in a cohesive approach. Once the process starts, everyone can then begin to experience the benefits that come from adopting digital business practices.

We developed the model Renovate to Innovate because it enables the move to Mode 2 development, to begin to take advantage of existing value — whether it’s clever functionality or unique data.

4 | © Daitan Group 2016 | Accelerating.TM

How Does Renovation Work?

RENOVATE TO INNOVATE MEANS PROTECT WHAT YOU HAVE WHILE TRANSFORMING TO A NEW, DIGITAL, FUTURE We’ve established what renovation is: the transition of legacy systems, organizations and culture to embrace and leverage the digital stack for the purpose of accelerating release cycles that lead to business benefits. This paper recommends process and best practices we have learned for making the full transition with lowest risk, while enabling benefits as quickly as possible. We concentrate on strategies to renovate both legacy systems and newer, but monolithic applications, in order to enable them for agility and interoperability.

In a renovation model, the migration of thin functional slices while rigorously maintaining compatibility is important, and fundamental to ensuring both technical and business goals are progressively and methodical achieved.

Protect What You Have, While Transforming to a New, Digital, Future A foundational tenet of Renovate to Innovate, is preservation of the existing “monolith” and not disrupting existing business, while concurrently—and incrementally—the system is renovated. Preventing disruption is critical.  Monolithic functionality is maintained until no longer needed.  In practice this could take months, years or never happen at all depending on the company’s business model.

What is Bimodal? Bimodal is the practice of managing two separate but coherent styles of work: one focused on predictability; the other on exploration.

Mode 1 is optimized for areas that are more predictable and well-understood. It focuses on exploiting what is known, while renovating the legacy environment into a state that is fit for a digital world. Mode 2 is exploratory, experimenting to solve new problems and optimized for areas of uncertainty. These initiatives often begin with a hypothesis that is tested and adapted during a process involving short iterations, potentially adopting a minimum viable product (MVP) approach.

White Paper: Renovate to Innovate

Both modes are essential to create substantial value and drive significant organizational change, and neither is static.

Marrying a more predictable evolution of products and technologies (Mode 1) with the new and innovative (Mode 2) is the essence of an enterprise bimodal capability.

Both play an essential role in the digital SOURCE: GARTNER GROUP, IT GLOSSARY transformation.

5 | © Daitan Group 2016 | Accelerating.TM

The Renovation Architecture The Renovation Architecture, its components, process and best practices emphasizes compatibility and stability while enabling forward progress and improvements. The two key components of a Renovation Architecture are Mediated APIs and Microservices. Mediated APIs and Microservices are the enablers to exposing reusable IP in existing products that can be consumed by a broader set of users. For example, it could result in new user-driven apps that deliver a strong competitive advantage, or open a new market opportunity. In addition to functionality, they can also enable broader access to valuable data for business intelligence purposes. In our approach, the first stage is implementation of a Mediated API layer, and the second is building the Microservices. Let’s walk through the role each plays in the renovation process and the benefits each provide.

White Paper: Renovate to Innovate

6 | © Daitan Group 2016 | Accelerating.TM

The Renovation Architecture

White Paper: Renovate to Innovate

7 | © Daitan Group 2016 | Accelerating.TM

The Renovation Architecture

White Paper: Renovate to Innovate

8 | © Daitan Group 2016 | Accelerating.TM

Mediated APIs Set the Framework for Renovation A Mediated API layer is the most critical element of the Renovation Architecture.

A mediated API is a design pattern in which an API is virtualized, managed, protected and enriched by a mediation layer. This layer can enforce policy and inject capabilities into the API interaction for increased agility, usability, performance, security and control. – MEDIATED API DEFINITION

Mediated APIs are part of an architectural model that inserts a management layer between an application service and service consumers (such as apps, “things” and other services). This management layer incorporates capabilities for versioning, security, performance, monitoring, translation etc. Such APIs are extensions of the well-known Facade pattern in software engineering. While Facades only provide a simpler interface for complex underlying structures, Mediated APIs also add functionality to that interface, even though the underlying code may still be monolithic. In other words, the Mediated API enables extraction of high-value data and functionality so it can be consumed into innovative, experimental projects as though it were invoking Microservices.

What are the Benefits of Mediated APIs to Transformation? A number of benefits around process standardization, management and performance are derived when an API Mediation layer is in place. Most important, it fundamentally becomes the transitional framework for any technology company to be able to leverage existing IP into broader, new innovative uses. There are multiple tangible benefits of Mediated APIs including:

APIs are foundational for any digital business. They enable the API economy, multichannel applications, pervasive integration and other digital business scenarios. Application architecture leaders should strategically adopt the Mediated APIs model to enable and protect their APIs. White Paper: Renovate to Innovate



A standard way to implement security and access control on all APIs.



A standard way to monitor services and to manage service-level agreements.



A mechanism that improves overall system performance, scalability and resiliency.



A mechanism that supports API versioning and development of custom APIs.



A mechanism that abstracts and simplifies complex service topologies to make it easier for service consumers to use the services.



A mechanism that allows services to be enabled through legacy systems and packaged applications.

9 | © Daitan Group 2016 | Accelerating.TM

Guiding Principles for Mediating APIs Once API Mediation is implemented—and the functionality from a monolithic system is accessible—experimental developers can begin to innovate. However, when any business opens up access, proper control is essential in order to avoid issues such as overloading the system with requests and preventing security attacks to mention a few. So, we recommend a few key guidelines to help ensure success. Know

your customer

Know Your “Customer” — AKA the Consumer of APIs Think of the API as a product, and developers are the customers of that product. For an API, customer focus really means developer focus.   Although the mediation layer will connect to multiple legacy and packaged applications, and multiple teams may be assigned to develop totally different APIs for the external world, remember it’s important to first consider who the real end user will be. Ask yourself, “Who would benefit from accessing functionality and/or data; and what do they need to do with it?” Think about the most typical use scenarios for the mediated APIs. Understanding the actual end-user is key—are they internal users looking for data for business intelligence or external third party developers accessing clever functionality to build something wholly new? It is often the case that APIs already exposed from your current applications were not designed for the new use scenarios you have in mind. In these cases, the mediation layer can provide combination and transformation capabilities so that the exposed API is friendly to the modern Web and mobile app developer. All of these considerations fundamentally affect how to design the API, the interface and anticipate performance needs. The objective is that all aspects of the (API) product should be consistent, stable, documented and timeefficient in order to be successfully ‘consumed’.

Standardize

Standardization Saves Time in the Long Run We recommend setting standards and best practices for modelling APIs.  Standards should cover naming conventions, parameters, calling sequences and return codes.   It is a best practice that all APIs should adhere to, because in the long run, it saves time and increases successful utilization.  Good standardization will shorten developer learning curves and improve the overall cohesiveness of resulting Microservices when they are implemented.

White Paper: Renovate to Innovate

10 | © Daitan Group 2016 | Accelerating.TM

Document

Quality Documentation and Support are a Must Whether APIs are accessible to just an internal development team or third party consumers, documentation and support are essential to implementing a strong Renovate to Innovate program. Plan to make a significant investment in effort and resource to build the proper documentation and responsive support. Throughout development, thorough documentation should be concurrently produced along with FAQs, videos, functional examples and code snippets if possible.  Documentation often determines the developer’s ‘first impression’ of the API.  Documentation must be easy to understand and efficient to use. Every API is just like a product, and in addition to documentation, products need high-quality support.  It’s important to have a viable support path for all common queries, and even the ability to handle unanticipated queries.  Some developers prefer online self-help and others want direct personal interaction, so the purpose of support is to enable handling all scenarios.   Time will pass and ultimately there will be a need to enhance and/or upgrade documentation based on new revelations that can result from support requests. Sustaining the documentation and support materials is part of the long-term program and as with any product lifecycle, anticipate a plan for ultimately retiring/or replacing the API.

Anticipate

Anticipate Creative Usages and Be Prepared to Respond Developers will use APIs in creative, unanticipated ways, which can reveal exciting new possibilities that can evolve the API.  After all, the API is a product. As these new usages are released with proper documentation and support, they improve the perceived flexibility and value of the API—which ultimately drives greater exposure and utilization. However, unanticipated usages can also introduce issues that expose weaknesses in process, support or performance. For example:

White Paper: Renovate to Innovate



Exposing defects not resolved from ordinary quality processes.



Creating challenges for unprepared support organizations.



Building unexpected loads that challenge or potentially defeat scaling.

11 | © Daitan Group 2016 | Accelerating.TM

These risks are mitigated in environments with a strong DevOps culture, skills, team structure and best practices. So migrating to an API architecture requires a working DevOps environment prepared to handle change. Let’s take two examples: Support & Escalation All support queries should have a clear escalation path directly to the multifunction DevOps team for the API and resulting Microservices.  Developers, testers and operators should collaborate in response to issues guided by a customer focus that determines solutions.  When change happens, the system of documentation and support should automatically be updated to incorporate new usages into the known best practices for the API. DevOps environments inherently operate this way and makes provisions for incorporating new “feedback” into the product’s lifecycle. Protective Mechanisms Consider “throttling” or other methods of shielding systems from unforeseen loads.  It is usually better to provide slowed responses for a short period, instead of threatening overall system stability at least until new usages are understood and fully supported. We experienced one case of an unexpected load increase when a consumer decided to use the API to create their own local history database by connecting to the API to retrieve updates many times per second. Because the architecture implemented used throttling, it mitigated the risk of downtime without disrupting the API user. In summary, using API Mediation is a great choice because it allows for a quick delivery of new API’s that interact directly with the monolith’s internal services, while the monolith’s services can be replaced. We have discussed the first phase of “renovation.” During this phase a company that implements a Mediated API layer allows innovative, experimental developers to leverage thin slices of functionality and data from monolithic code; and the organization actively relies on DevOps methods to rapidly cycle innovation, feedback and change into a responsive, repeatable mechanism. In short, the organization begins to perform like a digital business.   Next, is phase two—when Microservices are added to the design—the business can begin to truly unleash their monolith’s functionality and data into a scalable architecture for rapid service delivery.

White Paper: Renovate to Innovate

12 | © Daitan Group 2016 | Accelerating.TM

Microservices Transform Monolithic Code and Enable Future Expansion of New Functionality It may not be practical or economically viable to completely transform the monolith. It is quite feasible to run in a mixed mode with much of the monolith remaining intact.

Microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.

After API Mediation is implemented, tested, stable and published, attention turns to the underlying code. The underlying code and functionality can now be migrated to a true set of Microservices that align to the Mediating API. At this point you should have good visibility of the new usage patterns and which parts of your existing apps and services are in most demand. With this information you can begin to plan and break out specific pieces of functionality so that you can implement dynamic scaling and redundancy models required for the new app users’ demands. Your legacy monolithic applications typically cannot scale in an optimal way, so it is necessary now to break out the most in-demand functionality into Microservices and although this does bring risks to your existing apps the risks can be mitigated by incrementally developing only the required Microservices rather than tackling a major application rewrite. Depending on the size and number of legacy applications, building a network of Microservices may be a lengthy and resource-intensive process. This is why we recommend implementing the Mediated API layer first, to experience the value of an architecture based on services and to facilitate the organizational transformation into a strong DevOps culture—before tackling Microservices. Not only should you be considering migration of existing functionality to Microservices, but any incremental, new functionality should be built as a Microservice. That said, we do recognize that there are some business model scenarios that are best served by the company running in a mixed mode format—in other words, maintaining the original monolith in addition to leveraging the newly architected system into new market opportunities. This has been the case with some of our customers. Each company is different and must assess the right level of “renovation” needed to grow. During the second phase, we recommend companies methodically evolve to levels of confident performance—including product, infrastructure, people and culture—and then scale their expansion of Microservices. As Microservices are built, each one works across the pipeline including cycling feedback into the process and continuously innovating to rapidly deliver enhancements. Operating at full capacity is all about having an agile mind set and living a DevOps culture. The end result is a company that renovates progressively, and in the process realizes they have transformed code, architecture, infrastructure, best practices and teams to a new way of doing business—a digital business.

– MICROSERVICES DEFINITION, JAMES LEWIS AND MARTIN FOWLER

White Paper: Renovate to Innovate

13 | © Daitan Group 2016 | Accelerating.TM

Benefits of Microservices At Daitan, we have brought Microservices to many of our clients faced with the business imperative to become more nimble, agile and “digital.” We have successfully deployed the Renovate to Innovate approach in a variety of environments, industries and models. In every case, companies that migrate to Microservices and rapid service delivery architectures are better able to respond to customer need and market dynamics. To make it tangible, here are some of the most important benefits of Microservices: Shorter Release Cycles Because Microservices can be released in their own build-testrelease streams — in many ways independent from other large portions of the application — it is easier and quicker to release new services to production.

are responsible for, with minimal guidance from more experienced team members.  (We do always, however, recommend that all team members eventually become familiar with the whole structure in order to ensure the broader picture is always understood.) Ability to Scale by Service Depending on role and functionality, apps can potentially have a wide variety of scaling models but ultimately any design’s goal is to scale efficiently versus brute-force.  As an example, apps that present an intra-day usage variation (ie are typically subject to high load during business hours and little load during lunch time and at night) can benefit from Microservices, which provide finer control of computing resources and allow more efficient, flexible scaling in circumstances like these.

One of our clients is able to complete an entire new system integration — from feedback and requirements through development, testing, delivery and deployment into production (complete with API and documentation) — in a week because of their new Microservice-based architecture.

If an intra-day usage variable app is part of a legacy monolith, then all functionality, code and infrastructure must scale-up with demand.  With Microservices, only the small dependent slices of infrastructure must scale, requiring fewer compute resources.  A Microservice architecture is the only way to achieve such efficiency.

Improved Ramp-Up Time— Acceleration

Easier Integration with External Services

Microservices accelerate ramp-up because new team members do not need to understand the full application architecture from day one. Instead, team members can focus on the specific service they

White Paper: Renovate to Innovate

When the requirement is to integrate an application with a large number of external services, Microservices enable a streamlined approach. Such as, it is possible to model the application’s integration endpoints as services that can scale

independently.  It is also possible to prepare integration templates or ‘boilerplates’.  These are consistent models of all the common interface features common to all integrated services.  With a template model, only the unique characteristics need to be implemented, so it’s not necessary to spend effort on common features and interfaces. Improved Team Organization, Along End-to-End Services Microservices, DevOps and Agile methodologies have synergy because they all benefit from similar organizational structures.  Each benefits from multi-function teams that manage all aspects of development and operational lifecycle. A team responsible for one or more Microservices needs architecture, engineering, test planning and test execution skill sets.  It also requires build automation, logging, scaling and high availability expertise.  Placing all these skills on a single team promotes responsibility and ownership.  It also ensures that most problems can be solved within the team. Finally, Agile tools can be used to track work, completion, velocity and real-time management reporting.  These are all key attributes of a digital business culture. In summary, Microservices are one of the best ways to truly renovate an application infrastructure, allowing for faster release cycles, improved ramp up time, increased ability to scale up and down services on demand, among others, which enables quick experiments—a requirement for innovation. 14 | © Daitan Group 2016 | Accelerating.TM

Common Challenges Well-Suited to a Renovation Approach There are many possible challenges, but we’ve identified a few examples based on our experience. Considering migrating a complex, monolithic application is daunting for any organization—it affects architecture, code, infrastructure, process and people. But, it is doable and we have helped several customers using a Renovate to Innovate approach to make the journey manageable. As with any important business initiative, it starts with carefully planning renovation efforts to expose risks, and scope what is essential for successful execution. As mentioned in the beginning, there is no disruption to an existing monolith so the business can function while renovation-toinnovation is underway. If any of the following scenarios describe your environment, then consider a Renovate to Innovate approach.

White Paper: Renovate to Innovate

Legacy Systems Not Built to Deliver Agility

The Need for Optimized Infrastructure Costs

Legacy and packaged applications may not be adequately serviceenabled to participate in modern application architectures and therefore require a different approach. They often have proprietary data models, interaction patterns, and authentication and authorization structures that further complicate matters. And, many legacy and packaged applications provide arcane APIs (if any) that make it difficult to enable such integration. These APIs expose complex data models and operations that don’t match the specific needs of digital business applications.

Even when applications are quickly built to scale and running on demand, inefficiencies can result if the applications are not well designed. They must be designed so that the right modules scale up, and down, appropriately with demand.

New Applications Evolve with Unbalanced Scalability Demand Even new applications built from scratch such as in “Mode 2” Bimodal IT, and those found in startups, do not always follow best practices for interoperability and scalability. Frequently, we have noticed that teams see following these best practices generates overhead in the short term. So it is common for Minimum Viable Products, and first application versions, to be implemented in a monolithfashion.  Then, as the application starts to gain traction, demand on the application can increase unevenly, with some features or services subject to higher loads than others. So this may lead to a need to scale only part of the application — a challenge for monolith architectures.

15 | © Daitan Group 2016 | Accelerating.TM

Five Key Renovate to Innovate Takeaways With all the talk about digital business, the most common question we hear is, “Where do I begin?” So we’ve identified a few key takeaways below to guide your exploratory process with the suggestion that Daitan lives and breathes solving these problems for large and small companies, ISVs and enterprise IT departments, so always feel free to just ask us for help. 1. Start with a Focused Goal Completely renovating a large monolith can be a long-term project. As we mentioned earlier in this discussion, many companies today are having success initiating transformation with a focused objective that delivers short-term benefits. Examples of those kinds of projects include: * Simplifying and increasing release frequency for very specific services or functionality. * Creating an independent code/ infrastructure slice to drive DevOps culture. * Driving Agile best practices, revamping team skills/culture

White Paper: Renovate to Innovate

2. Plan the API Mediation Layer Components When we talk about implementing a Mediated API layer, there are two ways to accomplish that. Creating an API Mediation layer from scratch is a possibility for very specific requirements, but it is also possible to use one of the API Management tools available in the market. Start by identifying which functionality(security, monitoring, versioning etc.) your mediation layer should add to your mixed  monolith and Microservices architecture and check vendors that provide those functionalities. 3. Think of the API Mediation Layer as a Product and Standardize Take a customer-centric approach to your API Mediation layer. Start by thinking on the tasks your endusers would want to accomplish with your API. Discuss with your team about standards to be followed in such a way that different engineers will be able to create API’s that look consistent from the end user’s point of view. And package your API in a well-designed portal with lots of examples and sample code that can be easily adapted by other API consumers.

4. Implement New Functionality in Microservices Business requirements are always changing and evolving, and new functionality must be implemented in the applications to support these requirements. Instead of making your monoliths bigger, start implementing new functionality in Microservices to start reaping benefits quickly. 5. Forget ‘Big Bang’ Projects Like Rebuilding the Monolith Big Bang projects are expensive and risky. We recommend shifting to Microservices incrementally, by replacing functionality step by step in order to mitigate risks, leverage quick wins and optimize investments. Eventually, you may find out that not all functions should be implemented in Microservices, either because they are not used anymore or because the benefits of migration will not justify the costs. Or even because the monolith’s become so small that it is almost a Microservice itself.

16 | © Daitan Group 2016 | Accelerating.TM

Best Practices When Using Renovate to Innovate With every implementation Daitan has done, we have learned new lessons, which become “best practices” we like to share. Here are a few key best practices you should be aware of: Microservice Monitoring and Log Management In a Microservices architecture, logs are generated for each Microservice separately. To troubleshoot problems, it is often necessary to view a single log stream of all events across all Microservices. To facilitate this task, we strongly recommend that logs include the transaction ID and a Microservice ID (because Microservices can scale up and end up with many instances running in different servers) and that servers leverage NTP (Network Time Protocol) for clock synchronization. This allows teams to more quickly understand where and when issues occur. Similarly, log aggregation is also important. When troubleshooting issues, it’s important to see the end-to-end transaction, in the exact sequence it happened, and have those transactions be easily accessible. Microservice Architecture Must Support the Business Needs Each type of business requirement demands a different type of infrastructure architecture, and this may affect how Microservices are built. Understanding this is the first step in a successful Microservices implementation.

White Paper: Renovate to Innovate

It’s also important to understand where priorities and tradeoffs can occur. For example, applications dealing with real time communications may be permitted to lose some data packets, provided that most of the packets are delivered quickly to recipients. On the other hand, in the case of an application that performs critical security analytics, losing a transaction packet is absolutely not acceptable. Ensuring all packets are delivered reliably may be the priority over speed of processing. Microservices usually require a Service Discovery solution because services are deployed and destroyed more frequently either for scalability or to enable the “immutable infrastructure” pattern, so it is important to know where new services are available and how to connect (or reconnect) with them. Beware of Microservice Coupling Be careful with service coupling. Developing coupled services will bring you back to a situation similar to a monolith, where changes affect many parts of the application.

Two Approaches to Building an API Mediation Service Layer In some scenarios, building an API Mediation layer from the groundup may be the only option, due to specific project requirements. We have done this. If you are considering embarking down this path, we recommend you truly assess and understand the skills needed to ensure timely delivery and a result that smoothly facilitates Microservices.

Immutable Infrastructure is an approach to managing services and software deployments on IT resources wherein components are replaced rather than changed or modified. An application or services are effectively redeployed each time any change occurs. SOURCE: WIKIPEDIA.ORG

Avoid Technology ‘Spaghetti’ Microservices allow different technologies to be used to deliver different services, but we recommend avoiding multiple “best-of-breed” technologies or programming languages in order to prevent creating a complex environment that inhibits growth. Looking at the DevOps Toolchain helps to break apart various technologies and tools that apply in a Renovate to Innovate implementation.

Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packetswitched, variable-latency data networks. In operation since before 1985, NTP is one of the oldest Internet protocols in current use. SOURCE: WIKIPEDIA.ORG

17 | © Daitan Group 2016 | Accelerating.TM

Renovate to Innovate – Technologies and Tools to Consider For most scenarios we recommend leveraging tools available in the market. The functionality you want the mediation layer to inject in the service will drive the decision for the type of product to consider. API Gateways such as NGINX will do the job when the requirements are simple. API Management tools such as, 3scale, Apigee (Google-owned), Axway, Mashape and Mulesoft are able to provide a myriad of additional benefits such as monitoring, scalability, security, data transformations, context injection and more. Containers Containers encapsulate discrete components of application logic provisioned only with the minimal resources needed to do their job, decoupling them from the infrastructure. Although not a hard requirement, we recommend packaging individual Microservices within containers. This facilitates separation and independence between Microservices, and also promotes portability and scalability. Docker is the most popular container engine we use today and has gained market dominance.

Source: Stages in a DevOps Toolchain (https://commons.wikimedia.org/wiki/File:Devops-toolchain.svg) White Paper: Renovate to Innovate

18 | © Daitan Group 2016 | Accelerating.TM

Renovate to Innovate – Technologies and Tools to Consider Orchestration In order to be able to run and manage Microservices in production, especially in a cluster of servers in public or private clouds, an orchestration mechanism is needed. When the architecture does not use containers, other DevOps tools can be used to configure, provision, and deploy Microservices in the cloud. Popular options include Chef, Ansible, Puppet and SaltStack. Cloud-based tools such as AWS CloudFormation are also available. Both usually require additional tools or even coding to support the highly dynamic nature of Microservices. In the container management and orchestration space, popular orchestration tools include Kubernetes, Docker Swarm, Apache Mesos and Rancher. We have also successfully used Platform-as-a-Service (PaaS) providers such as Cloud 66, who provide full container management as a service. With Cloud 66, containers can be deployed flexibly to various Infrastructure-as-a-Service (IaaS) providers including Amazon Web Services (AWS), Microsoft Azure, Google Cloud, Rackspace and DigitalOcean. Management is also facilitated by policies that determine when to scale up or down. Monitoring Microservices present a new challenge for APM (Application Performance Management) tools because of its dynamism and scale. Traditional tools such as New Relic are evolving to be able to monitor bottlenecks in this fluid environment where Microservices can be provisioned, respond to a request and get destroyed in a matter of seconds. Orchestration platforms and container management PaaS usually can do basic monitoring, which allows them to scale up and down and self-heal according to preconfigured policies. Log Aggregation We already discussed the importance of log aggregation in the scenario of Microservices, which is known as distributed systems. Some of the tools we recommend for this task include Splunk, Logstash, Sumo Logic, and Log Entries. Documentation For API documentation, Open API Initiative (formerly Swagger) is a good choice.

White Paper: Renovate to Innovate

19 | © Daitan Group 2016 | Accelerating.TM

Conclusion When all is said and done, remember this: Every company today that wants to reach employees and customers in a highly-connected ‘digital’ world, must invest in new technology, business models and engineering processes that result in rapid innovation cycles and create greater value. This is how the world competes today. Renovate to Innovate is an excellent strategy to get there without jeopardizing existing business momentum.

About Daitan Group Daitan Group provides high quality software development services to significantly accelerate time to market for global technology companies. The company’s expert agile teams deliver full lifecycle software product development, maintenance and quality assurance services across today’s leading technologies, including: cloud computing and virtualization; communications, collaboration and messaging; and big data/analytics. For more information: http://www.daitangroup.com.

DAITAN GROUP HEADQUARTERS 2410 CAMINO RAMON, SUITE 285 | SAN RAMON, CALIFORNIA, 94583 | USA

White Paper: Renovate to Innovate

20 | © Daitan Group 2016 | Accelerating.TM