Oracle JD Edwards EnterpriseOne Notifications Performance ...

0 downloads 286 Views 1MB Size Report
security against the performance impact it will have on the EnterpriseOne architecture. .... The notification designer m
Oracle JD Edwards EnterpriseOne Notifications Performance Characterization Using JD Edwards EnterpriseOne Notifications WHITE PAPER August 30, 2018

Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle.

ORACLE JD EDWARDS ENTERPRISEONE NOTIFICAITONS – PERFORMANCE CHARACTERIZATION

Table of Contents Executive Summary ..................................................................................... 3 What are Notifications?................................................................................ 5 Understanding Notifications ......................................................................... 5 Delivery Method ........................................................................................................................... 6

Notification Load .......................................................................................................................... 7

Notification Architecture – Runtime .............................................................................................. 7

Designing Notifications................................................................................................................. 8 Notifications – My Subscriptions .................................................................................................. 9 Notification Delivery Method ........................................................................................................ 9 A Notification is an UDO Object ................................................................................................. 10

Notification Runtime Modes ....................................................................................................... 11 Performance Impact of Proxy User vs Run As Subscriber ......................................................... 13 Proxy User Performance Profile ................................................................................................. 15 Run As Subscriber Performance Profile..................................................................................... 16

Understanding the EnterpriseOne Scheduler ............................................ 16 EnterpriseOne Scheduler Configuration..................................................................................... 16

EnterpriseOne Scheduler Architecture ....................................................................................... 17 EnterpriseOne Scheduler In-Memory and Resilience Configurations ........................................ 17 Resilient Scheduler Architecture ................................................................................................ 19 EnterpriseOne Scheduler High AvailabIlity Configuration .......................................................... 19

Notifications – Performance Characterization............................................ 20

1 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Notification Testing Scenarios .................................................................................................... 20 Notification Scenarios ................................................................................................................ 21 Interactive User Scenarios ......................................................................................................... 21 Impact of Notifications on Interactive User Experience .............................................................. 21

Performance Analysis of Delivering Notifications ....................................................................... 22

Notification Types....................................................................................................................... 22

Notification Delivery Mechanisms .............................................................................................. 23 Run as Subscriber versus Proxy User ....................................................................................... 24 EnterpriseOne Scheduler Resiliency and Oracle Traffic Director Analysis ...................................................................................................................................... 24

EnterpriseOne Notification Tuning ............................................................. 26 EnterpriseOne Logic Server ....................................................................................................... 26

EnterpriseOne HTML Server ...................................................................................................... 27

EnterpriseOne AIS Server Tuning .............................................................................................. 29 Configuring the Association between an AIS Instance and an HTML server ............................. 30

Conclusion ................................................................................................. 31 Appendix A: Test Configuration ................................................................. 32

2 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

EXECUTIVE SUMMARY

The performance characterization of the EnterpriseOne Notification feature, introduced in EnterpriseOne Tools 9.2.2.1, enhanced in EnterpriseOne Tools 9.2.2.6, and within EnterpriseOne applications 9.0, 9.1, and 9.2, is the topic of this document. As part of these Tools releases, the Oracle JD Edwards EnterpriseOne development team has been engaged to characterize the performance of the various notification configurations. The purpose of the performance characterization is to demonstrate that there is minimal impact to interactive user performance while using the EnterpriseOne notification feature and to provide insight and recommendations on using the Notification feature.

The resulting performance characterization concluded that when the recommended architecture is used, there was no significant impact to interactive applications with notifications enabled. Increasing the number of EnterpriseOne notification subscribers resulted in linear increases in the notification runtimes, as expected. This impact to the efficiency varies depending on a number of factors discussed in this document, including notification configuration and how it is delivered to the end user. This document illustrates that the Run As Subscriber notification configuration has the largest performance impact to notification runtimes and must be configured with careful business considerations in mind.

For notifications, take into consideration the requirement of each subscriber’s EnterpriseOne user security against the performance impact it will have on the EnterpriseOne architecture. When initiating an EnterpriseOne notification as the proxy user, the proxy user security is used. The second business tradeoff around Run As Subscriber is the specific mappings of an EnterpriseOne user profile to specific data sources. It is possible that the proxy-user data source will be different from a specific EnterpriseOne user subscriber to the notification. Because of that, running the notification might return a different value, create a different performance profile, and create a different overall end user experience. Testing of alternate data sources is not the topic of this document; it is only mentioned as a required consideration when initiating a notification as Run As Subscriber, which results in more work. The Proxy user is unable to use alternate data sources for individual subscribers.

3 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Regarding the EnterpriseOne architecture configuration for notifications, a comparison of the EnterpriseOne Scheduler Resiliency and Oracle Traffic Director Load Balancer showed minimal performance impact. A summary of the results shows that either architecture can support the notification load that was tested. The main benefit of Oracle Traffic Director is that it is a hardware load balancing solution, unlike the fault tolerance solution provided by the EnterpriseOne Scheduler Resiliency solution.. The EnterpriseOne Scheduler Resiliency architecture model also requires manual tuning to achieve efficiency in notification message delivery. The guidelines in this document will assist you in tuning this EnterpriseOne Scheduler architecture.

4 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

INTRODUCTION

Oracle JD Edwards EnterpriseOne is an integrated applications suite of comprehensive enterprise resource planning (ERP) software that combines business value, standards-based technology, and deep industry experience into a business solution. The JD Edwards solution architecture can exist on multiple platforms and on multiple database architectures. This document describes performance characterization testing performed with the JD Edwards EnterpriseOne notification feature running on an Oracle Linux-based virtual machine. The target audience of this document are individuals involved in the design, planning, and implementation of EnterpriseOne notifications.

WHAT ARE NOTIFICATIONS? JD Edwards EnterpriseOne notifications enable you to improve your business efficiency through the use of proactive notifications that are actionable. Proactive notifications enable the system to notify users of business events as they happen without the need for the user to be online. Refer to the following guide on the Oracle Help Center for more information on notifications: JD Edwards EnterpriseOne Tools Notifications Guide As a notification designer, you decide and implement the notifications that provide pertinent business information to your users. Users can then choose to subscribe to available notifications and receive updates within EnterpriseOne, as browser pop-up messages, through the Work Center, by email, or by email to SMS (Short Message Service). By creating notifications that are valuable to your subscribers, you enable them to know when key transactions or events occur without even having to log in to EnterpriseOne. This close monitoring helps them respond quickly and perform their jobs more efficiently. With the JD Edwards EnterpriseOne Orchestrator Studio, you can devise processes called notifications that enable the transformation of data into actionable business processes in JD Edwards EnterpriseOne. For example, you can create notifications that enable EnterpriseOne to: • • •

Alert users to a required activity. Alert users when a Watchlist threshold has been surpassed. Broadcast an informational message to users.

Notifications that you create in the Orchestrator Studio are saved to EnterpriseOne. The JD Edwards EnterpriseOne Orchestrator then processes the notifications based on how often you schedule them to run and sends notification messages to subscribers.

UNDERSTANDING NOTIFICATIONS EnterpriseOne Notifications can be simple notifications such as a text message or complex in nature involving form service requests through an EnterpriseOne orchestration. When implementing EnterpriseOne notifications, you should consider use, function, and frequency of initiation for performance impact. This document describes some ramifications of those choices. This section describes the foundational concepts surrounding the notification process.

5 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Delivery Method The delivery method for EnterpriseOne notifications defines how the EnterpriseOne user sees the notification. There are three different delivery methods configured through the individual subscriber’s My Subscriptions (Subscription Manager Application).

After logging in to EnterpriseOne, Subscription Manager is located in the user drop-down menu as illustrated in the previous diagram.

In this example, the subscriber’s delivery method for the Credit Review notification is set to Work Center. The other valid delivery methods are Email and Notification List, also known as the EnterpriseOne notification bell. Note: Testing results provided in this document only cover the notification list and Work Center delivery mechanisms.

6 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Notification Load Another aspect related to performance is the notification load generated against the EnterpriseOne architecture. For the purpose of this document, here are the definitions of notification load and other key terms: Notification Load: The number of concurrent notification transactions. Transaction: An AIS REST call triggered by a scheduled event from the EnterpriseOne scheduler and REST request from an external system, to the AIS server in the EnterpriseOne architecture to initiate the actions defined in the notification or orchestration. Schedule Event: Notifications configured through the Orchestrator Studio typically include a schedule, normally in minutes, by which the EnterpriseOne Scheduler can operate. After the EnterpriseOne Scheduler starts and the timed event occurs, the EnterpriseOne Scheduler makes a REST notification transaction request to the AIS server. As a reminder, invocations of notifications, such as orchestrations, may start from any inbound REST call, not just from the EnterpriseOne Scheduler. For the purpose of performance, the timing of the scheduled events is important. For example, we may have two notifications, one with a 5-minute schedule and one with a 10-minute schedule. The concurrency at 5 minutes is one notification transaction, but at the 10-minute mark, there is a concurrency of two transactional notifications and as time moves on to the 15-minute mark, the notification transaction rate is again one. Averaged over time, this may not pose a risk or a performance concern, but with increasing notifications and schedules, it can cause spikes in resource needs in the EnterpriseOne architecture. These spikes can result in performance bottlenecks in the architecture. Avoiding these spikes involves carefully planning notification schedules and measuring the concurrent notification transactions. Please refer to the Oracle Application Interface Service (AIS) Performance Characterization paper for information on impact of load on the AIS Server. Recommendation: Scheduled events configured on a PRIME number, Fibonacci series, or other time-based scheme will avoid unintended performance consequences and spikes in notification transaction loads.

Notification Architecture – Runtime The basic architecture depicting the runtime of a notification request is provided in the following diagram. Runtime refers to the time from the notification scheduler event, through the period which the notification is executing, and the EnterpriseOne components that are involved. The diagram also depicts how a normal interactive user accesses the EnterpriseOne application.

7 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Basic Architecture of Notifications In the previous diagram, interactive users communicate with EnterpriseOne though a primary HTML1 server. Interactive users subscribe to notifications using the Subscription Manager application in EnterpriseOne. Those subscriptions are stored in the EnterpriseOne database. As with orchestrations, the AIS server manages the execution of notifications. Notifications in turn may execute orchestrations, invoke Watchlists, or issue simple messages to detect events and gather information to be include in the notification message. To isolate interactive users from any performance impact, Oracle recommends configuring the AIS server with a secondary HTML2 server to handle this workload. When the notification processing completes, the message is stored in the EnterpriseOne database and dispatched to the subscriber according to the chosen delivery method. If a notification is scheduled, the Scheduler will invoke it by sending a REST call to the Notification service on the AIS server. The system administrator can inquire, start, and stop scheduler jobs using a set of Scheduler REST APIs sent from a client such as Curl or Postman. If a notification is not scheduled, then it can be invoked by an external system by sending a REST call to the Notification service on the AIS server. Refer to the REST API for JD Edwards EnterpriseOne AIS Server for more information. Concerning performance, all of the HTML1 server, AIS server, HTML2 server, E1 Logic, and Database server configurations and resource allocations must be sufficient to not only support the interactive users and batch processes but the additional EnterpriseOne notification load. This document covers the performance characterization of the notifications feature on these components in the EnterpriseOne architecture. Designing Notifications Business analysts use Orchestrator Studio to design, test, deploy, and schedule notifications. The Orchestration Studio requires an Oracle Application Development Framework (ADF) application; therefore, it needs its own WebLogic Server and ADF Runtime environment. Refer to the chapter “Creating Notifications with Orchestrator Studio” in the below guide for more information about how to design and deploy notifications: JD Edwards EnterpriseOne Tools Notifications Guide

8 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Notifications, orchestrations, and their subcomponents are managed by EnterpriseOne as user defined objects (UDOs). As designers create and modify notifications and orchestrations using Orchestrator Studio, the Studio automatically stores the objects in the UDO repository. The objects execute on the AIS server, and the Studio also automatically deploys the objects to the AIS server where they are cached. If users are experiencing symptoms whereby changes made to objects in the Studio are not taking effect in the runtime environment, the issue could be stale objects in the AIS cache. See “Clearing Orchestration Cache on the AIS server” in the JD Edwards EnterpriseOne Tools Orchestrator Guide. NOTIFICATIONS – MY SUBSCRIPTIONS

After the creation of the notification and schedule assignment, an EnterpriseOne user may subscribe to that notification through My Subscriptions. A subscription is required in order for an EnterpriseOne user to receive a specific notification message. To create a new subscription, access the Subscription Manager by clicking your login name and selecting My Subscriptions under the Personalization category, as shown in the following figure:

Note: EnterpriseOne users may subscribe to notifications at ANY TIME. No recycling of services is required, and at the next scheduled notification timed interval, notifications for that subscription will be enabled.

NOTIFICATION DELIVERY METHOD

In the following figure, a number of subscriptions used for this performance characterization have already been created. For each subscription, the EnterpriseOne user can specify the delivery mechanism by which the notification message is received.

9 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

There are three different notification delivery methods for messages: Notification List (or notification bell), Work Center, and Email. Although not tested as part of the performance characterization, the email option for notifications can generate a significant load to an SMTP email server with a large number of notification subscribers. The notification designer should anticipate the possible performance impact to their email server and users when this delivery method is used. In testing, the Notification List delivery method provided the same impact to performance as the Work Center delivery method. Although later in this document, we will show that impact to EnterpriseOne architecture components is slightly higher for resources such as CPU and memory, the Notification List delivery method measured performance was of no concern for the EnterpriseOne runtime processing. The Notification List (notification bell) processes notification messages at every EnterpriseOne user login and every configurable Polling Interval (default is 60 seconds). In contrast, the Work Center notification messages write to the EnterpriseOne message table (F01131). Querying the Work Center is normally a manual process through the Work Center application. Frequent login/logout testing for performance impact to the notification list was not part of this performance characterization. Increased CPU and memory consumption of EnterpriseOne components would result from more frequent bell processing. Configuration of one or all three EnterpriseOne notification delivery methods is possible through the Subscription Manager application. The notification designer must evaluate the increased impact to performance if EnterpriseOne users choose multiple delivery mechanisms. This document does not cover the topic of more than one EnterpriseOne notification delivery method. A Notification is an UDO Object Notifications are EnterpriseOne user defined objects (UDOs) and have all the security and management of other UDO objects. Security and management of UDO object actions is through the Work with User Defined Objects EnterpriseOne application (P98220U). Configuration and management of the notification UDO through the EnterpriseOne application is not a performance concern; it is not a runtime component of this specific notification testing.

10 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Notification Runtime Modes There are four available modes of notification runtime. They are: 1.

Run As Subscriber

2.

Run As Subscriber with Overrides

3.

Run As Proxy User

4.

Run As Proxy User with Overrides

This section of the document describes some basic performance impacts that result when choosing the various notification runtime options.

The previous diagram shows the EnterpriseOne notification configuration of Run As Subscriber with No Overrides. When using Run As Subscriber, you may also configure the notification to use overrides from this application.

11 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

The EnterpriseOne notification configuration of Run As Proxy user (Run As Subscriber option off) with Allow Subscriber Overrides enabled is shown in the previous figure. Similarly, you can enable or disable the Allow Subscriber Overrides screen option via the sliding toggle. In the previous example, disabling Run As Subscriber is equivalent to Run As Proxy. There is a distinction between how the notifications execute based on whether you are allowing subscriber overrides while running as proxy. Later sections of this document discuss Run As Subscriber performance. The following four notification examples serve as a reference for the different notification runtime option combinations. Example 1: Company A’s Input Analysis (Run As Proxy; No Overrides)

Company A is sending a reminder message for time entry. Time entry is due at the same time for everyone and all employees use the same time entry system, so running the notification once, and sending the same message to all subscribers will suffice. For this notification, turn OFF the option to Run As Subscriber. No inputs are required. Example 2: Company B’s Input Analysis (Run As Subscriber; No Overrides)

Company B is using a notification based on a Watchlist for batches of month-end adjustment entries needing approval. As a notification designer, you want to make sure that the appropriate accounting managers receive the notification and shortcut to the approval application, so you will run the notifications separately for each subscriber and based on the subscriber’s security setting. For this notification, turn ON the option to Run As Subscriber so that the notification is run separately for each subscribing accounting manager using that subscriber’s user security setting. This notification does not require any input.

12 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Example 3: Company C’s Input Analysis (Run As Subscriber, Allow Overrides)

Because Company C wants to use a single notification for any purchase order, the purchase order number is defined as an input to the notification. Because different subscribers will input different purchase order numbers, you want to run this notification separately for each subscriber and give the subscriber the ability to specify which purchase order they are interested in. For this notification, turn ON the option to Run As Subscriber so that the notification is run separately for each subscriber. Also, turn on the option to Allow Overrides and add an input for the Purchase Order Number so that the subscriber can override this input value in Subscription Manager. When they set up their subscription, they can specify the purchase order they want to track. Example 4: Same Example 3 as above but with a few caveats (Run As Proxy, Allow Overrides)

The previous example of Company C would be more likely to use the Run As Proxy option if there were NO security restrictions to the subscribers and they were accessing the same data sources so the same data sets would be returned if executed as Run As Proxy user. Overrides in this configuration are allowed and have a much lower impact to performance than a Run As Subscriber configuration, so this would be the preferred solution. Run As Proxy is the preferred performance solution (as described in the following section) if the EnterpriseOne user security and data source access is not a concern to the notification designer.

PERFORMANCE IMPACT OF PROXY USER VS RUN AS SUBSCRIBER

As mentioned, if requirements for the Run As Proxy are met (proxy user security is sufficient or data sources are all common for all subscribers), it provides a better performance model for notifications.

As illustrated, the notification request initiated from a scheduled event through the EnterpriseOne Scheduler and initiated on the AIS server follows the same process flow, namely:

13 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Login processing: This includes the overhead of the login action, performing necessary caching, and cleanup after the notification is dispatched. Notification Processing: This includes execution of the notification, determining whether the notification should be dispatched, and processing of user overrides. Dispatch of the Notification: Dispatch of the notification is the sending of the message defined by the notification in the Orchestrator Studio to one of the three delivery methods (Notification List, Work Center, Email). Run As Proxy, whether the overrides are specified or not, requires only a single login processing event. The difference in performance for the overrides option is the increased amount of processing required in the notification processing action for each grouping of overrides that is specified. The performance impact for Run As Subscriber is seen in the increased processing needed in all three of these notification events (Login, Notification Processing, and Notification Dispatch). When Run As Subscriber is selected, these three actions must be performed for each subscriber (subscriber 1 through 5 to the Nth subscriber is illustrated above), every time the notification event occurs. This creates a much greater impact to CPU, required memory, and necessary execution time to complete the notification process. So for a typical notification process flow, where is the time being spent? An analysis of a simple notification provides a foundation upon which to base performance decisions for notifications.

For the simple notification analyzed, immediately after the restart of AIS and HTML services, and before any caching of objects has taken place, the above figure illustrates that the majority of time spent in the login process is 2.48 seconds for the initial instantiation of the login action. Subsequent login requests for the same notification after the initial caching event, the login runtime would be closer to 0.1 seconds (100 ms) for completion of those user logins for a Run As Subscriber option. The question then is how does this notification test scale with increasing subscribers? The following figures analyze the data of a simple notification submission with increasing EnterpriseOne subscribers. All four combinations are presented with the notification configurations of Run As Subscriber, Run As Subscriber With Overrides, Proxy User, and Proxy User with Overrides.

14 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

PROXY USER PERFORMANCE PROFILE

The figure below illustrates that increasing EnterpriseOne subscribers to a single notification increases the notification runtime of the process. Almost all of the increased time was due to the login processing of the notification. In the above example, there was a 2.48 second initial cost to the login for the single proxy user notification—this included the cost of the initial caching of information on the AIS server. All remaining notification subscribers will use this login to receive their notifications. When discussing the initial proxy login processing time, this includes the login action of the single EnterpriseOne user, any caching, cleanup processing, and logout processing of the EnterpriseOne notification process. The below diagram illustrates the notification runtime values for scaling subscribers in a Proxy configuration.

In this simple notification submission for a Proxy subscriber configuration, the scaling from 100 to 400 subscribers shows a fairly linear increase from 0.5 to 2.0 seconds. In other words, all 400 subscribers received their messages within 2.0 seconds. The execution time presented in the graph represents notification runtimes after the initial caching of the notification on the AIS server was performed. The figure represents a steady-state performance profile after the initial caching of proxy user information has been completed. If requirements allow, the Proxy with Overrides is the recommended notification configuration with regard to performance where certain requirements for the proxy user are met. The requirements of user security and identical data sources for the proxy user must be contrasted against the performance benefits to notification runtimes. The option to run as a proxy user with overrides incurs only a single 2.48 seconds for the initial login, but a little more time for notification processing and dispatch of messages than the pure Run As Proxy model. The range of notification runtimes was almost identical to proxy user with no overrides.

15 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

RUN AS SUBSCRIBER PERFORMANCE PROFILE

Recommendations for using Run As Subscriber include the evaluation of performance cost due to having a large number of EnterpriseOne subscribers, as shown here:

The impact to performance as the number of subscribers grows is illustrated in the previous comparison diagrams. This diagram shows the same 400 users subscribed with Run As Subscriber as was tested with the EnterpriseOne Proxy User (with or without overrides) configuration. Analysis demonstrates a vast difference in overhead for the execution of the notification (84 seconds (Run As Subscriber) versus a maximum of approximately 6 seconds (Proxy) for 400 users). This is because in the Run As Subscriber mode the notification login processing is repeated for each individual subscriber. Considering this performance impact, due mainly to the login processing, the notification designer might need to adjust the notification schedule if configured as Run As Subscriber and consider additional impact to server resources if this notification configuration is run frequently, or limit the number of total subscribers, thus login requests, to a small data set. The cost to resource consumption, notification execution time, and EnterpriseOne subscribers needs to be considered when configuring Run As Subscriber vs. Proxy User notification mechanisms. It is recommended that only those users that require strict user security or data source exceptions be included in a notification initiated as Run As Subscriber configuration.

UNDERSTANDING THE ENTERPRISEONE SCHEDULER EnterpriseOne Scheduler Configuration Another consideration when launching the notification schedules through the EnterpriseOne Scheduler is the architecture configuration of the scheduler. Three EnterpriseOne scheduler software architectures are available for notifications, namely: 1.

EnterpriseOne Scheduler In-Memory

16 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

2.

EnterpriseOne Scheduler Resiliency (Fault Tolerance solution)

3.

EnterpriseOne Scheduler High Availability

Each configuration has an impact on performance characterization. The comparison results of the varying EnterpriseOne Scheduler configurations are discussed at greater length later in this document. The EnterpriseOne scheduler is part of the AIS server EnterpriseOne Tools 9.2.2.1 and later releases. The fault tolerance feature of the EnterpriseOne Scheduler is available starting with EnterpriseOne Tools 9.2.2.4 release. A customer may choose to use another scheduler, but for the purposes of discussing different EnterpriseOne architectures, the EnterpriseOne scheduler is the focus of this document.

EnterpriseOne Scheduler Architecture ENTERPRISEONE SCHEDULER IN-MEMORY AND RESILIENCE CONFIGURATIONS

This diagram illustrates the notification architecture for EnterpriseOne scheduler in-memory and resilience configurations:

The EnterpriseOne scheduler in-memory configuration is the default for the AIS server; it is part of the Java process that is included in the AIS server EnterpriseOne component deployment. The benefits of using EnterpriseOne scheduler in the in-memory configuration or configuring EnterpriseOne scheduler for Resilience is outlined below. Refer to the My Oracle Support document for more information: Setting up Quartz Scheduler Resilience to be used with Notifications and Orchestrations (TR 9.2.2.4) (Doc ID 2368608.1) The EnterpriseOne Scheduler in both configurations acts like the CRON utility does in Linux. It only functions as a scheduler to initiate another process when a timed event occurs.

17 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Note: The performance overhead to the Java process for the EnterpriseOne Scheduler is minimal. There was no load impact during testing when launching the scheduled notification events. When deciding to use EnterpriseOne Scheduler In-Memory or Resiliency, consider these facts: EnterpriseOne Scheduler In-Memory •

Available in EnterpriseOne Tools 9.2.2.1.



EnterpriseOne Scheduler information for notification schedules is stored in active Java memory on the AIS server.



EnterpriseOne Scheduler information in memory is LOST if the AIS-managed server goes down or is stopped.



In the event of an AIS server failure or services restart, the restart of the scheduler management for notifications is needed using Curl, Postman, or other REST API calls to the recovered operating AISmanaged server instance.

EnterpriseOne Scheduler Resiliency •

EnterpriseOne Scheduler Resiliency is available in EnterpriseOne Tools 9.2.2.4.



EnterpriseOne Scheduler Resiliency is a fault-tolerant solution, not to be confused with high availability or clustering.



The fault-tolerance load balancing of EnterpriseOne Scheduler Resiliency is configured manually and might need to be adjusted depending on various parameters (discussed later in this document).



EnterpriseOne Scheduler information on schedules is stored in database tables.



EnterpriseOne Scheduler Resiliency configuration is in EnterpriseOne Server Manager.



EnterpriseOne Scheduler information on schedules remains PERSISTENT if the AIS server goes down or the managed instance is stopped.



EnterpriseOne Scheduler management is controlled externally through Curl/Postman REST calls and not EnterpriseOne Server Manager.

18 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

RESILIENT SCHEDULER ARCHITECTURE

The previous diagram provides a visual representation of the function of the scheduler. Note that if a nonEnterpriseOne scheduler is used, then it will not necessarily follow this process flow. For the process 1 action, a REST call is made to initiate the schedule on all of the AIS servers in the EnterpriseOne Resilient Architecture. This is a single call and includes all AIS server and port configuration to the Scheduler. The AIS servers all communicate to the EnterpriseOne database, which provides that central control point for all scheduler activities. The setup of the database schema is a separate step in the configuration of EnterpriseOne resiliency. There is no performance characterization between EnterpriseOne Scheduler In-Memory and Scheduler Resilience configurations presented in this document. The decision of using the EnterpriseOne FAULT TOLERANCE capabilities (Scheduler Resilience) in the EnterpriseOne architecture is part of the requirements review process.

ENTERPRISEONE SCHEDULER HIGH AVAILABILITY CONFIGURATION

Using notifications in a production environment may require a more robust implementation of the EnterpriseOne Scheduler, namely the introduction of a load balancer, such as Oracle Traffic Director (OTD), to enable a high-availability EnterpriseOne architecture.

19 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

The benefits of introducing load-balancing into the EnterpriseOne architecture are: •

Improved load-balancing results in shorter notification runtimes than EnterpriseOne Scheduler Resiliency.



Eliminates the 1:1 mapping between the AIS server and HTML2 server configuration. With the OTD server, the mapping is between the AIS server and OTD, and OTD makes the decision which HTML2 server receives any form and service requests from the AIS server.



Fewer AIS servers can support a larger block of HTML2 servers for processing notifications that have form and service requests. HTML2 servers are typically the performance bottleneck for efficient notification runtimes in the notification process where notifications issue form and service requests. The AIS server and EnterpriseOne Scheduler do not consume a large amount of resources.

NOTIFICATIONS – PERFORMANCE CHARACTERIZATION Up to this point in the document, performance was identified as an important consideration when evaluating notification design and choosing the different notification configuration options. The remainder of the document provides a more in-depth analysis of specific notification load testing. The following performance topics are covered: Notification Types in the Performance Characterization: Discussion of the three types of notifications tested. Notification Scenarios in the Performance Characterization: A brief listing of the 14 notifications used in the performance characterization. Results in this document cover increasing interactive users and notification subscriber loads with these 14 notifications. Notification Load Impact on Interactive Users: This section contains analysis of the notification load described above and the results. Notification Load Impact on Notification Runtimes: This section describes the impact of notification load on notification runtime. Notification List and Work Center Notification Delivery Analysis: Analysis of any performance concerns between the Notification List and Work Center messages. A notification designer should be aware of any performance concerns with this delivery mechanism. •

EnterpriseOne Scheduler Resiliency and Oracle Traffic Director Analysis



EnterpriseOne Notification Tuning and Sizing

Notification Testing Scenarios As a statement of reference, the notification load that was tested was 90% Run As Subscriber and 10% Run As Proxy with notification loads of 100/250/500 subscribers. For example, a 100-user subscriber test had 90 users initiated in the configuration of Run As Subscriber, whereas only 10 users were initiated as the Proxy User. The notification schedule was 10 minutes, which is a heavy load by testing standards. Interactive user loads were also scaled in similar loads of 100/250/500 EnterpriseOne interactive users to simulate typical “day-inthe-life” transactional workload.

20 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

NOTIFICATION SCENARIOS

Notification testing used three scenarios for a baseline comparison for EnterpriseOne interactive users for a comparison for notification subscribers. Performance testing these three baselines for both an EnterpriseOne Scheduler Resiliency and Oracle Traffic Director EnterpriseOne architecture configuration is included in the result analysis of this document. Testing included increasing both the notifications load to 100, 250, and 500, and interactive user loads of 100, 250, and 500, respectively, yielding 12 total scenarios. INTERACTIVE USER SCENARIOS

The Oracle EnterpriseOne Day-in-the-Life (DIL) Kit was used for testing EnterpriseOne notifications. The DIL Kit is a set of automated load-testing scripts for the purpose of generating various interactive loads in an EnterpriseOne architecture environment. The DIL Kit comprises 25 interactive applications across five functional modules, including Customer Service Management, Finance Management, Human Capital Management, Supplier Relationship Management, and Supply Chain Management. Another component of the DIL Kit is the 1.2 TB foundational database that application and batch processes use, designed to represent an average-sized JD Edwards EnterpriseOne customer. Exercising the DIL Kit for 100, 250, and 500 interactive users and comparing the metrics with the notifications feature enabled with scaling numbers of notification subscribers. Metrics such as end-to-end response time and operating system metrics (processor, memory, and network) comprised the bulk of the comparisons used for measuring the performance characterizations. Impact of Notifications on Interactive User Experience Using the DIL Kit for interactive user load and exercising increasing users and increasing subscribers showed a consistently identical profile for both the Average Server Time and the Average End-to-End Time. Regardless of the scaling of interactive users or number of notification subscribers, the performance is nearly identical with regard to the interactive user experience. This expected result is because the notification components that are common between the interactive users and the notification load are the Database server and EnterpriseOne Logic server processes. In the case of the Database server, database server requests from the notification load did not interfere with any of the database requests from the interactive users. Other metrics that confirmed this were the consumption of database process resources and database connections. There was no marked increase in these metrics and the comparison of consumption of Database server CPU and memory resources did not show any marked differences. In the case of the EnterpriseOne logic server tuning for notifications, in anticipation of and resulting from initial unit testing, the EnterpriseOne security kernels increased from a default value of 5 to a tuned value of 10. This was done to handle the additional load that stemmed from the increased number of login security requests that were anticipated from the notification’s Run As Subscriber configuration. Similarly, there were no significant differences between the consumption of CPU and memory between interactive users and the introduction of notification load into the EnterpriseOne architecture. The EnterpriseOne logic server as well as the other EnterpriseOne architecture components used in testing are in Appendix A.

21 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Performance Analysis of Delivering Notifications When subscribing to a notification, the end user (the subscriber) can choose among three delivery channels to receive the notifications: the notification list, the Work Center, and email. While an increased number of email messages will certainly have a performance impact on the SMTP email server that is delivering these messages, performance analysis of that external SMTP server is beyond the scope of this paper. This analysis will focus on the two delivery channels that are internal to the JD Edwards system: the notification list and the Work Center. The metrics used in this testing for performance comparison was a server response time and end user experience. The server response time represents computer-to-computer direct communications, whereas in an end user experience, the metric also includes the browser rendering time. When comparing these metrics for notification list and Work Center messages, the choice for the notification delivery mechanism did not result in any noticeable differences. For the 14 scenarios presented in this testing, representing the increase of both the interactive user load and the number of notification subscribers, in all instances, the levels of response time for both the server processes and the end user experience remained essentially the same. Notification Types Three notification types were included in the performance characterization testing for notifications, namely Watchlists, orchestrations, and simple notifications. A summary of the 14 scenarios used in testing follows. Notifications Based on One View Watchlists Notifications based on Watchlists provide the notification designer with information from a Watchlist, for example, the number of records the Watchlist is showing and whether that number of records exceeds the warning or critical threshold for the Watchlist. This table lists the Watchlist notifications used in the performance characterization:

For the Watchlists in the table above, the associated queries return a range of database record results. Six different applications returning database query results ranging from 1 to 43,584 made up the Watchlist notification design. Notifications Based on Orchestrations Notifications based on orchestrations that included data requests and form service requests comprised the notification testing design for the orchestration notification type.

22 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Simple Notifications A simple notification is a text message of a certain size delivered directly through the notification mechanism. A simple notification does not do any processing, nor does it return any data from the EnterpriseOne application. You can think of a simple notification like a simple email with a subject and body. The only concern in a performance analysis is the payload size.

In the above list, two payloads of 100 bytes and 12 kilobytes comprised the testing of simple notifications. A simple message can be as small as a “working from home (WFH)” message or as large as a 12-kilobyte payload message stating that an account password needs to be changed to comply with company policy. There was no impact to the performance of the notification runtime with the varying load sizes of these simple messages. Notification Delivery Mechanisms The different notification delivery mechanisms of Work Center messages and the notification bell is the topic of this section of the document. The combined set of orchestration, watchlist, and simple notification were exercised concurrently against the delivery mechanisms of notification list and Work Center messages. In all of the notification runtime scenarios, the notification runtime for Work Center messages was marginally larger than the notification runtime for the notification list regardless of the number of notification subscribers. The notification list EnterpriseOne processing was characterized by a simple set of insert commands to the two notification tables F980051 (Bell Status) and F980052 (Notification Execution History). The Work Center messages delivery mechanism, on the other hand, processes additional EnterpriseOne logic as well as the insert into the Work Center Messaging table F01131 (PPAT Message Control File). This is not the case in all notification circumstances; however, as a rule, Work Center messages perform at a marginally higher cost than the Notification List. The conclusion to this comparison between the Work Center messages and Notification List is that the difference is marginal and not a critically important factor when choosing the delivery mechanism for notifications. The email notification delivery mechanism uses an external resource for delivering the mechanism. Because of the wide variety of email configurations, performance, and options, the email notification delivery mechanism was not evaluated for its performance characterization.

23 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

RUN AS SUBSCRIBER VERSUS PROXY USER

The configuration of the above results was to run each of the notifications in a Run As Subscriber mode and to measure the time it took (notification runtime) for all notification subscribers to receive the notification message. Interpreting this data, some EnterpriseOne subscribers will receive the notification immediately, while others will receive the message later in the processing of the notification until all subscribed users have received the notification. This is one of the major limitations of Run As Subscriber performance: an extended runtime before all of the subscribed users can receive the notification message. For scaling large numbers of subscribers where the notification schedule runtime is critical, evaluating running the notification in a Proxy User configuration would be a better choice. As was stated previously in this document, users that require strict user security or data source exceptions should be included in a notification initiated as Run As Subscriber configuration. In a Proxy User configuration, the subscribed users will receive the notification in a much more uniform fashion. EnterpriseOne Scheduler Resiliency and Oracle Traffic Director Analysis This section provides an analysis of the function of the EnterpriseOne Scheduler Resiliency configuration compared to a similar configuration with an Oracle Traffic Director (OTD) Architecture configuration. The difference was the configuration of the AIS server (see EnterpriseOne Notification Tuning) pointing to the OTD server and port instead of an individual HTML server illustrated in the figure below.

In the case of EnterpriseOne Scheduler Resiliency, the scheduler handles on which of the two AIS servers the notification is launched. The scheduler makes this determination based on two criteria. At the time of the scheduled event, the time of initiation of the notification(s), the scheduler will first check the first AIS server available and secondly the number of scheduler threads. The scheduler will then initiate all the scheduled notifications to those threads on the first AIS server. If the number of scheduled notifications exceeds the number of available threads on the first AIS server, then a similar process of initiating the remaining notifications triggered by the scheduler occurs on the second AIS server. This will continue until all the submissions of the notifications are completed. A load balancer is introduced in front of the HTML2 servers because most of the work for notifications occur there. There can be a load balancer architecture configuration in front of the AIS servers as well (this configuration architecture was not part of the testing), but since a single AIS server can handle the load of multiple HTML2 servers, there is no reason to introduce another level of redirection to the architecture. Load balancers are introduced and provide the best benefit in areas of the architecture that are identified as a bottleneck.

24 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

This method of load balancing is a fault-tolerance solution, for it depends solely on the availability of threads on each of the AIS servers to perform its functions. Understanding of the total number of notifications initiated at a given scheduled time and manually limiting the number of threads on each of the AIS servers is the only way to perform any load balancing of notification load to the different AIS server and HTML server one-to-one combination in this architecture configuration. Because most of the processing of the notifications occur on the HTML servers, this is not an ideal configuration for distributing load. The AIS server is limited to only sending a message to direct the traffic of the notification processing to the HTML server assigned to it. For the 14 notifications tested in this configuration, the manual setting of the scheduled threads was set to seven. This way each notification event processed seven notifications on HTML2a and seven notifications on HTML2b. Because notification events were uniform, achieving a good manual balancing of notifications across the HTML2 servers was easy. This may not always be the case for more complex notification schedules, thus a more elegant solution may be required, a solution involving an Oracle Traffic Director server or other load-balancing server hardware components such as an F5 switch. Furthermore, an Oracle Traffic Director server would be required if the notifications to be processed would exceed performance requirements of a notification, or if operating system resource limitations are encountered on a single HTML2 server. To achieve a load-balanced high availability architecture configuration for HTML2 notification traffic, a load balancer such as an Oracle Traffic Director server is required for distributing notification load. In this configuration, whether in an Oracle In-Memory (single AIS server) or Oracle Scheduler Resilience (multiple AIS server) configuration, the load balancing of the processing of the notifications will occur equally to the target HTML2 servers. The testing results presented in this document reflect a comparison of two architecture configurations: an Oracle Scheduler Resiliency only and an Oracle Scheduler Resiliency (two AIS servers) with load balancing the HTML2 load to HTML2a and HTML2b.

In testing with 100 interactive users and 500 subscriptions, a comparison between Oracle Scheduler Resiliency and Oracle Traffic Director is performed. For all of the notifications tested, the Oracle Traffic Director configuration performed slightly better than the Oracle Scheduler Resiliency architecture did. This result demonstrates that although a uniform schedule existed for all of the notifications and optimal tuning of the Oracle Scheduler Resiliency is present, Oracle Traffic Director provided better notification runtime efficiencies, even with the same computing resources of two AIS servers and two HTML2 servers. A system designer, when presented with a requirement of varying notification load to the HTML2 servers, and noting the fact that Oracle Traffic Director performs better than the simple Oracle Scheduler Resiliency, might decide to implement the latter architecture for performance and scaling reasons alone. Scale testing comparison between a single AIS-HTML2 server and two or more AIS-HTML2 combinations in either the Oracle Scheduler Resiliency or Oracle Traffic Director is not part of this document. A notification designer, though, can use the initial points in this document to tune, configure, and design such an architecture as to meet their business requirements and performance needs. With that in mind, here are a few guidelines that will conclude this section of the document. 1.

Oracle Scheduler Resilience requires manual tuning of scheduler threads for balancing loads between AIS servers and their target HTML2 counterparts. For uniform schedules, this configuration can be easy to achieve; however, it is not as easy a task where the notification load varies greatly and performance and server resource contention is a concern.

25 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

2.

The one-to-one mapping in the Oracle Scheduler Resilience configuration will require more server components when scaling as compared to Oracle Traffic Director. In Oracle Scheduler Resilience, there is a one-to-one relationship between the AIS server and target HTML2 server. For Oracle Traffic Director, a single or multiple set of AIS servers can point to n-number of HMTL2 servers to process notifications. This one-to-many relationship between AIS and HTML2 is very scalable for performance and load concerns and requires the least amount of servers and resources.

3.

A single Oracle Traffic Director can manage multiple domains and thus provide more than the AISHTML2 specific server load balancing. Oracle Traffic Director can load balance another area of the EnterpriseOne architecture.

4.

In testing, the AIS server was not consuming a large amount of operating system resources; the majority of the processing work for notifications resided on the HTML2 server. The notification architect should keep this fact in mind and place more hardware and software resources for HTML2 servers or managed instances on the same HTML2 server. The resulting architecture design would be more streamlined and provide a more economical choice for the notification designer.

ENTERPRISEONE NOTIFICATION TUNING This section of the document describes the tuning performed for the performance characterization of the EnterpriseOne notifications. The main goal of the tuning was to optimize the notification runtime, eliminate any identified bottlenecks, and provide a consistent end user experience, commensurate with the increasing interactive user load and increased number of EnterpriseOne subscribers. A combination of the EnterpriseOne architecture and tuning the EnterpriseOne logic server security kernels mentioned in this section achieved the last goal of limiting the impact to EnterpriseOne interactive users. The remainder of this section discusses how to identify and tune the other parameters to avoid performance bottlenecks and choke points in notification processing. The section is broken down into the EnterpriseOne server components that were tuned. The discussion in each of the sections describes the logic behind this tuning. The various tuning values covered in this section are only a few possible values that may be required for the EnterpriseOne notification performance. Performance tuning is an iterative process that depends on factors such as the types of EnterpriseOne notifications defined, number of notification subscribers, and notification concurrency. EnterpriseOne Logic Server Consider tuning the two EnterpriseOne logic server values of EnterpriseOne security kernels and EnterpriseOne call object kernels for EnterpriseOne notification processing performance. Increasing one or both of these values depends on the specific EnterpriseOne notification requirements and configuration. The EnterpriseOne security kernel is responsible for servicing authentication requests. The default recommendation for security kernels is to have one security kernel for every 100 interactive users. A notification using a Proxy User or Proxy User with Overrides configuration will be performing a single authentication for each notification launched. This single authentication of the proxy user serves the notification for all subscribers, regardless of how many; therefore no additional EnterpriseOne security kernels were required for testing.

26 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

For the Run As Subscriber notification configuration, an authentication is required for each of the notification subscribers. In the latter case, more security kernels are required to handle this additional load. To allow for the EnterpriseOne notification security requests, add an additional EnterpriseOne security kernel for every 100 concurrent notification subscribers in order to avoid additional performance tuning. This suggestion is consistent with the recommended best practices of 100 interactive users per EnterpriseOne security kernel. Memory consumption for these additional EnterpriseOne kernel processes is not a major concern. Testing indicated each security kernel only consumed an average of 14 MB of memory even at increasing notification subscriber loads between 100 and 500 concurrent subscribers. For this testing effort, no additional resources were required for the EnterpriseOne call object kernels. Call object kernels process business logic requests from the EnterpriseOne notifications and in particular, notifications based on the two orchestrations did not consume enough call object kernel processing resources to warrant an increase in the number of call object kernels. Evaluate any change in the number of call object by measuring the notification load impact. This evaluation is on a case-by-case situation and there is no easy method of providing a best practice for this setting For the EnterpriseOne notifications defined in this testing, there were no sufficiently large amounts of additional logic processes to require further tuning of this value for the EnterpriseOne logic server. However, if your business requirements dictate notifications based on very complex orchestrations, those orchestrations might require an increase in the number of EnterpriseOne call object kernel processes. EnterpriseOne HTML Server The single most significant recommendation of this performance analysis with respect to the HTML server is to deploy on HTML server (or set of servers) to accommodate the notifications function, apart from the HTML server(s) that accommodate interactive users. In this way, you can isolate materially all impact of the notification function from the interactive users. This paper refers to the HTML server for interactive users as “HTML1” and the HTML server for notifications as “HTML2”. Tuning the HTML Managed Instance Thread Pool Size The EnterpriseOne HTML server configuration change related to EnterpriseOne notifications is the Maximum Thread Pool Size. For the WebLogic server, it was found that large numbers of notifications that were configured to Run As Subscriber and that had large numbers of subscribers required a larger number of threads for each JVM process. This setting is for all HTML2 servers that service form and service requests for notifications. Tuning Java managed instance threads for the HTML process is not available in the EnterpriseOne Server Manager interface; configure the value through the WebLogic Administrative Console as shown in the following figure:

27 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

You can do this for the HTML managed instance in the WebLogic Administrative Console by modifying the -> Tuning -> Advanced -> Self Tuning Thread Maximum Pool Size value. The default for the maximum pool threads is 400 for WebLogic. This maximum value is reached within a single notification of 500 subscribers; the resulting effect is an increase in the notification runtime. Many more thread processes must be available for the EnterpriseOne notification to execute efficiently. It is recommended as an initial setting to be: (Maximum Thread Pool Size) = (# notifications) * (# subscribers for each notification) + 20%

In this example, the rule shown was applied for a high load throughput test of many notifications, all with the Run As Subscriber configuration of 500 subscribers. A value of 10,000 for threads configured in the previous example and used in testing resulted in no increases in notification runtime for high EnterpriseOne notification loads configured for Run As Subscriber. Tuning the HTML2 Managed Instance Maximum Connections The EnterpriseOne HTML2 server connections value used in this architecture handles form service and database request limits. Depending on the notification, increase the available Maximum Connections for handling the additional database requests that perform SQL. Tuning of this parameter is through the EnterpriseOne Server Manager Console as shown in the following figure. Access this from the -> -> Configuration -> Advanced -> Database > JDBj Connection Pools screen.

28 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

The default value of 50 is normally insufficient and will result in a bottleneck of database requests for EnterpriseOne notification processes. Increasing this value will help maintain a consistent EnterpriseOne notification time and avoid any database and form service request timeouts that may occur. For the testing in this performance characterization, we increased this value to 500. EnterpriseOne AIS Server Tuning As mentioned previously, the manual tuning of the EnterpriseOne Scheduler for the EnterpriseOne Scheduler Resiliency architecture is required for the proper functioning of the scheduler across multiple AIS servers. This tuning involves modifying the configuration of the number of EnterpriseOne scheduler threads. Perform the configuration changes of the EnterpriseOne Scheduler through the Server Manager Console for the AIS server instance. To reach the configuration screen shown, navigate the Server Manager console to -> Configuration -> Advanced -> Miscellaneous.

In the example, we are trying to load balance 14 notifications, all with the same schedule across a 2-AIS server, 2-HTML2 server configuration. To force notification job requests from being initiated on a single AIS server, set the Number of Threads to a value lower than the default of 30. Otherwise all of the 14 notifications will initiate on the first AIS server and none will reach the second server. Use the following rule when manually configuring the Number of Threads value. (Number of Threads) = (Number of concurrent notifications in a schedule) / (Number of AIS servers)

29 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

Configuring the number of AIS servers depends greatly on the amount of processing required for each notification. In the testing scenarios for this performance characterization, even though a single AIS server was adequate to handle all the load, two AIS servers were used in the EnterpriseOne architecture to fully exercise the EnterpriseOne Scheduler Resiliency and Oracle Traffic Director architecture configurations. CONFIGURING THE ASSOCIATION BETWEEN AN AIS INSTANCE AND AN HTML SERVER

Configuring the recommended architecture of a second HTML2 server to which the AIS server will direct form and service requests is done through the EnterpriseOne Server Manager Console. Navigate the Server Manager Console to the screen -> Configuration -> Advanced -> HTML2 Information.

The pertinent configuration variables are the HTML server End Point Host Name and Port fields, and the Keep JAS Session Open check box. In this configuration, the standard AIS to HTML2 server one-to-one mapping is specified on port 7777 with Keep JAS Session Open selected. There is a performance benefit to having the Keep JAS Session Open selected. It saves time by not having the application continually establish a new session, create a cache area, and so on, for each initiation. For notifications that may use these sessions frequently, it can save 10-20% in overhead costs for these events.

This configuration is similar to the previous one, but for this configuration, an Oracle Traffic Director (OTD) server is specified. In this configuration, all that is needed is the OTD server name and port. OTD automatically configures which HTML2 server will be used to execute notifications.

30 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

In this example, the AIS configuration is still utilizing an EnterpriseOne Scheduler Resiliency or In-Memory architecture design. However, in this case, the one-to-one mapping is replaced with a load-balancing solution. This makes the HTML2 servers highly available.

CONCLUSION When notifications are enabled according to the recommended architecture, there is minimal impact to the interactive end users. Testing with the Oracle DIL Kit demonstrated that the notification runtime impact was minimal and consistent throughout all the testing and was independent of the scaling load. Because the notifications may run orchestrations, there may be a commensurate load on the system. The Run As Subscriber option may multiply that workload. Therefore, we recommend testing and planning for additional server capacity if the notification load requirement requires heavy Run As Subscriber usage. Notification architectural design, design implementation choices, and the performance characterization of those selections played an important part in this document. For the notification designer, the importance of evaluating the notification schedules, architectural choices of Oracle Scheduler Resilience, choosing to implement a load balancer such as Oracle Traffic Director, and even the notification delivery methods of email, Work Center, or notification bell, and proper tuning play a crucial role in implementing the EnterpriseOne notifications feature. This document’s goal is to educate, through the performance characterization of the EnterpriseOne notifications, so that customers may design and implement this feature with greater knowledge. The result is an implementation that will provide an invaluable additional service and utility to a customer’s business processes.

31 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

APPENDIX A: TEST CONFIGURATION The performance characterization of the testing environment is a JD Edwards EnterpriseOne system. Below are the compute resources used. Machines and Platforms Enterprise Server: • Oracle Enterprise Linux 6 • Oracle Database12c (12.1.0.2) 32-bit Client • 4 VCPUs x Intel Xeon CPU E5-2697 @ 2.90 GHz • 12 GB RAM Database Server: • Oracle Enterprise Linux 6 • Oracle Database12c (12.1.0.2) Enterprise Edition 64 bit • 8 VCPUs x Intel Xeon CPU E5-2697 @ 2.90GHz • 30 GB RAM HTML Server (HTML1, HTML2a, and HTML2b): • Oracle Enterprise Linux 6 • 6 VCPUs x Intel Xeon CPU E5-2697 @ 2.90 GHz • 16 GB RAM • WebLogic Server 12c (12.1.3); Java JDK (1.8) • Single Managed Instance (4 GB Heap Size) AIS Server: • Oracle Enterprise Linux 6 • 4 VCPUs x Intel Xeon CPU E5-2697 @ 2.90 GHz • 16 GB RAM • WebLogic Server 12c (12.1.3); Java JDK (1.8) • Single Managed Instance (4 GB Heap Size) Deployment Server: • Windows 2012 R2 Enterprise Edition • 2 VCPUs x Intel Xeon CPU E5-2697 @ 2.90 GHz • 8 GB RAM Server Manager Console: • Oracle Enterprise Linux 6 • 1 VCPUs x Intel Xeon CPU E5-2697 @ 2.90 GHz • 132 GB RAM OATS Test Controller: • Windows 2012 R2 Enterprise Edition • 8 VCPUs x Intel Xeon CPU E5-2697 2.90 GHz • 32 GB RAM

32 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

• Oracle Application Testing Suite 12.3.0.1.0.376 Software • JD Edwards EnterpriseOne Application 9.2 with Tools 9.2.1.2

33 | ORACLE JD EDWARDS ENTERPRISEONE NOTIFICATIONS – PERFORMANCE CHARACTERIZATION

ORACLE CORPORATION Worldwide Headquarters 500 Oracle Parkway, Redwood Shores, CA 94065 USA Worldwide Inquiries TELE + 1.650.506.7000 + 1.800.ORACLE1 FAX + 1.650.506.7200 oracle.com CONNECT W ITH US Call +1.800.ORACLE1 or visit oracle.com. Outside North America, find your local office at oracle.com/contact. blogs.oracle.com/oracle

facebook.com/oracle

twitter.com/oracle

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0818 JD Edwards EnterpriseOne Notifications – Performance Characterization July 2018