Sep 28, 2016 - 4.5.2 Consistency Confirmation 1 (Transactions to one server for one .... demonstration experiment, and c
Mijin Project Demonstration Experiment Report
August 28, 2016 ver. 1.1
arara Inc.
Table of Contents Mijin Project .............................................................................................................................................................................. 1 Demonstration Experiment Report ............................................................................................................................................ 1 1
Background and Purpose of the Demonstration Experiment............................................................................................. 1
2
Result ............................................................................................................................................................................... 1
3
Outline of the Demonstration Experiment ......................................................................................................................... 3 3.1
Period......................................................................................................................................................................... 3
3.2
Target System ............................................................................................................................................................ 3
3.3
Arara Coin System .................................................................................................................................................... 4
3.3.1 Use Case ................................................................................................................................................................... 4 3.3.2 List of Administrative Functions .............................................................................................................................. 6 3.3.3 List of User Functions .............................................................................................................................................. 6 3.3.4 Screen Transition Diagram ....................................................................................................................................... 7 4
Details of the Demonstration Experiment ......................................................................................................................... 8 4.1
System Configuration ................................................................................................................................................ 8
4.2
Points of the Demonstration Experiment ................................................................................................................. 10
4.3
Advance Preparation................................................................................................................................................ 10
4.4
Demonstration Experimental Method...................................................................................................................... 11
4.5
Demonstration Experiment Description .................................................................................................................. 12
4.5.1 Summary................................................................................................................................................................. 12 4.5.2 Consistency Confirmation 1 (Transactions to one server for one department/service) .......................................... 13 4.5.3 Consistency Confirmation 2 (Transactions to four servers for one department/service)........................................ 15 4.5.4 Consistency Confirmation 3 (Rebooting servers) ................................................................................................... 17 4.5.5 Performance Confirmation 1 (Transactions to one department)............................................................................. 19 4.5.1 Performance Confirmation 2 (Transactions to multiple departments).................................................................... 21 5
Results and Consideration of the Demonstration Experiment ......................................................................................... 23 5.1
Data Consistency ..................................................................................................................................................... 23
5.2
Performance............................................................................................................................................................. 23
5.2.1 Server Load ............................................................................................................................................................ 23 5.2.2 Transaction Approval Confirmation Processing..................................................................................................... 23 5.2.3 Transaction Performance ........................................................................................................................................ 24 6
Conclusion..................................................................................................................................................................... 26
Copyright© 2016 arara Inc.
i
Update History Date
Version
Update Content
August 28,
1.00
Creation of the first edition
1.10
Update of the Result
2016 September 28, 2016
Copyright© 2016 arara Inc.
ii
1 Background and Purpose of the Demonstration Experiment (1) In 2008, a person named himself Satoshi Nakamoto submitted a paper about Bitcoin, a non-centralized and distributed virtual currency system called Blockchain, and in 2009, he released the software of Bitcoin on the Internet and its operation began. (2) Blockchain has the following characteristics: (A) A distributed type that synchronizes the same data on arbitrary multiple computers (B) By binding the transaction data blocks with chains, falsification of data will become very difficult (C) The mechanism in which any number of nodes can participate regardless of PCs and servers connected to the Internet is called the public type. The mechanism that designates an administrator and a company operates as a non-distributed type is called the private type. (3) Mijin is the software developed as the private type by Tech Bureau, Corp. based on NEM, public type Blockchain software published as open source for which the development began as the Blockchain 2.0 Project in January 2014. (4) The usefulness and cost benefit of the system for sending and receiving currency on a many-to-many basis have been verified by other companies’ demonstration experiments, but its usefulness for the system in which high-volume transactions occur on a many-to-one basis, just like our point + plus system, is unknown. (5) So, the purpose is to help clarify the direction of our services and product development by assuming the system configuration that is close to the point + plus system, which is our flagship service, conducting the demonstration experiment, and comprehending the performance, cost, and characteristics of the system. (6) Regarding many-to-many transactions, a wide variety of methods for using other than currency have been proposed but what can be done and what cannot be done are obscure; therefore, obtaining the know-how through this demonstration experiment will be one of the purposes.
References (1) Satoshi Nakamoto: https://en.wikipedia.org/wiki/Satoshi_Nakamoto (2) Bitcoin: https://en.wikipedia.org/wiki/Bitcoin (3) Blockchain : https://en.wikipedia.org/wiki/Blockchain_(database) (4) Tech Bureau mijin: http://mijin.io/en/about-mijin
Copyright© 2016 arara Inc.
1
2 Result Although there are some issues, it is possible to do 3,000 or more transactions per minute without causing inconsistencies, and it has been verified that the mechanism can make it difficult to falsify data and can prevent data loss. We also reached the conclusion that it is sufficiently applicable to the systems in which high-volume transactions occur on a many-to-one basis such as our point + plus. However, as functions other than the functions that mijin can realize need to be absorbed by peripheral systems, securing their credibility and availability will become important.
Copyright© 2016 arara Inc.
2
3 Outline of the Demonstration Experiment 3.1 Period Preparations of the design, development, and set up for the demonstration experiment system June 21 to August 12, 2016 Conduction of the demonstration experiment August 15 to 26, 2016
3.2 Target System We assume the system in which we prepare our own arara coins and our employees can make various kinds of payments in arara with arara coins (hereafter referred to as the “arara coin system”). Mainly, we assume the payments in the service of in house cafeteria called Happy Cafeteria and Office FamilyMart. *There was a proposal to use arara coins as virtual currency that can be used publicly in the future but we did not consider this in this demonstration experiment.
Copyright© 2016 arara Inc.
3
3.3
arara Coin System
3.3.1 Use Case The main use cases of the arara coin system are as follows: The following is the image of a small-scale system. There is one company card and many cards for all employees, the payment between the cards will be processed by the system.
Charging to the employee card (Example)
Employee A
800ARC Administrator
800ARC
Company Card
Employee B
800ARC Employee Card
Employee C
Use by the employee (Example)
8ARC
Employee Employee Card
Administrator Company Card
The employee makes a payment from the website or smartphone before use.
※ARC=arara coin
Copyright© 2016 arara Inc.
4
On the other hand, if the company is large, we will use the following structure to make it possible to control with respect to each department and to avoid transactional concentration on the company card.
Recharging to the employee card (Example)
Departmental
管Employee A
800ARC
Administrator
800ARC Department 800ARC Card
Overall Administrator 10,000ARC
Employee Card
Department B
Company Card
部Employee B
10,000ARC
理Employee A
Departmental
800ARC
Administrator
800ARC
門 Employee B
Department 800ARC Card Department B
社Employee C
Employee Card
員Employee C
Use by the employee (Example)
Departmental Administrator
Employee
8ARC Employee Card
Overall Administrator ??ARC
Department Card
Company Card
The Employee makes a payment from the website or smartphone before use. Transactions between the department and company will not be done at real time but be done in a batch as needed.
Copyright© 2016 arara Inc.
5
3.3.2 List of Administrative Functions #
A1 A2 A3 A4
Implementation in the Demonstration Experiment
Function
Content
Multi-Organization Support Card Issuance
Independent processing and management for each office or department can be done You can issue cards with a unique number system.
Expiration Date Extension of the Expiration Date
Expiration date can be set for each card.
No
You can extend the expiration date when used, etc.
No
A5
Status Change
A6
Recharging Function
A7
Lump-Sum Recharging Function
A8
Totalization Function
A9
Total Amount Adjustment
You can change the status of the specific card such as activation/deactivation and expiration date. An organization can send or receive money to/from a specific card. An organization can send money to multiple specific cards. It can specify the period and display the total of sending/receipt of Mosaic (coins) with respect to each individual card, individual organization, and upper organization. You can add coins (Mosaic) when the coins are about to run out.
Yes No
No Yes Yes No No
3.3.3 List of User Functions Content
Implementation in the Demonstration Experiment
You can use it with a smartphone app (iPhone).
Yes
You can use it with a smartphone app (Android).
Yes
# Function
U2
Compatible with Smartphones Compatible with Smartphones
U3
Payment
U4
Aggregate Calculation
U5
History Sending and Receiving of Money Between Employees
U1
U6
You can easily make payments of 100 yen, 200 yen, etc., that you frequently use and you can also make a payment by designating an amount. You can refer the amounts of the monthly charge and payment. You can refer the monthly charge and payment history. You can send and receive Arara coins to/from another employee.
Copyright© 2016 arara Inc.
Yes No Yes No
6
3.3.4 Screen Transition Diagram
login
operation menu
account information
payment menu
payment history
execute payment
confirm payment
Copyright© 2016 arara Inc.
7
4 Details of the Demonstration Experiment 4.1
System Configuration The following is the system configuration diagram used in the demonstration experiment.
Figure 4-1. Entire System Configuration Diagram Table 4-1. arara Coin Server Specification
Location
amazon aws Tokyo-ap-northeast-1a
Instance Type
t2.medium
CPU
2 cores
Memory
4G
Storage Type
SSD
gp2
Database
PostgreSQL
ver9.5
Copyright© 2016 arara Inc.
8
Figure 4-2. arara Mijin Cloud Configuration Diagram 1 (Low latency: RTT average is about 0.5 msec)
Figure 4-3 arara Mijin Cloud Configuration Diagram 2 (High latency: RTT average is about 75 msec)
Copyright© 2016 arara Inc.
9
Table 4-2. arara Mijin Cloud Server Specification
Location
amazon aws Tokyo-ap-northeast-1a Tokyo-ap-northeast-1b Singapre-ap-southeast-1a Singapre-ap-southeast-1b
Instance Type
m3.large
CPU
2 cores
Memory
7G
Storage
SSD
*When allocated 7 gigabytes to JavaVM, the process frequently went down; therefore, we set 4 gigabytes gp2
Table 4-3. Client PC Specification for Mass Data Transaction
Location
arara headquarters
CPU
Intel Core i3 2 cores 4 threads 3.1 GHz
Memory
12 G
Storage Type
HDD (SATA) 7200 rpm
4.2 Points of the Demonstration Experiment We implemented the demonstration experiment this time by focusing on the following points: 1.
Consistency Whether or not a discrepancy occurs when we do transactions on multiple mijin servers. Whether or not a discrepancy occurs when servers do not synchronize for a period.
2.
Performance Does it have a processing capability for many transactions to one card at a short period and does it have a practical processing capability when processing a large amount between multiple departments? Whether or not a bottleneck occurs between Client → arara coin server and arara coin server → mijin Cloud.
4.3
Advance Preparation We made the following preparations in conducting verifications. Number of Users (Cards)
100,000 users
Number of Departments
1,024 departments
Recharge
About 5.6 million transactions
Copyright© 2016 arara Inc.
10
4.4
Demonstration Experimental Method We conduct the demonstration experiment by installing the test tool (the one separately developed this time) on the test client and changing the parameter. The parameter is as follows: Table 4-4. List of Parameter Items
Number of Users (Cards)
Fixed in all tests (100,000 users)
Number of Departments
Fixed in all tests (1,024 departments)
Client Latency
Latency until the test client sends a request for the next transaction
Number of Client Threads
The parallel degree that the test client concurrently executes transactions
Server Connection Mode
Designate how to connect to multiple mijin servers. -Department/service fixed mode: The mode to always connect to one server when the department and service are the same -Round robin mode: The mode to connect to servers in a sequential order
Network Latency
Network latency between servers
For the test, we conduct transactions with the above parameter for about five minutes and acquire the following data: Server Load
Brief load average
Processing Time
Number of transactions processed per minute
arara Coin Server
arara coin server’s processing time. The time from receiving a request to
Processing Time
receiving mijin’s transaction request response (unapproved state)
mijin Processing Time
The time from sending a transaction request to mijin to receiving the response (unapproved state)
Data Capacity
Data size (volume of data depends on the number of items) *Implement in specific test cases
Copyright© 2016 arara Inc.
11
4.5
Demonstration Experiment Description
4.5.1 Summary Test #
Test Content
Confirmation of the Data
We make payments from 100,000 cards to 1,024 departments. The
Consistency 1
connecting mijin server will be fixed for the department and will confirm the condition where inconsistency is difficult to occur. We randomly select the cards.
Confirmation of the Data
We make payments from 100,000 cards to one department. We randomly
Consistency 2
select the cards, but we conduct transactions for four consecutive times and allocate them to four mijin servers to confirm the condition where inconsistency easily occurs.
Confirmation of the Data
We reboot several servers under the condition of the Confirmation of the
Consistency 3
Data Consistency 2 and, after that, confirm that the data between the servers will be synchronized.
Performance Confirmation 1
We send a large amount of requests from 100,000 cards to one department. We randomly select the cards and allocate them to four mijin servers in a round-robin fashion.
Performance Confirmation 2
We send a large amount of requests from 100,000 cards to 1,024 departments. We randomly select the cards and departments and allocate them in a round-robin fashion.
Copyright© 2016 arara Inc.
12
4.5.2 Consistency Confirmation 1 (Transactions to one server for one department/service) 4.5.2.1 Content
All four mijin servers have the same data. In other words, three servers will synchronize the data processed by one server. Therefore, if we conduct multiple transactions from one card to multiple mijin servers in a short time, there is a concern that we cannot maintain consistency. Therefore, we confirmed that we could maintain consistency with the configuration in which we made transactions for one card to one server first. 4.5.2.2 System Configuration
Figure 4-4. Request to One Server for One Department/Service
Copyright© 2016 arara Inc.
13
4.5.2.3 Transaction Sequence
Figure 4-5. Transaction Sequence 4.5.2.4 Verification Parameter
Number of Users (Cards)
100,000 users
Number of Departments
1,024 departments
Client Latency
0 msec
Number of Client Threads
1
We set one to check consistency
Server Connection Mode
Fixed
We fixed the connection servers for each department/service
Copyright© 2016 arara Inc.
14
4.5.3 Consistency Confirmation 2 (Transactions to four servers for one department/service) 4.5.3.1 Content
All four mijin servers have the same data. In other words, three servers will synchronize the data processed by one server. Therefore, if we conduct multiple transactions from one card to multiple mijin servers in a short time, there is a concern that we cannot maintain consistency. Therefore, we confirmed that we could maintain consistency with the configuration in which we made transactions for one card to four servers first. 4.5.3.2 System Configuration
Figure 4-6. Request to Four Servers for One Department/Service
Copyright© 2016 arara Inc.
15
4.5.3.3 Transaction Sequence
Figure 4-7. Transaction Sequence 4.5.3.4 Verification Parameter
Number of Users (Cards)
100,000 users
Number of Departments
1 department
Client Latency
0 msec
Number of Client Threads
1
We set one to check consistency
Server Connection Mode
Round robin
Connect in a sequential order
Copyright© 2016 arara Inc.
16
4.5.4 Consistency Confirmation 3 (Rebooting servers) 4.5.4.1 Content
All four mijin servers have the same data. We believe that actively conducting the distributed connection is preferable in view of the load distribution and performance. However, there may be a case that if one server goes down, reboots, or recovers, we will not be able to take data consistency between servers and in this case, we need to manually recover it; therefore, there is a concern about operational load. Therefore, we confirm to maintain consistency even if one server repeats rebooting. 4.5.4.2 System Configuration
Figure 4-8. Request to Four Servers for Multiple Departments/Services
Copyright© 2016 arara Inc.
17
4.5.4.3 Transaction Sequence
Figure 4-9. Transaction Sequence 4.5.4.4 Verification Parameter
Number of Users (Cards)
100,000 users
Number of Departments
1 department
Client Latency
0 msec
Number of Client Threads
1
Server Connection Mode
Round robin
Connect in a sequential order
4.5.4.5 Verification Method
While consecutively conducting the above transactions, reboot two of the four servers at fixed intervals. To avoid mijin from appropriately shutting down, turning off the power would be a good reboot method but we decided to realize it by force-quiting (kill-KILL) mijin’s process to save time.
Copyright© 2016 arara Inc.
18
4.5.5 Performance Confirmation 1 (Transactions to one department) 4.5.5.1 Content
We considered the possibility of affecting performance in confirming consistency when conducting concentrated transactions to one card without considering the department and confirmed performance. 4.5.5.2 System Configuration
Figure 4-10. Request to Four Servers for One Department/Service
Copyright© 2016 arara Inc.
19
4.5.5.3 Transaction Sequence
Figure 4-11. Transaction Sequence 4.5.5.4 Verification Parameter
Number of Users (Cards)
100,000 users
Number of Departments
1 department
Client Latency
0 msec
Number of Client Threads
10
We set ten to confirm performance
Server Connection Mode
Round robin
Connect in a sequential order
Copyright© 2016 arara Inc.
20
4.5.1 Performance Confirmation 2 (Transactions to multiple departments) 4.5.1.1 Content
We confirmed the difference in performance in the case of conducting concentrated transactions to one card without considering the department and in the case of distributing the transactions to multiple servers. 4.5.1.2 System Configuration
Figure 4-12. Request to Four Servers for One Department/Service
Copyright© 2016 arara Inc.
21
4.5.1.3 Transaction Sequence
Figure 4-13. Transaction Sequence 4.5.1.4 Verification Parameter
Number of Users (Cards)
100,000 users
Number of Departments
1,024 departments
Client Latency
0 msec
Number of Client Threads
10
We set ten to confirm performance
Server Connection Mode
Round robin
Connect in a sequential order
Copyright© 2016 arara Inc.
22
5 Results and Consideration of the Demonstration Experiment 5.1
Data Consistency In any of the tests, we could not confirm the case in which consistency collapsed and which could not be approved.
However, because the current mijin always charges a fee (xem) when conducting a transaction using Mosaic (arara coin) and there were some events where errors occurred due to insufficient xen balance. As we received an answer from Tech Bureau, Corp. that they just charge fees as a countermeasure against malicious API attacks when placed in a standard and public environment, and it is possible to set the fee as zero; therefore, we determined that this was not particularly a problem. When we conducted Test 4.5.3
Consistency Confirmation 2 (Transactions to four servers for one
department/service), a phenomenon frequently occurred; when the transaction date and time, sender, receiver, and transaction amount were the same, the result of the transaction will be neutral, an abnormal status because they were overlapping transactions. We believe that a mechanism to prevent overlapping transactions functioned on the mijin side when we conducted the same transactions at the same time. We think that we should have conducted tests to do the transactions by changing the transaction amount in each case. Regarding fault tolerance, we rebooted at fixed intervals while conducting transactions but we could confirm from the logs that synchronization normally completed after recovery and the fact that we could confirm past transactions with the recovered server; therefore, it was not particularly a problem.
5.2
Performance
5.2.1 Server Load When the network distance between the arara coin server and mijin server was short and latency was small, the mijin server’s load average did not exceed 2 in any of the tests. Therefore, if it is about 100 transactions per second, we believe that we can do the processing without any problem. On the other hand, the AP server side on the arara coin side had a somewhat higher load than that of the mijin server. We believe the cause was the use of CPU power for calculating hash and signing to create the request data to mijin. The runtime error infrequently occurred on the side of the arara coin server. Its cause is still unknown but the occurrence rate was extremely low; therefore, we believe we can cope with it by retrying.
5.2.2 Transaction Approval Confirmation Processing Initially, we assumed the approval check in Figure 5 below in all tests, but we found out that the runtime error occurred on the client side with high probability when conducting the confirmation request for the target transactions (occurred about 80% of the time). We guess it was the specification to return twenty previous transactions, including the target transaction. If conducting a check for each transaction, the specification would be to obtain and return the data for the number of transactions x 20 cases; therefore, processing would become slow due to the logic related to this condition, which would lead to cause errors. We think this is the future issue. *As shown above, we conducted all tests under the condition of not carrying out the check.
Copyright© 2016 arara Inc.
23
Figure 5-1. Transaction Sequence
5.2.3 Transaction Performance Any of the processing was about the same in terms of performance. The performance was about 3,000 transactions per minute in multiple cards-to-1 department configuration and about 3,800 transactions per minute in multiple cardsto- multiple department configuration. Considering the test data variation, we think we cannot conclude that the characteristics of mijin caused the difference. There is a report that the nominal value is several tens of thousands of transactions per second, and our results this time were low but the server load in both of the tests was not high. Therefore, we think the cause was the lack of processing capacity on the client side or insufficient optimization of the number of threads on the side of the arara coin server. Therefore, there is a possibility of conducting more transactions but we found out that we could do 3,800 transactions per minute and 220,000 transactions per hour in these tests. Table 5. Transaction Performance (Figure 4-2. arara Mijin Cloud Configuration Diagram 1 (Low latency: RTT average is about 0.5 msec)
Number of Transactions (Per minute) arara API Time (Right after the sequence 2- 4) mijin Response Time (Sequence 3-4) mijin Request Delay Approval Time
Multiple Cards-to-One Department About 3,000 transactions
Multiple Cards-to-Multiple Departments About 3,800 transactions
About 600 msec
About 700 msec
About 10 msec
About 10 msec
None None Unmeasurable (Seemed to be completed within several seconds)
Copyright© 2016 arara Inc.
24
Results of the tests under the environment of high latency were strange. Considering the fact that mijin’s response time was about five-fold and latency was about 150-fold, we did not think of it as particularly strange. On the other hand, arara’s API time took as long as about 310 seconds. However, there was no server load at all; therefore, we assumed that there was a bottleneck somewhere, and we investigated the cause. In conclusion, the cause was that a mechanism of not putting an excessive load on the mijin server was coded in the client program sample we used on the arara coin server that NEM development members released and NEM core client API and accumulation of request cues occurred on the client side (use of SleepFuture). Under the environment of low latency, it could still process 3,000 requests per minute without delay but under the environment of high latency, it could not process 3,000 requests per minute and accumulated request cues on the client side, which seemed to end up with an average processing time of 310 seconds. We corrected the program to solve this problem and tested again, and we obtained results under the environment of high latency that were equivalent to the results under the environment of low latency. Table 6. Transaction Performance (Figure 4-3 arara Mijin Cloud Configuration Diagram 2 (High latency: RTT average is about 75 msec) No Program Correction
Multiple Cards-to-One Department About 3,300 transactions
Multiple Cards-to-Multiple Departments About 3,400 transactions
About 310 sec (Increasing
About 310 sec (Increasing
tendency)
tendency)
mijin Response Time (Sequence 3-4)
About 50 msec
About 45 msec
mijin Request Delay
Yes Yes Unmeasurable (Seemed to be completed within several minutes)
Number of Transactions (Per minute) arara API Time (Right after the sequence 2-4)
Approval Time
Table 7. Transaction Performance (Figure 4-3 arara Mijin Cloud Configuration Diagram 2 (High latency: RTT average is about 75 msec) With Program Correction
Multiple Cards-to-One Department Not measured
Multiple Cards-to-Multiple Departments About 3,100 transactions
Not measured
About 114 msec
Not measured
About 108 msec
mijin Request Delay
None
Approval Time
Not measured
None Unmeasurable (Seemed to be completed within several minutes)
Number of Transactions (Per minute) arara API Time (Right after the sequence 2-4) mijin Response Time (Sequence 3-4)
Copyright© 2016 arara Inc.
25
6 Conclusion (1)
We found that consistency did not collapse according to the method of use, and it could stand against about 200,000 transactions per hour.
(2) We found out that it was difficult to make falsification and could ensure data integrity due to the mechanism of data synchronization. (3)
We found out that it had a high level of availability due to the distributed system. (4)
We believe that feasibility of the e-money system is highly possible in terms of consistency and performance.
(5)
Based on the above (1) – (4), the possibility of substantial cost reduction can be expected.
Copyright© 2016 arara Inc.
26