[JAVA] How Dubbo-based Microservices Change Insurer Architecture

China Life's Xiaobin shares how Dubbo has changed its architecture and is considering migrating to ** Alibaba Cloud **.

• This blog is a translation from the English version. You can check the original from here. We use some machine translation. We would appreciate it if you could point out any translation errors. *

About Dubbo

In 2013, China Life Insurance announced the [RPC Framework](https://searchapparchitecture.techtarget.com/definition/Remote-Procedure-Call-RPC?spm=a2c65.11461447.0.0.552b21f2f6yvpZ] to transform the entire business database. ) Was looking for. At that time, there were few mature products on the market, such as Spring Cloud and Dubbo. There were many products on the market such as .apache.org/en-us/?spm=a2c65.11461447.0.0.552b21f2f6yvpZ). What we were looking for was a framework that was widely applied in a production environment. Dubbo has been introduced in Taobao for a long time, and Alibaba's business model was comparable to our business model. For example, our business model requires us to respond to requests from multiple regions overseas, each with its own business requirements.

With that in mind, we started using Dubbo in 2013 as well. We started business systems in Hong Kong and Macau in 2016, and in Indonesia in May 2019. In the future, as we are using Dubbo, we plan to rapidly expand our business system throughout Southeast Asia, including Singapore.

Dubbo has helped me a lot in terms of streamlining deployments and reducing costs. In this article, I'll discuss the pre-Dubbo architecture that is common to many traditional insurers, and then talk about how Dubbo was used and how the architecture changed.

What insurance companies have done in the past

When it comes to server hardware, we usually use IBM and HP minicomputers (https: // www), as many other traditional insurance business systems use. It was .techopedia.com/definition/4615/minicomputer). Only these two systems were available at the time, both running UNIX-based operating systems.

Regarding business architecture, client / server architecture has been mainly used in software development so far. Under such an architecture, a powerful middleware called Tuxedo is a distributed transaction. It was used for management and provided excellent consistency and high currency performance. So why do we need to replace Tuxedo in the first place? The first reason is high prices, the second reason is the lack of relevant O & M personnel, the third reason is more business integration, and performance and scheduling in distributed and cross-platform scenarios. Is to get worse. The fourth reason is that Tuxedo is designed for monolithic applications. As a result, there is no intention of splitting.

How to transform your mission-critical business system

This figure shows the advantages of microservices over monolithic applications. Dubbo serves for the interaction between different components, which are similar to the convex and concave parts of Lego toys. It implements communications management and service governance. Supported by this design, it breaks through the large traditional core architecture.

OneLife is our business support platform. As the name implies, one platform can handle all insurance operations, form an in-house ecosystem and support several different areas such as imaging, new services, storage and claims settlement. We also provide engines such as workflow, product engine, underwriting engine and message engine. These business functions are essential for any insurance company.

Insurance business processing platform developed by Dubbo

With the support of Dubbo, we have built a new insurance business processing platform to realize “Six Multi” insurance business processing. Six Multi represents the idea of having multiple business systems, product lineups, regulations to follow, engines, and currency languages. For example, to generate an insurance policy, you need to decide whether to generate it in English, Chinese, or Indonesian. Of course, for that purpose, it is necessary to cooperate with each business division, and it is also necessary to design and develop an appropriate processing platform.

Formation of OneLife distributed system

As shown in the figure, a closed loop can be formed with the following four parts. Dubbo-based microservices call, Jenkins -based continuous integration, Rancher a2c65.11461447.0.0.552b21f2f6yvpZ) -based cloud deployment application, Pinpoint-based chain monitoring. We have been discussing with Alibaba Cloud about the Indonesian version of cloud deployment. In the future, Indonesian systems may be the first to move to Alibaba Cloud. In terms of pinpoint chain monitoring, the topology displayed is very easy to understand, but based on this topology, we can see some problems such as broken formations and failed calls. .. This year, with the help of Pinpoint chain monitoring, we were able to see over 100 bugs. The combination of Pinpoint chain monitoring and Dubbo is excellent.

How Dubbo Disperses in Hong Kong and Macau

Dubbo has more than 150 servers, more than 210 applications, more than 2,100 consumers and more than 1,300 providers distributed across Hong Kong and Macau. Insurance companies, especially their business systems, do not have many frequent transactions. Most high-frequency trading occurs on the front end. Business systems are not always in a high frequency state, but it is still important that all business systems are output stably and accurately. This is one of the reasons I chose Dubbo.

How Dubbo Works China Life Insurance (Overseas) Co., Ltd.

About 70% of our work uses the early Dubbo version (specifically 2.4.9). Previously, we made some code fixes based on Dubbo, such as compensation for distributed transactions. After that, I found out that the version could not be upgraded due to a major code modification. Then, you may be wondering what framework is used for the remaining 30% of business. I'm just trying the latest version on peripherals. In addition, this version has been applied to the line-of-business architecture after it has been proven to be practical and feasible. Currently, Dubbo of China Life Insurance (Overseas) Co., Ltd. receives more than 21 million calls a day, and there have been no system crashes since the introduction of Dubbo.

Dubbo structure

We will share the configuration here. Two issues are highlighted. The first problem is the retry mechanism. The problem with this is that if the service is interrupted, it can be manually intervened using the control platform, or if the service is important, it can be compensated by repeating the transaction. The second challenge is that using the ZooKeeper Registration Center has the disadvantage of enormous peak network consumption. The Dubbo version 2.7 metadata concept is promising. In the future, I would like to try using Dubbo version 2.7 or later to see if I can overcome the shortcomings of ZooKeeper or optimize ZooKeeper.

Dubbo Microservices Application Scenario

The figure above shows a Dubbo microservices application. The application starts on the online services page and goes to the new service component. After that, the workflow is launched and the underwriting result is queried. Based on the underwriting result, the premium calculation result is automatically queried and returned to the front end. Low cost is a prerequisite for deploying mission-critical and sales systems in Indonesia. You also need to separate the business units. This involves the Base development model. What is the Base development model?

If your business is spread across multiple regions, you need to have a unique version for each region. If the published basic version is the same, you can improve the efficiency of your business. For example, the headquarters has basic services, but Indonesia has its own regulations. If there is such demand in Hong Kong in the future, we will roll back the basic version and release different basic versions for each region once the basic version has passed a certain examination. In short, in complex situations, the base version supports hierarchical separation of different business logics, hierarchical code management, and hierarchical and regional service governance.

Suggestion

First, it enhances visual control.

Second, it deploys and packages service grids to provide more functionality in microservices systems such as network calls, traffic restrictions, line blockages, and inter-service monitoring.

Third, it supports multiple languages. Dubbo currently offers PHP, Node.js, Python and Go clients, but plans to support more in the future.

Fourth, we do not recommend Dubbo to support distributed transaction management. When I first used Dubbo, I decided that I needed to support distributed transactions. Therefore, I changed the code based on Dubbo. The application was smooth and the code was able to ensure transactional consistency across platforms. However, when the service failed, all the transactions it sent collapsed, creating a fatal risk to the database.

Therefore, I think it is desirable to have Dubbo increase support for the message mechanism and develop business while compensating for distributed transactions. These are some ideas and suggestions that came up through practical application.

Frequently Asked Questions

Q1: Is the process of replacing monolithic applications with microservices components step-by-step or bulk? Also, how much manpower and resources do you need to complete the replacement process?

A1: First, the existing structure is modular and the modules are independent of each other. For the most important data structures, you need to separate your business logic. Then set the database shards and permissions and gradually replace the modules. Prior to development, the entire replacement process should be well planned. When we launched the first module, there were only five people, but all were technically skilled and familiar with their respective fields of business. Getting the support of management was a top priority.

Q2: How can I detect data inconsistencies when distributed transaction control is not available in my system? And how can this problem be solved?

A2: Pinpoint-based chain monitoring allows O & M personnel to quickly detect problems, fix failures, and then perform manual intervention. The MQ system is used for the key service. However, this mechanism consumes a lot of time and manpower. We believe that large-scale distributed transaction control should be avoided.

Q3: If the amount of data in the company is large, how should I replace the Oracle database when migrating data from the old system to the new system? Can you answer this question?

A3: At the database level, it is necessary to confirm that the table structure and data of the Oracle database match those of other databases. Tools such as ETL can be used for comparison. At the application level, there are two steps you can take. The first step is to automate the process. You can use the self-compilation tool to execute your application's SQL statements on both databases and fix the errors in a timely manner. The third execution will allow almost all SQL statements to be executed successfully. The next step is to spotcheck key algorithms and interfaces and compare the data output by the new interface with the business data.

• Alibaba Cloud is the No. 1 (2019 Gartner) cloud infrastructure operator in the Asia Pacific region with two data centers in Japan and more than 60 availability zones in the world. Click here for more information on Alibaba Cloud. Alibaba Cloud Japan Official Page *