Analysis of Evolution Process of E-commerce System Based on Javaweb
Let's take javaweb as an example to build a simple e-commerce system and see how this system can evolve step by step.
The system has the features:
User Module: User Registration and Management
Commodity Module: Product Display and Management
Trading Module: Creating Transactions and Management
Phase one, stand-alone construction site
At the beginning of the website, we often run all our programs and software on a single machine. At this point we use a container, such as tomcat, jetty, jboos, and then directly use JSP / servlet technology, or use some open source framework such as maven + spring + struct + hibernate, maven + spring + springmvc + mybatis; finally choose a database Manage the system to store data, such as mysql, sqlserver, oracle, and then connect and operate the database through JDBC.
Put all the above software on the same machine, the application runs, it is a small system. The system results are as follows:
Phase II, separation of application server and database
With the launch of the website, the amount of traffic is gradually increasing, and the load on the server is slowly increasing. When the server is not overloaded, we should be prepared to improve the load capacity of the website. If our code level is difficult to optimize, adding a machine is a good way to improve the performance of a single machine, not only can effectively improve the load capacity of the system, but also cost-effective.
What is the added machine used for? At this point we can separate the database and web server, which not only improves the load capacity of a single machine, but also improves the disaster tolerance.
The architecture of the application server and the database is separated as shown below:
Phase III, application server cluster
As the number of visits continues to increase, a single application server is no longer sufficient. Assuming that the database server is not under pressure, we can change the application server from one to two or more, and spread the user's request to different servers to improve the load capacity. There is no direct interaction between multiple application servers, they rely on the database to provide external services. The famous failover software is keepalived. Keepalived is a software similar to layer3, 4, and 7 switch. It is not a specific software failover, but a product that can be applied to various software. Keepalived with ipvsadm can do load balancing, it can be described as an artifact.
Let's take the example of adding an application server. The added system structure is as follows:
The system has evolved here, and the following four problems will occur:
1. Who is the user's request forwarded to the specific application server?
2. What is the forwarding algorithm?
3. How does the application server return the user's request?
4. If the server is different every time, how to maintain the consistency of the session?
Let's take a look at the solution to the problem:
1. The first problem is the problem of load balancing. There are generally five solutions:
1. http redirect. HTTP redirection is the request forwarding of the application layer. The user's request has actually arrived at the HTTP redirect load balancing server. The server requires the user to redirect according to the algorithm. After the user receives the redirect request, the user requests the real cluster again.
Advantages: Simple.
Disadvantages: poor performance.
2. DNS domain name resolution load balancing. DNS domain name resolution load balancing is when the user requests the DNS server to obtain the IP address corresponding to the domain name, the DNS server directly gives the server IP after load balancing.
Advantages: Handed over to DNS, no need to maintain a load balancing server.
Disadvantages: When an application server hangs, DNS cannot be notified in time, and the control of DNS load balancing is in the domain name service provider, the website can not do more improvement and more powerful management.
3. Reverse proxy server. When the user's request arrives at the reverse proxy server (has arrived at the website), the reverse proxy server forwards it to the specific server according to the algorithm. Commonly used apache, nginx can act as a reverse proxy server.
Advantages: Simple deployment.
Disadvantages: Proxy servers can become a performance bottleneck, especially when uploading large files at once.
4. IP layer load balancing. After the request arrives at the load balancer, the load balancer implements the request forwarding by modifying the destination IP address of the request to achieve load balancing.
Advantages: Better performance.
Disadvantages: The bandwidth of the load balancer becomes the bottleneck.
5. Data link layer load balancing. After the request arrives at the load balancer, the load balancer performs load balancing by modifying the requested mac address. Unlike IP load balancing, when the request is accessed, the server directly returns to the client. No need to go through the load balancer.
2. The second problem is the cluster scheduling algorithm problem. There are 10 common scheduling algorithms.
1. rr polls the scheduling algorithm. As the name implies, polling for distribution requests.
Advantages: simple implementation
Disadvantages: do not consider the processing power of each server
2. wrr weighted scheduling algorithm. We set the weight of each server, the load balancing scheduler dispatches the server according to the weight, and the number of times the server is called is proportional to the weight.
Advantages: Consider the difference in server processing power
3. sh original address hash: extract the user IP, get a key according to the hash function, and then check the corresponding value according to the static mapping table, that is, the target server IP. If the target machine is overloaded, it will return empty.
4. dh destination address hash: Same as above, except that the IP of the destination address is now extracted for hashing.
Advantages: Both of the above algorithms can achieve the same user access to the same server.
5. lc is the least connected. The request is forwarded to the server with a small number of connections.
Advantages: Make the load of each server in the cluster more uniform.
6. wlc weights the least connection. On the basis of lc, add weights to each server. The algorithm is: (active connection number * 256 + inactive connection number) ÷ weight, the calculated value is small, the server is selected first.
Advantages: Requests can be assigned based on the capabilities of the server.
7. sed the shortest expected delay. In fact, sed is similar to wlc, the difference is that it does not consider the number of inactive connections. The algorithm is: (active connection number +1) * 256 ÷ weight, the server with the same calculated value is preferred.
8. nq never queues. Improved sed algorithm. We think about what can be "never queued", that is, when the number of connections to the server is 0, then if the number of server connections is 0, the equalizer forwards the request directly to it without going through the calculation of sed.
9. LBLC is based on the least connection of locality. The equalizer finds the server whose IP address has been used recently according to the requested destination IP address, and forwards the request. If the server is overloaded, the least connection number algorithm is used.
10. LBLCR with locality-based minimal connections with replication. The equalizer finds the "server group" that the IP address has recently used according to the requested destination IP address. Note that it is not a specific server, and then selects a specific server from the group with a minimum number of connections. Request forwarding. If the server is overloaded, then according to the least connection number algorithm, in the server of the non-server group of the cluster, find a server to join, join the server group, and then forward the request.
3, the third problem is the cluster mode problem, generally three solutions:
1. NAT: The load balancer receives the user's request and forwards it to the specific server. After the server processes the request and returns it to the equalizer, the equalizer returns to the user again.
2. DR: The load balancer receives the user's request and forwards it to the specific server. The server returns to the user directly after playing the request. The system needs to support the IP Tunneling protocol, which is difficult to cross-platform.
3. TUN: Same as above, but without IP tunneling protocol, cross-platform is good, most systems can support.
4, the fourth problem is the session problem, there are generally four solutions:
1. Session Sticky. Session sticky is to assign the same user's request in a certain session to a fixed server, so we do not need to solve the cross-server session problem. The common algorithm is ip_hash, which is mentioned above. Two hashing algorithms.
Advantages: Simple implementation.
Disadvantages: The session disappears when the application server restarts.
2. Session Replication. Session replication is to copy the session in the cluster, so that each server holds the session data of all users.
Advantages: To alleviate the stress of the load balancing server, there is no need to implement the ip_hasp algorithm to forward requests.
Disadvantages: Broadband overhead when copying, if the amount of access is large, the session takes up a lot of memory and is wasted.
3. Session data centralized storage: session data centralized storage is to use the database to store session data, to achieve the decoupling of session and application server.
Advantages: Compared to the session replication solution, the pressure on the broadband and memory between the clusters is much reduced.
Disadvantages: need to maintain a database to store sessions.
4. Cookie Base: The cookie base is to have the session in the cookie. There is a browser to tell the application server what my session is. It also implements the decoupling of the session and the application server.
Advantages: Simple to implement, basically maintenance free.
Disadvantages: cookie length limit, low security, broadband consumption.
It is worth mentioning that:
The load balancing algorithms currently supported by nginx are wrr, sh (supporting consistent hashing), and fair (I think it can be attributed to lc). However, if nginx is used as an equalizer, it can also be used as a static resource server.
Keepalived+ipvsadm is more powerful. Currently supported algorithms are: rr, wrr, lc, wlc, lblc, sh, dh
Keepalived supports cluster mode: NAT, DR, TUN
Nginx itself does not provide a solution for session synchronization, while apache provides support for session sharing.
Ok, after solving the above problems, the structure of the system is as follows:
Phase IV, database read and write separation
Above we always assume that the database load is normal, but as the number of visits increases, the load on the database is slowly increasing. Then someone may immediately think of the same as the application server, and then load the database into two and then load balance. But for the database, it is not that simple. If we simply split the database into two, and then the request for the database, respectively, to the A machine and the B machine, then it will obviously cause the problem of the two database data is not uniform. So for this situation, we can first consider the use of read and write separation.
The structure of the database system after reading and writing is as follows:
This structural change will also bring two problems:
1. Data synchronization between master and slave databases
2. Application selection for data sources
Solution to the problem:
1. We can use master/slave with MYSQL to implement master-slave replication.
2. Use third-party database middleware, such as mycat. Mycat was developed from cobar, and cobar is the database middleware of Ali open source, and then stopped development. Mycat is a relatively good domestic mysql open source database sub-library middleware.
Stage 5, using search engines to ease the pressure of reading the library
If the database is used as a reading library, it is often unclear to the fuzzy search power. Even if the reading and writing separation is done, this problem has not been solved. Take the trading website we take as an example. The published products are stored in the database. The most common function used by users is to find the products, especially the corresponding products according to the title of the products. For this kind of demand, we generally use the like function to achieve, but this method is very expensive. At this point we can use the search engine's inverted index to complete.
Search engines have the following advantages:
It can greatly improve the speed of the query.
The introduction of search engines will also bring the following overhead:
Bringing a lot of maintenance work, we need to implement the index construction process ourselves, and design a full/increased build method to deal with non-real-time and real-time query requirements.
Need to maintain a search engine cluster
The search engine is not a substitute for the database. He solves the problem of "reading" in some scenarios. Whether to introduce a search engine requires a comprehensive consideration of the needs of the entire system. The system structure after the introduction of the search engine is as follows:
Stage 6, use the cache to ease the pressure of reading the library
1, the background application layer and database layer cache
As the number of visits increases, there are gradually many users accessing the same part of the content. For these more popular content, it is not necessary to read from the database every time. We can use caching techniques, such as google's open source caching technology guava or use memcacahe as the application layer cache, or redis as the database layer cache.
In addition, in some scenarios, the relational database is not very suitable, for example, I want to do a "restriction of the number of passwords per day error" function, the idea is probably when the user logs in, if the login error, record the user The IP and the number of errors, then where is this data to be placed? If it is placed in memory, it will obviously take up too much content; if it is placed in a relational database, then it is necessary to create a database table, but also a corresponding java bean for the resume, but also write SQL and so on. To analyze the data we want to store, it is nothing more than key:value data like {ip:errorNumber}. For this kind of data, we can use the NOSQL database instead of the traditional relational database.
2, page cache
In addition to data caching, there is also a page cache. For example, use HTML5's localstroage or cookie.
advantage:
Reduce the pressure on the database
Significantly increase access speed
Disadvantages:
Need to maintain the cache server
Increased coding complexity
It is worth mentioning that:
The scheduling algorithm for the cache cluster differs from the application server and database mentioned above. It is best to use a "consistent hash algorithm" to improve the hit rate. This will not be discussed. If you are interested, you can check the relevant information.
Join the cached structure:
Phase VII, database horizontal split and vertical split
Our website has evolved to the present, and the data of transactions, commodities, and users are still in the same database. Although the method of increasing cache and reading and writing is adopted, as the pressure of the database continues to increase, the bottleneck of the database becomes more and more prominent. At this time, we can have two options: vertical data splitting and horizontal splitting.
7.1, data vertical split
Vertical splitting means splitting different business data in the database into different databases. Combined with the current example, the data of transactions, commodities, and users are separated.
advantage:
Solved the problem of putting all the business in one database.
More optimizations can be made according to the characteristics of the business
Disadvantages:
Need to maintain multiple databases
problem:
1. Need to consider the original cross-business transactions
2. Cross-database join
Solution to the problem:
1. We should try to avoid cross-database things in the application layer, if you want to cross the database, try to control in the code.
2. We can solve it through third-party applications. As mentioned above, mycat provides a rich cross-database join scheme. For details, please refer to the official documentation of mycat.
The structure after vertical splitting is as follows:
7.2, data level split
Data level splitting is the splitting of data from the same table into two or more databases. The reason for the data level split is that the data volume or update amount of a certain service reaches the bottleneck of a single database, and the table can be split into two or more databases.
advantage:
If we can customer service, then we will be able to improve the amount of data and writes.
problem:
1. The application system that accesses user information needs to solve the problem of SQL routing, because the user information is now divided into two databases, and it is necessary to know where the data to be operated is when performing data operations.
2. The processing of the primary key also becomes different. For example, the original is from the added field, and now you can't simply continue to use it.
3. If you need paging, you are in trouble.
Solution to the problem:
1. We can still solve third-party middleware, such as mycat. Mycat can parse our SQL through the SQL parsing module, and then forward the request to a specific database according to our configuration.
2. We can solve this by UUID guaranteeing a unique or custom ID scheme.
3. mycat also provides a rich paging query scheme, such as first paging query from each database, then merging data to do a paging query and so on.
Structure after data level splitting:
Phase VIII, application split
8.1, split application
With the development of business, more and more businesses are being used, and applications are getting bigger and bigger. We need to think about how to avoid getting the application getting more and more bloated. This requires the application to be taken apart, from one application to two or more. Still with our example above, we can separate users, goods, and transactions. Become the "user, commodity" and "user, transaction" two subsystems.
Split structure:
problem:
1. After splitting this way, there may be some identical code, such as user-related code, goods and transactions that require user information, so the code that manipulates the user information is retained in both systems. How to ensure that these codes can be reused is a problem that needs to be solved.
Solve the problem:
1. Solve by taking the service route
8.2, take the road of service
In order to solve the problems that arise after splitting the application above, we split the public service to form a service model, referred to as SOA.
Adopt the system structure after service:
advantage:
The same code is not scattered in different applications, these implementations are placed in various service centers to make the code better maintained.
We put the interaction of the database in each service center, so that the "front-end" web application pays more attention to the interaction with the browser.
problem:
1. How to make a remote service call
Solution:
1. We can solve this problem by introducing the following message middleware
Phase IX, the introduction of message middleware
As the site continues to evolve, sub-modules for different language developments and subsystems deployed on different platforms may appear in our systems. At this point we need a platform to deliver reliable, platform- and language-independent data, and to be able to transparently load balance, collect call data and analyze it during the call, and guess the website's access growth rate, etc. Demand, predicting how the site should grow. Open source messaging middleware has Ali's dubbo, which can be used with Google's open source distributed program coordination service zookeeper to achieve server registration and discovery.
The structure after the introduction of message middleware:
Ten, summary
The above evolution process is only an example and is not suitable for all websites. In fact, the website evolution process is closely related to its own business and different problems encountered, and there is no fixed pattern. Only a careful analysis and constant exploration can find the architecture that suits your website.
DVI, known as Digital Visual Interface, is a Digital Display Working Group DDWG(Digital Display Working Group) launched in 1999 by Silicon Image, Intel(Intel), Compaq(Compaq), IBM, HP(HP), NEC, Fujitsu(Fujitsu) and other companies. Its appearance is a 24-pin connector. Display device adopts DVI interface has basically has the following two advantages: one, the high speed: DVI is digital signal transmission, do not need any conversion, digital image information will be transmitted directly to the display device, thus reducing the digital and analog to digital conversion process trival, greatly save the time, so it is faster, effectively eliminate the phenomenon of ghosting, DVI and used for data transmission, signal attenuation, no colour more pure, more vivid.
Second, the picture is clear: the computer inside is A binary digital signal transmission, using A VGA interface LCD connection requires the first signal through D/A in the graphics card, digital/analog converter into R, G, B color signal and line, A synchronous signal, the signal through the analog signal transmitted to the LCD inside also need corresponding A/D (analog/digital) converter will once again into A digital signal to analog signal in the image is shown on the LCD. In the above D/A, A/D conversion and signal transmission process, signal loss and interference will inevitably occur, resulting in image distortion and even display error. However, DVI interface does not need to carry out these conversions to avoid signal loss and greatly improve the clarity and detail expression of the image.
Rgb To Dvi,Dvi To Sdi,Dvi Lcd Controller For 7 Inch,Dvi Input For Auo Lvds Lcd
Shenzhen Zhenniu Technology Co., Ltd , https://www.tydisplay.com