Cloud Performance Test Service (CPTS) provides performance test services for cloud applications that are built based on HTTP, HTTPS, TCP, UDP, WEBSOCKET, RTMP or HLS. CPTS quickly simulates service peaks with a large number of concurrent users. It allows you to define the contents and time sequences of packets and supports different combinations of multiple transactions for complex scenario tests. After tests are complete, CPTS provides professional test reports to evaluate your service quality.
You can complete a performance test in four steps listed as follows.
- Test Project: CPTS manages test projects that you created. Test transactions, pressure test tasks, and test reports are shared in a test project. You can create test projects for different test programs.
- Transaction: A transaction indicates a user-defined test operation model, which includes four parts: HTTP/HTTPS/TCP/UDP/WEBSOCKET/RTMP/HLS packet, think time, response extraction, and checkpoint.
- Packet: Packets are data blocks transmitted between HTTP-based applications. These data blocks start with text metadata that describes the packet content and meaning. The text metadata is followed by optional data. These packets are transmitted among clients, servers, and agents.
- Think Time: To better simulate user behavior, it is necessary to simulate the waiting time between different operations. For example, when a user receives data from a server, the user may wait for several seconds to view the data before responding. This delay is called Think Time.
- Response Extraction: If multiple packets exist in the same transaction, use regular expressions to extract the output of the previous packet for the input of the next packet.
- Checkpoint: Checkpoints are used to verify whether contents returned by servers are correct through self-defined verification information.
- Number of Concurrent Users: In performance tests, the number of concurrent users refers to the number of virtual users. Be aware of the differences between the number of concurrent users and the number of online users. The former causes pressure on the server, while the latter does not.
- TPS: The number of transactions per second is an important metric for measuring the system performance. The larger the value is, the better the performance is. For example, the TPS value is 1 if a user completes 1 transaction within 1s. The value is 1000 if a user completes 1000 transactions within 1s.
Acceptable TPS values vary depending on industries and services. Generally, the TPS value for e-commerce, small-sized, and medium-sized websites is 10,000–100,000, 50–100, and 100–500 respectively.
- Response Time: It refers to the duration from the time when a user initiates a request to the time when the user receives a response from the server. In performance tests, the duration from the time when the pressure is initiated to the time when the server returns the processing result is measured. The unit is second or millisecond. This duration is different from the user experience time simulated in the real environment.
The response time varies depending on industries and services. Generally, the acceptable response time for Internet, financial, and insurance enterprises is less than 500 ms, 1s, and 3s respectively.