Performance Benchmark MySQL on Cloud

Do MySQL instances with the same specifications (e.g., 4 vCPU and 16 GB) from different cloud providers offer the same performance? This series aims to explore and answer this question by Sysbench benchmark.

threads/qps	aliyun	tencent	huawei	baidu	aws	azure	google	oracle
4	7102	5592	2557	2206	1639	2025	723	3551
8	9702	9936	4674	4101	3313	3654	1341	5936
16	14660	16141	8229	7298	6427	6548	2502	8054
32	22155	22336	13520	12022	12157	10363	4857	8317
48	27905	24770	17849	16448	16516	11973	6745	8130
64	32704	26495	20114	18187	18118	12761	8071	7838
96	36846	29077	20883	21007	20782	13300	9675	8504
128	39697	29918	20128	21029	22446	13388	10620	8198
192	38999	30610	20521	22091	22590	13478	11507	8043
256	38356	31052	21187	21665	22323	12985	11872	7907
384	39679	31224	21729	21167	21902	12904	12131	8209
512	40333	31805	22647	21627	21591	12930	12106	8386
have_ssl	DISABLED	DISABLED	DISABLED	DISABLED	YES	YES	YES	YES
innodb_buffer_pool_size	9.75GB	12GB	9GB	12GB	11GB	12GB	11GB	17GB
innodb_doublewrite	ON	ON	ON	ON	OFF	OFF	ON	ON
innodb_flush_log_at_trx_commit	1	1	1	1	1	1	1	1
innodb_flush_method	O_DIRECT	O_DIRECT	O_DIRECT	fsync	O_DIRECT	fsync	O_DIRECT	O_DIRECT
innodb_io_capacity	20000	20000	12000	2000	200	200	5000	1250
innodb_read_io_threads	4	4	4	8	4	NA	4	2
innodb_write_io_threads	4	4	4	8	4	NA	4	4
log_bin	ON	ON	ON	ON	OFF	ON	ON	ON
performance_schema	OFF	OFF	OFF	OFF	OFF	ON	ON	ON
rpl_semi_sync_master_enabled	ON	ON	ON	ON	NA	NA	NA	NA
rpl_semi_sync_master_timeout	1000	10000	10000	10000	NA	NA	NA	NA
sync_binlog	1	1	1	1000	1	1	1	1
thread_pool_size	8	4	NA	NA	NA	4	NA	16
version	8.0.36	8.0.30-txsql	8.0.28-231003	8.0.32-2.0.0.2	8.0.35	8.0.37-azure	8.0.31-google	8.0.39-cloud
cpu_capacity	80.4	93.3	163.6	73.9	110.9	56.3	49.9	114.7

How to repeat the benchmark ?

You can repeat the benchmark with the same sysbench parameters and the same instance shape in the cloud provider

MySQL instance specifications

In this test, we selected the managed MySQL services (RDS MySQL) from Alibaba Cloud, Huawei Cloud, Tencent Cloud, AWS, Azure, Oracle Cloud, and Google Cloud. The specifications for the test instances meet the following conditions:

4 vCPU and 16 GB memory, with 100 GB of storage
If IOPS needs to be specified, it is set to 3000
High availability across multiple availability zones
Very high data reliability, using synchronous replication, semi-synchronous replication, MGR replication, or synchronous replication at the storage layer
Excellent performance consistency (dedicated computing resources)

Cloud Provider	MySQL specifications
Amazon RDS	db.m6i.xlarge
Oracle Cloud	MySQL.4
Google Cloud	db-custom-4-16384
Alibaba Cloud	mysql.x4.large.2c
Tencent Cloud	cpu = 4; mem_size = 16000; device_type=EXCLUSIVE
Azure	GP_Standard_D4ds_v4
Huawei Cloud	rds.mysql.x1.xlarge.4.ha
Baidu Cloud	`cpu_count = 4 memory_capacity = 16`

Sysbench CMD and parameters

The latest Sysbench is used, which actually is the master branch on GitHub .

lua benchmark file : oltp_read_write_with_hooks.lua (which is almost the same as the sysbnech default oltp_read_write.lua)
--time=300
--table_size=10000 --tables=10
--skip_trx=on --db-ps-mode=disable
--rand-type=uniform
--threads=[4|8|16|32|48|64|96|128|192|256|384|512]

sysbench oltp_read_write_with_hooks --threads=$conthread --time=$run_time \
 --report-interval=3 --percentile=95 --histogram=on --db-driver=mysql \
 $sysb_mysql_conn \
 --skip_trx=on --db-ps-mode=disable --rand-type=uniform $ssl_param \
 --table_size=$table_size --tables=$tables run >> $run_file 2>&1

Limitation of the benchmark

This section provides additional notes on some limitations of the benchmark

In this series of benchmark, the region selection was relatively random. For example, Alibaba Cloud was tested in Hangzhou, Baidu Cloud in Beijing, AWS/Google Cloud in Tokyo, and Azure Cloud in the Eastern US, Hong Kong, and Tokyo. The underlying assumption here is that the RDS MySQL performance across different regions of each cloud provider should be the same or similar.
Due to personal time and resource constraints, only the commonly used 4 vCPU 16 GB instance type was tested, with each single concurrent test lasting 300 seconds.
Although MySQL 8.0 was used in all cases, the minor database versions differed among the cloud providers.
The CPU, disk types, and pricing vary across providers, so this is not a perfectly equal comparison, nor could it be.
The default parameter templates of RDS instances also vary between providers, and even a single provider may adjust these parameters at different stages.
Some cloud providers offer multiple disk storage options. For instance, Alibaba Cloud supports ESSD PL1/2/3, and AWS offers gp3/io1 storage options. These different storage choices result in different performance outcomes.
Regarding the choice of availability zones: In the tests, I tried to place the ECS/EC2/VM instances and the database master nodes in the same availability zone whenever possible. However, there were cases where this wasn’t feasible. For example, during the tests, GCP’s availability zone b in the Tokyo region allowed the creation of the database but had no resources available to create VM nodes (“A n2-highcpu-8 VM instance is currently unavailable in the asia-northeast1-b zone.”).

more about the benchmark

The main goal of the test is to understand the performance differences of MySQL across different cloud providers with the same specifications. This helps developers adapt to the architecture and performance of RDS when migrating between different cloud providers.
About the Author "orczhou" : Oracle ACE (MySQL focused), Co-founder of NineData, and former Senior Database Expert at Alibaba.