高效的MySQL分頁
Jun 07, 2016 pm 04:36 PMPERCONA PERFORMANCE CONFERENCE 2009上,來自雅虎的幾位工程師帶來了一篇”Efficient Pagination Using MySQL“的報告,有很多亮點,本文是在原文基礎上的進一步延伸。 分頁 首先看一下分頁的基本原理: 本文來自leo108's blog mysql explain SELECT * FROM
PERCONA PERFORMANCE CONFERENCE 2009上,來自雅虎的幾位工程師帶來了一篇”Efficient Pagination Using MySQL“的報告,有很多亮點,本文是在原文基礎上的進一步延伸。分頁
首先看一下分頁的基本原理:
本文來自leo108's blog
mysql> explain SELECT * FROM message ORDER BY id DESC LIMIT 10000, 20\G
***************** 1. row **************
id: 1
select_type: SIMPLE
table: message
type: index
possible_keys: NULL
key: PRIMARY
key_len: 4
ref: NULL
rows: 10020
Extra:
1 row in set (0.00 sec)
http://leo108.com/pid-1914.asp
limit 10000,20的意思掃描滿足條件的10020行,扔掉前面的10000行,返回最后的20行,問題就在這里,如果是limit 100000,100,需要掃描100100行,在一個高并發(fā)的應用里,每次查詢需要掃描超過10W行,性能肯定大打折扣。文中還提到limit n性能是沒問題的,因為只掃描n行。
本文來自http://leo108.com
文中提到一種”clue”的做法,給翻頁提供一些”線索”,比如還是SELECT * FROM message ORDER BY id DESC,按id降序分頁,每頁20條,當前是第10頁,當前頁條目id最大的是9527,最小的是9500,如果我們只提供”上一頁”、”下一頁”這樣的跳轉(zhuǎn)(不提供到第N頁的跳轉(zhuǎn)),那么在處理”上一頁”的時候SQL語句可以是:
本文來自http://leo108.com
SELECT * FROM message WHERE id > 9527 ORDER BY id ASC LIMIT 20;
處理”下一頁”的時候SQL語句可以是:http://leo108.com/pid-1914.asp
SELECT * FROM message WHERE id <p>?</p><p class="Gar360">本文來自leo108's blog </p> <p>不管翻多少頁,每次查詢只掃描20行。</p><p class="Gar360">Mysql</p> <p>缺點是只能提供”上一頁”、”下一頁”的鏈接形式,但是我們的產(chǎn)品經(jīng)理非常喜歡”5?6 7 8 9 下一頁>”這樣的鏈接方式,怎么辦呢?<span class="Gar360">分頁</span> </p> <p>如果LIMIT m,n不可避免的話,要優(yōu)化效率,只有盡可能的讓m小一下,我們擴展前面的”clue”做法,還是SELECT * FROM message ORDER BY id DESC,按id降序分頁,每頁20條,當前是第10頁,當前頁條目id最大的是9527,最小的是9500,比如要跳到第8頁,我看的SQL語句可以這樣寫:</p><p class="Gar360">分頁</p> <pre class="brush:php;toolbar:false">SELECT * FROM message WHERE id > 9527 ORDER BY id ASC LIMIT 20,20;
跳轉(zhuǎn)到第13頁:
本文來自leo108's blog
SELECT * FROM message WHERE id <p>?<span class="Gar360">采集者爛JJ</span> </p> <p>原理還是一樣,記錄住當前頁id的最大值和最小值,計算跳轉(zhuǎn)頁面和當前頁相對偏移,由于頁面相近,這個偏移量不會很大,這樣的話m值相對較小,大大減少掃描的行數(shù)。其實傳統(tǒng)的limit m,n,相對的偏移一直是第一頁,這樣的話越翻到后面,效率越差,而上面給出的方法就沒有這樣的問題。</p><p class="Gar360">http://leo108.com/pid-1914.asp</p> <p>注意SQL語句里面的ASC和DESC,如果是ASC取出來的結果,顯示的時候記得倒置一下。</p><p class="Gar360">Mysql</p> <p>已在60W數(shù)據(jù)總量的表中測試,效果非常明顯。<span class="Gar360">分頁</span> </p> <p>轉(zhuǎn)自:http://www.fuchaoqun.com/2009/04/efficient-pagination-using-mysql/</p><p class="Gar360">Mysql</p>

Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

mysqldump is a common tool for performing logical backups of MySQL databases. It generates SQL files containing CREATE and INSERT statements to rebuild the database. 1. It does not back up the original file, but converts the database structure and content into portable SQL commands; 2. It is suitable for small databases or selective recovery, and is not suitable for fast recovery of TB-level data; 3. Common options include --single-transaction, --databases, --all-databases, --routines, etc.; 4. Use mysql command to import during recovery, and can turn off foreign key checks to improve speed; 5. It is recommended to test backup regularly, use compression, and automatic adjustment.

When handling NULL values ??in MySQL, please note: 1. When designing the table, the key fields are set to NOTNULL, and optional fields are allowed NULL; 2. ISNULL or ISNOTNULL must be used with = or !=; 3. IFNULL or COALESCE functions can be used to replace the display default values; 4. Be cautious when using NULL values ??directly when inserting or updating, and pay attention to the data source and ORM framework processing methods. NULL represents an unknown value and does not equal any value, including itself. Therefore, be careful when querying, counting, and connecting tables to avoid missing data or logical errors. Rational use of functions and constraints can effectively reduce interference caused by NULL.

GROUPBY is used to group data by field and perform aggregation operations, and HAVING is used to filter the results after grouping. For example, using GROUPBYcustomer_id can calculate the total consumption amount of each customer; using HAVING can filter out customers with a total consumption of more than 1,000. The non-aggregated fields after SELECT must appear in GROUPBY, and HAVING can be conditionally filtered using an alias or original expressions. Common techniques include counting the number of each group, grouping multiple fields, and filtering with multiple conditions.

MySQL paging is commonly implemented using LIMIT and OFFSET, but its performance is poor under large data volume. 1. LIMIT controls the number of each page, OFFSET controls the starting position, and the syntax is LIMITNOFFSETM; 2. Performance problems are caused by excessive records and discarding OFFSET scans, resulting in low efficiency; 3. Optimization suggestions include using cursor paging, index acceleration, and lazy loading; 4. Cursor paging locates the starting point of the next page through the unique value of the last record of the previous page, avoiding OFFSET, which is suitable for "next page" operation, and is not suitable for random jumps.

To view the size of the MySQL database and table, you can query the information_schema directly or use the command line tool. 1. Check the entire database size: Execute the SQL statement SELECTtable_schemaAS'Database',SUM(data_length index_length)/1024/1024AS'Size(MB)'FROMinformation_schema.tablesGROUPBYtable_schema; you can get the total size of all databases, or add WHERE conditions to limit the specific database; 2. Check the single table size: use SELECTta

MySQL supports transaction processing, and uses the InnoDB storage engine to ensure data consistency and integrity. 1. Transactions are a set of SQL operations, either all succeed or all fail to roll back; 2. ACID attributes include atomicity, consistency, isolation and persistence; 3. The statements that manually control transactions are STARTTRANSACTION, COMMIT and ROLLBACK; 4. The four isolation levels include read not committed, read submitted, repeatable read and serialization; 5. Use transactions correctly to avoid long-term operation, turn off automatic commits, and reasonably handle locks and exceptions. Through these mechanisms, MySQL can achieve high reliability and concurrent control.

Character set and sorting rules issues are common when cross-platform migration or multi-person development, resulting in garbled code or inconsistent query. There are three core solutions: First, check and unify the character set of database, table, and fields to utf8mb4, view through SHOWCREATEDATABASE/TABLE, and modify it with ALTER statement; second, specify the utf8mb4 character set when the client connects, and set it in connection parameters or execute SETNAMES; third, select the sorting rules reasonably, and recommend using utf8mb4_unicode_ci to ensure the accuracy of comparison and sorting, and specify or modify it through ALTER when building the library and table.

To set up asynchronous master-slave replication for MySQL, follow these steps: 1. Prepare the master server, enable binary logs and set a unique server-id, create a replication user and record the current log location; 2. Use mysqldump to back up the master library data and import it to the slave server; 3. Configure the server-id and relay-log of the slave server, use the CHANGEMASTER command to connect to the master library and start the replication thread; 4. Check for common problems, such as network, permissions, data consistency and self-increase conflicts, and monitor replication delays. Follow the steps above to ensure that the configuration is completed correctly.
