国产av日韩一区二区三区精品,成人性爱视频在线观看,国产,欧美,日韩,一区,www.成色av久久成人,2222eeee成人天堂

Home Database Mysql Tutorial How to Efficiently Select Random Rows in PostgreSQL?

How to Efficiently Select Random Rows in PostgreSQL?

Jan 21, 2025 am 05:32 AM

How to Efficiently Select Random Rows in PostgreSQL?

PostgreSQL efficient random row selection method

To select random rows in PostgreSQL, the best method depends on the size of the table, available indexes, and the level of randomness required.

For a very large table with 500 million rows and a numeric ID column (e.g. id):

  • Fastest method:

    • Use CTE and random() functions to generate random IDs within the ID space.
    • Join the generated ID with the table using the id column.
    • Filter out duplicates and remove redundant IDs. 
WITH params AS (
   SELECT 1       AS min_id,           -- 最小id
        , 5100000 AS id_span          -- 四舍五入。(max_id - min_id + buffer)
)
SELECT *
FROM  (
   SELECT p.min_id + trunc(random() * p.id_span)::integer AS id
   FROM   params p
        , generate_series(1, 1100) g  -- 1000 + buffer
   GROUP  BY 1                        -- 去除重復(fù)項(xiàng)
) r
JOIN   big USING (id)
LIMIT  1000;                          -- 去除多余項(xiàng)

  • Improvement method:

    • Use recursive CTE (random_pick) to eliminate any gaps in the ID space.
    • Merge recursive results to eliminate duplicates.
    • Apply externalLIMIT to satisfy constraints. 
WITH RECURSIVE random_pick AS (
   SELECT *
   FROM  (
      SELECT 1 + trunc(random() * 5100000)::int AS id
      FROM   generate_series(1, 1030)  -- 1000 + 百分之幾 - 根據(jù)需要調(diào)整
      LIMIT  1030                      -- 查詢(xún)規(guī)劃器提示
      ) r
   JOIN   big b USING (id)             -- 消除缺失

   UNION                               -- 消除重復(fù)項(xiàng)
   SELECT b.*
   FROM  (
      SELECT 1 + trunc(random() * 5100000)::int AS id
      FROM   random_pick r             -- 加上百分之三 - 根據(jù)需要調(diào)整
      LIMIT  999                       -- 小于1000，查詢(xún)規(guī)劃器提示
      ) r
   JOIN   big b USING (id)             -- 消除缺失
   )
TABLE  random_pick
LIMIT  1000;  -- 實(shí)際限制

  • General functions:

    • Wrap the above queries into a function so they can be reused for any table with unique integer columns. 
CREATE OR REPLACE FUNCTION f_random_sample(_tbl_type anyelement
                                         , _id text = 'id'
                                         , _limit int = 1000
                                         , _gaps real = 1.03)
  RETURNS SETOF anyelement
  LANGUAGE plpgsql VOLATILE ROWS 1000 AS
$func$
DECLARE
   _tbl text := pg_typeof(_tbl_type)::text;
   _estimate int := (...);
BEGIN
   RETURN QUERY EXECUTE format(
   $$
   WITH RECURSIVE random_pick AS (
      SELECT ...
      FROM  ...
     ...
   )
   TABLE  random_pick
   LIMIT  ;
   $$
 , _tbl, _id
   )
   USING (...);
END
$func$;

For scenarios that don’t require precise randomness or repeated calls:

  • Materialized view:

    • Create a materialized view to store approximately randomly selected rows.
    • Refresh materialized views periodically.

  • TABLESAMPLE SYSTEM (n):

    • Introduced in PostgreSQL 9.5, TABLESAMPLE SYSTEM (n) provides a fast and inexact random sampling method.
      • The
    • n parameter represents the percentage of tables to be sampled. 
SELECT * FROM big TABLESAMPLE SYSTEM ((1000 * 100) / 5100000.0);

Other notes:

  • For best performance, use an index on the ID column.
  • The random() functions in PostgreSQL are not cryptographically secure.
  • The proposed approach provides a high degree of randomness for most practical use cases.

The above is the detailed content of How to Efficiently Select Random Rows in PostgreSQL?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undress AI Tool

Undress AI Tool

Undress images for free

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

What is GTID (Global Transaction Identifier) and what are its advantages? What is GTID (Global Transaction Identifier) and what are its advantages? Jun 19, 2025 am 01:03 AM

GTID (Global Transaction Identifier) ??solves the complexity of replication and failover in MySQL databases by assigning a unique identity to each transaction. 1. It simplifies replication management, automatically handles log files and locations, allowing slave servers to request transactions based on the last executed GTID. 2. Ensure consistency across servers, ensure that each transaction is applied only once on each server, and avoid data inconsistency. 3. Improve troubleshooting efficiency. GTID includes server UUID and serial number, which is convenient for tracking transaction flow and accurately locate problems. These three core advantages make MySQL replication more robust and easy to manage, significantly improving system reliability and data integrity.

What is a typical process for MySQL master failover? What is a typical process for MySQL master failover? Jun 19, 2025 am 01:06 AM

MySQL main library failover mainly includes four steps. 1. Fault detection: Regularly check the main library process, connection status and simple query to determine whether it is downtime, set up a retry mechanism to avoid misjudgment, and can use tools such as MHA, Orchestrator or Keepalived to assist in detection; 2. Select the new main library: select the most suitable slave library to replace it according to the data synchronization progress (Seconds_Behind_Master), binlog data integrity, network delay and load conditions, and perform data compensation or manual intervention if necessary; 3. Switch topology: Point other slave libraries to the new master library, execute RESETMASTER or enable GTID, update the VIP, DNS or proxy configuration to

How to connect to a MySQL database using the command line? How to connect to a MySQL database using the command line? Jun 19, 2025 am 01:05 AM

The steps to connect to the MySQL database are as follows: 1. Use the basic command format mysql-u username-p-h host address to connect, enter the username and password to log in; 2. If you need to directly enter the specified database, you can add the database name after the command, such as mysql-uroot-pmyproject; 3. If the port is not the default 3306, you need to add the -P parameter to specify the port number, such as mysql-uroot-p-h192.168.1.100-P3307; In addition, if you encounter a password error, you can re-enter it. If the connection fails, check the network, firewall or permission settings. If the client is missing, you can install mysql-client on Linux through the package manager. Master these commands

How to add the MySQL bin directory to the system PATH How to add the MySQL bin directory to the system PATH Jul 01, 2025 am 01:39 AM

To add MySQL's bin directory to the system PATH, it needs to be configured according to the different operating systems. 1. Windows system: Find the bin folder in the MySQL installation directory (the default path is usually C:\ProgramFiles\MySQL\MySQLServerX.X\bin), right-click "This Computer" → "Properties" → "Advanced System Settings" → "Environment Variables", select Path in "System Variables" and edit it, add the MySQLbin path, save it and restart the command prompt and enter mysql--version verification; 2.macOS and Linux systems: Bash users edit ~/.bashrc or ~/.bash_

What are the transaction isolation levels in MySQL, and which is the default? What are the transaction isolation levels in MySQL, and which is the default? Jun 23, 2025 pm 03:05 PM

MySQL's default transaction isolation level is RepeatableRead, which prevents dirty reads and non-repeatable reads through MVCC and gap locks, and avoids phantom reading in most cases; other major levels include read uncommitted (ReadUncommitted), allowing dirty reads but the fastest performance, 1. Read Committed (ReadCommitted) ensures that the submitted data is read but may encounter non-repeatable reads and phantom readings, 2. RepeatableRead default level ensures that multiple reads within the transaction are consistent, 3. Serialization (Serializable) the highest level, prevents other transactions from modifying data through locks, ensuring data integrity but sacrificing performance;

What are the ACID properties of a MySQL transaction? What are the ACID properties of a MySQL transaction? Jun 20, 2025 am 01:06 AM

MySQL transactions follow ACID characteristics to ensure the reliability and consistency of database transactions. First, atomicity ensures that transactions are executed as an indivisible whole, either all succeed or all fail to roll back. For example, withdrawals and deposits must be completed or not occur at the same time in the transfer operation; second, consistency ensures that transactions transition the database from one valid state to another, and maintains the correct data logic through mechanisms such as constraints and triggers; third, isolation controls the visibility of multiple transactions when concurrent execution, prevents dirty reading, non-repeatable reading and fantasy reading. MySQL supports ReadUncommitted and ReadCommi.

Why do indexes improve MySQL query speed? Why do indexes improve MySQL query speed? Jun 19, 2025 am 01:05 AM

IndexesinMySQLimprovequeryspeedbyenablingfasterdataretrieval.1.Theyreducedatascanned,allowingMySQLtoquicklylocaterelevantrowsinWHEREorORDERBYclauses,especiallyimportantforlargeorfrequentlyqueriedtables.2.Theyspeedupjoinsandsorting,makingJOINoperation

Where does mysql workbench save connection information Where does mysql workbench save connection information Jun 26, 2025 am 05:23 AM

MySQLWorkbench stores connection information in the system configuration file. The specific path varies according to the operating system: 1. It is located in %APPDATA%\MySQL\Workbench\connections.xml in Windows system; 2. It is located in ~/Library/ApplicationSupport/MySQL/Workbench/connections.xml in macOS system; 3. It is usually located in ~/.mysql/workbench/connections.xml in Linux system or ~/.local/share/data/MySQL/Wor

See all articles