Backend Development
PHP Tutorial
How to Avoid Memory Exhaustion Errors when Using file_get_contents() with Large Files?
How to Avoid Memory Exhaustion Errors when Using file_get_contents() with Large Files?
Oct 17, 2024 pm 01:43 PM
File_get_contents Memory Exhaustion: A Comprehensive Solution
When dealing with large file processing, the infamous PHP Fatal error: Allowed memory exhausted error can be a recurring issue. This problem arises when file_get_contents() attempts to read the entire contents of a sizeable file into memory, often exceeding the allocated memory limit.
Alternative to file_get_contents()
Instead of loading the entire file into memory, a more efficient approach is to open the file as a pointer and read it in smaller chunks using fread(). This allows for memory management, which is critical for handling large files.
Below is a custom function that mimics the functionality of Node.js's file processing API:
<code class="php">function file_get_contents_chunked($file, $chunk_size, $callback)
{
try {
$handle = fopen($file, "r");
$i = 0;
while (!feof($handle)) {
call_user_func_array($callback, array(fread($handle, $chunk_size), &$handle, $i));
$i++;
}
} catch (Exception $e) {
trigger_error("file_get_contents_chunked::" . $e->getMessage(), E_USER_NOTICE);
return false;
}
fclose($handle);
return true;
}</code>
This function accepts three parameters: the file path, the desired chunk size, and a callback function that will be called for each chunk read.
Usage of Custom Function
The file_get_contents_chunked() function can be used as follows:
<code class="php">$success = file_get_contents_chunked("my/large/file", 4096, function ($chunk, &$handle, $iteration) {
/* Process the chunk here... */
});</code>
Regex Considerations
Performing multiple regex operations on a large chunk of data is inefficient. Consider using native string manipulation functions like strpos(), substr(), trim(), and explode().
Example Cleanup
Instead of:
<code class="php">$newData = str_replace("^M", "", $myData);</code>
Use:
<code class="php">$pattern = '/\r\n/'; $replacement = ''; $newData = preg_replace($pattern, $replacement, $myData);</code>
By utilizing the aforementioned techniques, it is possible to effectively process large files without encountering memory exhaustion errors.
The above is the detailed content of How to Avoid Memory Exhaustion Errors when Using file_get_contents() with Large Files?. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1794
16
1739
56
1590
29
1467
72
267
587
How to upgrade PHP version?
Jun 27, 2025 am 02:14 AM
Upgrading the PHP version is actually not difficult, but the key lies in the operation steps and precautions. The following are the specific methods: 1. Confirm the current PHP version and running environment, use the command line or phpinfo.php file to view; 2. Select the suitable new version and install it. It is recommended to install it with 8.2 or 8.1. Linux users use package manager, and macOS users use Homebrew; 3. Migrate configuration files and extensions, update php.ini and install necessary extensions; 4. Test whether the website is running normally, check the error log to ensure that there is no compatibility problem. Follow these steps and you can successfully complete the upgrade in most situations.
PHP beginner guide: Detailed explanation of local environment configuration
Jun 27, 2025 am 02:09 AM
To set up a PHP development environment, you need to select the appropriate tools and install the configuration correctly. ①The most basic PHP local environment requires three components: the web server (Apache or Nginx), the PHP itself and the database (such as MySQL/MariaDB); ② It is recommended that beginners use integration packages such as XAMPP or MAMP, which simplify the installation process. XAMPP is suitable for Windows and macOS. After installation, the project files are placed in the htdocs directory and accessed through localhost; ③MAMP is suitable for Mac users and supports convenient switching of PHP versions, but the free version has limited functions; ④ Advanced users can manually install them by Homebrew, in macOS/Linux systems
Steps to configure a PHP development environment on Linux
Jun 30, 2025 am 01:57 AM
TosetupaPHPdevelopmentenvironmentonLinux,installPHPandrequiredextensions,setupawebserverlikeApacheorNginx,testwithaPHPfile,andoptionallyinstallMySQLandComposer.1.InstallPHPandextensionsviapackagemanager(e.g.,sudoaptinstallphpphp-mysqlphp-curlphp-mbst
How to combine two php arrays unique values?
Jul 02, 2025 pm 05:18 PM
To merge two PHP arrays and keep unique values, there are two main methods. 1. For index arrays or only deduplication, use array_merge and array_unique combinations: first merge array_merge($array1,$array2) and then use array_unique() to deduplicate them to finally get a new array containing all unique values; 2. For associative arrays and want to retain key-value pairs in the first array, use the operator: $result=$array1 $array2, which will ensure that the keys in the first array will not be overwritten by the second array. These two methods are applicable to different scenarios, depending on whether the key name is retained or only the focus is on
How do I prevent cross-site request forgery (CSRF) attacks in PHP?
Jun 28, 2025 am 02:25 AM
TopreventCSRFattacksinPHP,implementanti-CSRFtokens.1)Generateandstoresecuretokensusingrandom_bytes()orbin2hex(random_bytes(32)),savethemin$_SESSION,andincludetheminformsashiddeninputs.2)ValidatetokensonsubmissionbystrictlycomparingthePOSTtokenwiththe
php regex for password strength
Jul 03, 2025 am 10:33 AM
To determine the strength of the password, it is necessary to combine regular and logical processing. The basic requirements include: 1. The length is no less than 8 digits; 2. At least containing lowercase letters, uppercase letters, and numbers; 3. Special character restrictions can be added; in terms of advanced aspects, continuous duplication of characters and incremental/decreasing sequences need to be avoided, which requires PHP function detection; at the same time, blacklists should be introduced to filter common weak passwords such as password and 123456; finally it is recommended to combine the zxcvbn library to improve the evaluation accuracy.
How to handle File Uploads securely in PHP?
Jul 08, 2025 am 02:37 AM
To safely handle PHP file uploads, you need to verify the source and type, control the file name and path, set server restrictions, and process media files twice. 1. Verify the upload source to prevent CSRF through token and detect the real MIME type through finfo_file using whitelist control; 2. Rename the file to a random string and determine the extension to store it in a non-Web directory according to the detection type; 3. PHP configuration limits the upload size and temporary directory Nginx/Apache prohibits access to the upload directory; 4. The GD library resaves the pictures to clear potential malicious data.
How Do Generators Work in PHP?
Jul 11, 2025 am 03:12 AM
AgeneratorinPHPisamemory-efficientwaytoiterateoverlargedatasetsbyyieldingvaluesoneatatimeinsteadofreturningthemallatonce.1.Generatorsusetheyieldkeywordtoproducevaluesondemand,reducingmemoryusage.2.Theyareusefulforhandlingbigloops,readinglargefiles,or
