How to download a file from a URL and save it on my server (how to avoid running out of RAM when downloading large files)

In PHP, you can download a file via a link (URL) and save it on your server in several ways. Some of them have important differences that can be key when choosing a way to download and save a file in PHP.

1. file_get_contents and file_put_contents

The first very obvious option is to use file_get_contents and file_put_contents:

file_put_contents("Tmpfile.zip", file_get_contents("http://someurl/file.zip"));

Real example:

file_put_contents("mial.jpg", file_get_contents("https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg"));

The disadvantage of this method is that the file is completely stored in RAM before being saved to disk. If you are uploading a very large file (gigabytes in size), then you may encounter the problem of insufficient memory on the server that is allocated for executing one script.

2. fopen and file_put_contents to immediately write data to disk as it arrives

If it is important to you that the RAM does not become full when downloading a large file, then use a combination of fopen and file_put_contents:

file_put_contents("Tmpfile.zip", fopen("http://someurl/file.zip", 'r'));

In this case, the file will be written to disk as it is downloaded and will not be kept in RAM until the download is completed.

Real example:

file_put_contents("mial2.jpg", fopen("https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg", 'r'));

But this method also has a drawback – in the PHP settings, the allow_fopen_url value can be set to Off. In this case, the specified construction will not work due to the prohibition of opening files using fopen if the file is specified as a URL.

3. Custom function to upload files using PHP

If you need maximum flexibility, then you can use the following function. Please note that you can configure exactly what size chunks the file should be read and written to:

function downloadFile($url, $path)
{
	$newfname = $path;
	$file = fopen ($url, 'rb');
	if ($file) {
		$newf = fopen ($newfname, 'wb');
		if ($newf) {
			while(!feof($file)) {
				fwrite($newf, fread($file, 1024 * 8), 1024 * 8);
			}
		}
	}
	if ($file) {
		fclose($file);
	}
	if ($newf) {
		fclose($newf);
	}
}

Real working code example:

<?php

function downloadFile($url, $path)
{
	$newfname = $path;
	$file = fopen ($url, 'rb');
	if ($file) {
		$newf = fopen ($newfname, 'wb');
		if ($newf) {
			while(!feof($file)) {
				fwrite($newf, fread($file, 1024 * 8), 1024 * 8);
			}
		}
	}
	if ($file) {
		fclose($file);
	}
	if ($newf) {
		fclose($newf);
	}
}

downloadFile ('https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg', 'mial3.jpg');

4. Using the curl library to download a file in PHP

The curl library supports many protocols, you can also configure a custom User Agent, you can use authentication and cookies.

An example of downloading a file using the PHP curl library:

function downloadUrlToFile($url, $outFileName) {
	if(is_file($url)) {
	copy($url, $outFileName); 
	} 
	else {
	$options = array(
		CURLOPT_FILE    => fopen($outFileName, 'w'),
		CURLOPT_TIMEOUT =>  28800, // set this to 8 hours so we dont timeout on big files
		CURLOPT_URL     => $url
	);

	$ch = curl_init();
	curl_setopt_array($ch, $options);
	curl_exec($ch);
	$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
	curl_close($ch);
	return $httpcode;
	}
}

Real working code example:

function downloadUrlToFile($url, $outFileName) {
	if(is_file($url)) {
	copy($url, $outFileName); 
	} 
	else {
	$options = array(
		CURLOPT_FILE    => fopen($outFileName, 'w'),
		CURLOPT_TIMEOUT =>  28800, // set this to 8 hours so we dont timeout on big files
		CURLOPT_URL     => $url
	);

	$ch = curl_init();
	curl_setopt_array($ch, $options);
	curl_exec($ch);
	$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
	curl_close($ch);
	return $httpcode;
	}
}

downloadUrlToFile ('https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg', 'mial4.jpg');

5. Downloading files in PHP using wget

exec ('wget http://someurl/file.zip');

Real working code example:

exec ('wget https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg');

You need to make sure that the wget utility is installed on the server and can be run from a PHP script.

When using this option, the file is saved to disk as it is loaded and does not fill up the RAM.

While the script is running, wget information is displayed, including the file download progress.

It is not necessary to specify a file name – the original name will be used by default. If desired, you can explicitly specify the filename using the “-O” option:

exec ('wget -O mial5.jpg https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg');

6. Downloading files in PHP using cURL utility

You can use the Linux cURL utility, similar to how we used the wget utility in the previous example.

The capabilities of the PHP curl library and the Linux cURL utility are very similar. But this method should be placed in a separate group, since many people are much more comfortable with the cURL command line syntax than the intricate PHP syntax of the curl library.

For example, instead of all the intricate designs discussed above for the PHP curl library, a similar result can be achieved using the following line of code:

exec ('curl -o Tmpfile.zip http://someurl/file.zip');

You can still use different protocols, authentication, POST requests, referrer and User Agent spoofing, cookies and all the other numerous cURL features. But at the same time, you do not need to relearn the syntax of the PHP curl library – you can use the familiar command line syntax.

Real working code example:

exec ('curl -o mial6.jpg https://suay.ru/wp-content/uploads/2024/03/20231213_151132.jpg');

Please note that the “-o” option indicating the file name is required for binary files (and for text files too, since without specifying this option the file contents will be output to standard output without saving to disk).