Reading large CSV Files as a Collection in Laravel
Large-scale projects would often require to import data from large CSV files at some point. And if not handled properly, this can very well lead to memory issues.
Thankfully, you can use League’s CSV package to read CSV files conveniently and iterate over them as a collection in Laravel. Especially, using lazy collections. This way, you can read CSV files as a collection without loading the entire file into memory.
To get started, you can first install the
league/csv package using Composer like so.
composer require league/csv:^9.0
Once done, you can use the
League\Csv\Reader class to read the CSV file. You can pass the path to the CSV file to the constructor of the
Reader class. This will return an instance of the
use League\Csv\Reader; $cities = Reader::createFromPath('path/to/cities.csv');
Now, you can set the header offset to
0 using the
setHeaderOffset() method. This will set the first row of the CSV file as the header row.
Next, you can use the
getRecords() method to iterate over the CSV file as a collection. This method returns a generator that can be used to iterate over the CSV file as a collection.
Next, you can pass in the generator to the
LazyCollection::make() method to create a lazy collection. This will return an instance of the
And finally, you can use any of the methods available on the
LazyCollection class to iterate over the CSV file as a collection.
For instance, you can use the
groupBy() method to group the records by a specific column, the
map() method to map over the records, the
filter() method to filter the records, and so on.
Here’s putting it all together.
use Illuminate\Support\LazyCollection; use League\Csv\Reader; $cities = Reader::createFromPath('path/to/cities.csv'); $cities->setHeaderOffset(0); $citiesCollection = LazyCollection::make(static fn () => yield from $cities->getRecords()); ->groupBy('country') ->map(fn ($cities) => $cities->count()) ->toArray();
As you can tell, we are using a
static closure to create a generator that will be passed to the
LazyCollection::make() method. This is because the
LazyCollection::make() method expects a generator as an argument.
And that’s it. This way, you can read CSV files as a collection in Laravel without loading the entire file into memory.
I got to know about this courtesy of this tweet.