Get "PHP 8 in a Nuthshell" (Soon includes PHP 8.4)
Amit Merchant

Amit Merchant

A blog on PHP, JavaScript, and more

Using any large JSON as a lazy collection in Laravel

If you’re working with Laravel (or any PHP application for that matter), you might have stumbled upon a situation where you need to process a large JSON input. For instance, you might want to use a JSON file to seed your database.

The problem is, that if the JSON is large, it will take a lot of memory to load it into the memory. And eventually, it will result in a memory exhaustion error.

A new package called lazy-json attempts to fix this issue. It allows you to use any large JSON file as a lazy collection in your application. It’s a framework-agnostic package that can be used with any PHP application.

Under the hood, it uses Laravel’s Lazy Collections to achieve this. So, you can use all the methods available in the LazyCollection class.

Installation

You can install the package via composer like so.

composer require cerbero/lazy-json

Usage

There are three ways you can use this package in your project.

use Cerbero\LazyJson\LazyJson;
use Illuminate\Support\LazyCollection;

use function Cerbero\LazyJson\lazyJson;

// auto-registered lazy collection macro
$lazyCollection = LazyCollection::fromJson($source);

// static method
$lazyCollection = LazyJson::from($source);

// namespaced helper
$lazyCollection = lazyJson($source);

The source here can be a string, an array, a file path, a resource, or a URL.

Here’s the list of all the supported sources.

  • strings, e.g. {"foo":"bar"}
  • iterables, i.e. arrays or instances of Traversable
  • file paths, e.g. /path/to/large.json
  • resources, e.g. streams
  • API endpoint URLs, e.g. https://endpoint.json or any instance of Psr\Http\Message\UriInterface
  • PSR-7 requests, i.e. any instance of Psr\Http\Message\RequestInterface
  • PSR-7 messages, i.e. any instance of Psr\Http\Message\MessageInterface
  • PSR-7 streams, i.e. any instance of Psr\Http\Message\StreamInterface
  • Laravel HTTP client requests, i.e. any instance of Illuminate\Http\Client\Request
  • Laravel HTTP client responses, i.e. any instance of Illuminate\Http\Client\Response
  • user-defined sources, i.e. any instance of Cerbero\JsonParser\Sources\Source

An example

Here’s a basic example of how to turn JSON into a lazy collection using the global helper function.

use function Cerbero\LazyJson\lazyJson;

$source = '{
  "users": [
    {
      "id": 3,
      "name": "Amit Merchant",
      "email": "[email protected]"
    },
    {
      "id": 1,
      "name": "John Doe",
      "email": "[email protected]"
    },
    {
      "id": 2,
      "name": "Jane Doe",
      "email": "[email protected]"
    }
  ]
}';

$lazyCollection = lazyJson($source, ['users.*.name']);

$mapped = $lazyCollection->map(function (string $value, string $key) {
    return $value;
});

$mapped->each(function ($value, $key) {
    dump($key . ' => ' . $value);
    // iteration 1: "name => Amit Merchant" 
    // iteration 2: "name => John Doe" 
    // iteration 3: "name => Jane Doe" 
});

As you can tell, the lazyJson helper function takes two arguments.

The first one is the source of the JSON and the second one is the path to the data you want to extract from the JSON. You can pass it as dot-notation syntax to extract the sub-data from the JSON.

If you want multiple paths, you can pass them as an array like so.

$lazyCollection = lazyJson($source, [
    'users.*.name', 
    'users.*.email'
]);

In closing

So, this is how you can use any large JSON as a lazy collection in your Laravel application. I think it’s a pretty handy package to have in your toolkit if you’re working with large JSON files.

Learn more about the package: lazy-json

Learn the fundamentals of PHP 8 (including 8.1, 8.2, and 8.3), the latest version of PHP, and how to use it today with my new book PHP 8 in a Nutshell. It's a no-fluff and easy-to-digest guide to the latest features and nitty-gritty details of PHP 8. So, if you're looking for a quick and easy way to PHP 8, this is the book for you.

Like this article?

Buy me a coffee

👋 Hi there! I'm Amit. I write articles about all things web development. You can become a sponsor on my blog to help me continue my writing journey and get your brand in front of thousands of eyes.

Comments?