how to make pagination more efficient with large arrays and db support

Question

i have a pagination library in php. i want to know:

• how can i make it more efficient when passing large arrays into the constructor
• how can i add db support

paginator.php

public function __construct($options = [])
    {
        $this->items = $options['data'];
        $this->limit = $options['limit'] ?? 10;
        $this->currentPage = $options['currentPage'] ?? 1;
        //$this->urlPattern = $options['urlPattern'];
        $this->updateNumPages();
    }

    public function paginate(int $page = 1): PaginatorInterface
    {
        if ($page <= 0 || $this->limit <= 0) {
            throw new \LogicException("Invalid parameters.");
        }
        $offset = ($page - 1) * $this->limit;
        $pagination = new DefaultPaginator();
        $items = new Collection($this->items, $offset, $this->limit);
        if ($this->items instanceof \ArrayObject || is_array($this->items)) {
            $data = new ArrayData();
            $data->fetchItems($items);
        } else {
            throw new \RuntimeException("Data type not supported for pagination.");
        }
        $pagination->setCurrentPageNumber($page);
        $pagination->setNumberOfPages((int) ceil($items->getCount() / $items->getLimit()));
        $pagination->setItems($items->getItems());
        $pagination->setTotal($items->getCount());
        $pagination->setTotalOnCurrentPage(count($items->getItems()));
        $pagination->setTotalPerPage($this->limit);
        return $pagination;
    }

my repository is here https://github.com/shorif2000/pagination

here is where i am slicing the array.

class ArrayData implements DataInterface
{

    public function fetchItems(Collection &$items): void
    {
        if (is_array($items->getCollection())) {
            $items->setItems(array_slice(
                $items->getCollection(),
                $items->getOffset(),
                $items->getLimit()
            ));
            $items->setCount(count($items->getCollection()));
        } elseif ($items->getCollection() instanceof ArrayObject) {

            $collection = $items->getCollection();
            $items->setItems(new ArrayObject(array_slice(
                $collection->getArrayCopy(),
                $items->getOffset(),
                $items->getLimit()
            )));
            $items->setCount($collection->count());
        }
    }
}

UPDATE

I created this class to add support for db or array/ArrayObject

class Pagination
{
    public function __construct($options = [], $mode = 'Default')
    {
        eval('$this = Pagination::factory($options, $mode);');
    }

    public static function &factory($options, $mode)
    {
        $classname = ($mode == 'Default') ? 'Pagination\\Paginator' : 'Pagination\\DbPage' ;
        // If the class exists, return a new instance of it.
        if (class_exists($classname)) {
            $pagination = new $classname($options);            
            return $pagination;
        }

        $null = null;
        return $null;
    }
}

UPDATE

with the answer to test i have the following

$pageNumber = 1;
        $itemsPerPage = 10;
        $input = range(0, 100);
        $input = array_slice($input, 0, 10);
        $total = count($input);
        print_r((new PaginatorFactory(new ArrayPageProvider($input)))->createPaginator($pageNumber, $itemsPerPage));

Answer 1

1) Simply don't pass large arrays, use some abstraction.

2) Once you add the abstraction, you can provide db support.

The following abstraction might show how to do it:

EDIT: ok so i have added some more to point at the core of your problems:

The paginator doesnt really need the setters to be part of its interface.

Also see I use Iterator to represent the page items, instead of array. The array will only be used in specific implementation. You should avoid the array type in the abstraction.

interface PaginatorInterface
{
    public function getItems();

    public function getCurrentPageNumber(): int;

    public function getNumberOfPages(): int;

    public function getTotal(): int;

    public function getTotalOnCurrentPage(): int;

    public function getTotalPerPage(): int;
}

In fact, the class implementing that interface doesnt really need those setters too. It's actualy not even desired.

Also the paginator can do simple computations. You dont have to precompute and assign static values which actualy opens space for corrupted state.

class Paginator implements PaginatorInterface
{
    private $pageNumber;
    private $itemsPerPage;
    private $items;
    private $total;

    public function __construct(int $pageNumber, int $itemsPerPage, \Iterator $pageItems, int $total)
    {

        if ($pageNumber < 1) {
            throw new \InvalidArgumentException();
        }
        if ($itemsPerPage < 1) {
            throw new \InvalidArgumentException();
        }
        $this->pageNumber = $pageNumber;
        $this->itemsPerPage = $itemsPerPage;
        $this->items = $pageItems;
        $this->total = $total;
    }

    public function getItems(): \Iterator
    {
        return $this->items;
    }

    public function getCurrentPageNumber(): int
    {
        return $this->pageNumber;
    }

    public function getNumberOfPages(): int
    {
        return \ceil($this->getTotal() / $this->itemsPerPage);
    }

    public function getTotal(): int
    {
        return $this->total;
    }

    public function getTotalOnCurrentPage(): int
    {
        if ($this->items instanceof \Countable) {
            return \count($this->items);
        }
        return \iterator_count($this->items);
    }

    public function getTotalPerPage(): int
    {
        return $this->itemsPerPage;
    }
}

Here I define the page provider (extended by getTotalCount method since my edit):

interface PageProviderInterface
{
    public function getTotalCount(): int;
    public function getPage(int $offset, int $limit): \Iterator;
}

You then use the page provider within a paginator factory (that is your pagination interface but it actualy is paginator factory so i named it that way).

interface PaginatorFactoryInterface
{
    public function createPaginator(int $page, int $pageSize): PaginatorInterface;
}

class PaginatorFactory implements PaginatorFactoryInterface
{
    /** @var PageProviderInterface */
    private $pageProvider;

    public function __construct(PageProviderInterface $pageProvider)
    {
        $this->pageProvider = $pageProvider;
    }

    public function createPaginator(int $page, int $pageSize): PaginatorInterface
    {
        $total = $this->pageProvider->getTotalCount();
        $offset = ($page - 1) * $pageSize;
        if ($offset >= $total) {
            $items = new \EmptyIterator();
        } else {
            $items = $this->pageProvider->getPage($offset, $pageSize);
        }
        return new Paginator($page, $pageSize, $items, $total);
    }
}

And lastly some implementation of the page provider.

The array one which provider slices of an array that resides in memory:

class ArrayPageProvider implements PageProviderInterface
{
    private $items;

    public function __construct(array $items) {

        $this->items = $items;
    }

    public function getTotalCount(): int
    {
        return \count($this->items);
    }

    public function getPage(int $offset, int $limit): \Iterator
    {
        return new \ArrayIterator(\array_slice($this->items, $offset, $limit));
    }
}

And the silly PDO implementation, which would need some improvement to be actualy usable, but you should get the point:

class DbPageProvider implements PageProviderInterface
{
    private $pdo;
    private $table;

    public function __construct(\PDO $pdo, string $table) {
        $this->pdo = $pdo;
        $this->table = $table;
    }

    public function getTotalCount(): int
    {
        $statement = $this->pdo->prepare('select count(*) from ' . $this->table);
        return $statement->fetchColumn();
    }

    public function getPage(int $offset, int $limit): \Iterator
    {
        $statement = $this->pdo->prepare('select * from ' . $this->table . ' limit ' . $offset . ',' . $limit);
        $result = $statement->execute();
        $i = 0;
        while ($row = $result->fetch()) {
            yield $i++ => $row;
        }
    }

}

And that should actualy be all you need. No need for some crazy collection outsourcing or too much of new new new in one method. Keep it simple and remember the naming is really important for whoever wants to understand the code.

Oh and one more, you hade a getViewData() method there, but that should be done by someone else:

function getPaginatorViewData(PaginatorInterface $paginator) {
  return [
    'elements' => $paginator->getItems(),
    // ...

  ];
}

This could be a global function, or it could be static method of its own class, or could even be elevated to an interface level. But actualy i dont think it belong to the lib, it may be part of some framework bundle, or something like that, but not the lib itself. The moment it becomes shapeless array, it becomes hard to work with...

And here is a simple example of usage:

// in DI container
$pageProvider = new DbPageProvider($pdo, 'users');
$usersPaginatorFactory = new PaginatorFactory($pageProvider);

// in lets say /users/list constroller we have the paginator factory injected
// $pageNumber is coming from request
$paginator = $this->usersPaginatorFactory->createPaginator($pageNumber, 20);

// maybe you wanna pass this to laravel view?
return view('users.list', ['paginator' => $paginator]);
// or maybe if you defined the getPaginatorViewData() function
return view('users.list', getPaginatorViewData($paginator));

PS: try to avoid constructor with $options array. It really is better to name all the parameters separately, even omit default value, let the user or a factory (if there is one) decide. Default values usualy just complicate things even if it seems the opposite at first glance...

oh and PS2: although this should have been a big warning on the beginning. Dont use eval(), avoid references, definitely dont return references, avoid instantiation of classes by string containing class name. The Pagination::factory is completly wrong. Immediately get rid of that file and never write anything like it again!!!