how to make pagination more efficient with large arrays and db support

i have a pagination library in php. i want to know:

• how can i make it more efficient when passing large arrays into the constructor
• how can i add db support

paginator.php

public function __construct($options = []) {$this->items = $options['data'];$this->limit = $options['limit'] ?? 10;$this->currentPage = $options['currentPage'] ?? 1; //$this->urlPattern = $options['urlPattern'];$this->updateNumPages();
}

public function paginate(int $page = 1): PaginatorInterface { if ($page <= 0 || $this->limit <= 0) { throw new \LogicException("Invalid parameters."); }$offset = ($page - 1) *$this->limit;
$pagination = new DefaultPaginator();$items = new Collection($this->items,$offset, $this->limit); if ($this->items instanceof \ArrayObject || is_array($this->items)) {$data = new ArrayData();
$data->fetchItems($items);
} else {
throw new \RuntimeException("Data type not supported for pagination.");
}
$pagination->setCurrentPageNumber($page);
$pagination->setNumberOfPages((int) ceil($items->getCount() / $items->getLimit()));$pagination->setItems($items->getItems());$pagination->setTotal($items->getCount());$pagination->setTotalOnCurrentPage(count($items->getItems()));$pagination->setTotalPerPage($this->limit); return$pagination;
}

my repository is here https://github.com/shorif2000/pagination

here is where i am slicing the array.

class ArrayData implements DataInterface
{

public function fetchItems(Collection &$items): void { if (is_array($items->getCollection())) {
$items->setItems(array_slice($items->getCollection(),
$items->getOffset(),$items->getLimit()
));
$items->setCount(count($items->getCollection()));
} elseif ($items->getCollection() instanceof ArrayObject) {$collection = $items->getCollection();$items->setItems(new ArrayObject(array_slice(
$collection->getArrayCopy(),$items->getOffset(),
$items->getLimit() )));$items->setCount($collection->count()); } } } UPDATE I created this class to add support for db or array/ArrayObject class Pagination { public function __construct($options = [], $mode = 'Default') { eval('$this = Pagination::factory($options,$mode);');
}

public static function &factory($options,$mode)
{
$classname = ($mode == 'Default') ? 'Pagination\\Paginator' : 'Pagination\\DbPage' ;
// If the class exists, return a new instance of it.
if (class_exists($classname)) {$pagination = new $classname($options);
return $pagination; }$null = null;
return $null; } } UPDATE with the answer to test i have the following$pageNumber = 1;
$itemsPerPage = 10;$input = range(0, 100);
$input = array_slice($input, 0, 10);
$total = count($input);
print_r((new PaginatorFactory(new ArrayPageProvider($input)))->createPaginator($pageNumber, $itemsPerPage)); 1 Answer 1) Simply don't pass large arrays, use some abstraction. 2) Once you add the abstraction, you can provide db support. The following abstraction might show how to do it: EDIT: ok so i have added some more to point at the core of your problems: The paginator doesnt really need the setters to be part of its interface. Also see I use Iterator to represent the page items, instead of array. The array will only be used in specific implementation. You should avoid the array type in the abstraction. interface PaginatorInterface { public function getItems(); public function getCurrentPageNumber(): int; public function getNumberOfPages(): int; public function getTotal(): int; public function getTotalOnCurrentPage(): int; public function getTotalPerPage(): int; } In fact, the class implementing that interface doesnt really need those setters too. It's actualy not even desired. Also the paginator can do simple computations. You dont have to precompute and assign static values which actualy opens space for corrupted state. class Paginator implements PaginatorInterface { private$pageNumber;
private $itemsPerPage; private$items;
private $total; public function __construct(int$pageNumber, int $itemsPerPage, \Iterator$pageItems, int $total) { if ($pageNumber < 1) {
throw new \InvalidArgumentException();
}
if ($itemsPerPage < 1) { throw new \InvalidArgumentException(); }$this->pageNumber = $pageNumber;$this->itemsPerPage = $itemsPerPage;$this->items = $pageItems;$this->total = $total; } public function getItems(): \Iterator { return$this->items;
}

public function getCurrentPageNumber(): int
{
return $this->pageNumber; } public function getNumberOfPages(): int { return \ceil($this->getTotal() / $this->itemsPerPage); } public function getTotal(): int { return$this->total;
}

public function getTotalOnCurrentPage(): int
{
if ($this->items instanceof \Countable) { return \count($this->items);
}
return \iterator_count($this->items); } public function getTotalPerPage(): int { return$this->itemsPerPage;
}
}

Here I define the page provider (extended by getTotalCount method since my edit):

interface PageProviderInterface
{
public function getTotalCount(): int;
public function getPage(int $offset, int$limit): \Iterator;
}

You then use the page provider within a paginator factory (that is your pagination interface but it actualy is paginator factory so i named it that way).

interface PaginatorFactoryInterface
{
public function createPaginator(int $page, int$pageSize): PaginatorInterface;
}

class PaginatorFactory implements PaginatorFactoryInterface
{
/** @var PageProviderInterface */
private $pageProvider; public function __construct(PageProviderInterface$pageProvider)
{
$this->pageProvider =$pageProvider;
}

public function createPaginator(int $page, int$pageSize): PaginatorInterface
{
$total =$this->pageProvider->getTotalCount();
$offset = ($page - 1) * $pageSize; if ($offset >= $total) {$items = new \EmptyIterator();
} else {
$items =$this->pageProvider->getPage($offset,$pageSize);
}
return new Paginator($page,$pageSize, $items,$total);
}
}

And lastly some implementation of the page provider.

The array one which provider slices of an array that resides in memory:

class ArrayPageProvider implements PageProviderInterface
{
private $items; public function __construct(array$items) {

$this->items =$items;
}

public function getTotalCount(): int
{
return \count($this->items); } public function getPage(int$offset, int $limit): \Iterator { return new \ArrayIterator(\array_slice($this->items, $offset,$limit));
}
}

And the silly PDO implementation, which would need some improvement to be actualy usable, but you should get the point:

class DbPageProvider implements PageProviderInterface
{
private $pdo; private$table;

public function __construct(\PDO $pdo, string$table) {
$this->pdo =$pdo;
$this->table =$table;
}

public function getTotalCount(): int
{
$statement =$this->pdo->prepare('select count(*) from ' . $this->table); return$statement->fetchColumn();
}

public function getPage(int $offset, int$limit): \Iterator
{
$statement =$this->pdo->prepare('select * from ' . $this->table . ' limit ' .$offset . ',' . $limit);$result = $statement->execute();$i = 0;
while ($row =$result->fetch()) {
yield $i++ =>$row;
}
}

}

And that should actualy be all you need. No need for some crazy collection outsourcing or too much of new new new in one method. Keep it simple and remember the naming is really important for whoever wants to understand the code.

Oh and one more, you hade a getViewData() method there, but that should be done by someone else:

function getPaginatorViewData(PaginatorInterface $paginator) { return [ 'elements' =>$paginator->getItems(),
// ...

];
}

This could be a global function, or it could be static method of its own class, or could even be elevated to an interface level. But actualy i dont think it belong to the lib, it may be part of some framework bundle, or something like that, but not the lib itself. The moment it becomes shapeless array, it becomes hard to work with...

And here is a simple example of usage:

// in DI container
$pageProvider = new DbPageProvider($pdo, 'users');
$usersPaginatorFactory = new PaginatorFactory($pageProvider);

// in lets say /users/list constroller we have the paginator factory injected
// $pageNumber is coming from request$paginator = $this->usersPaginatorFactory->createPaginator($pageNumber, 20);

// maybe you wanna pass this to laravel view?
return view('users.list', ['paginator' => $paginator]); // or maybe if you defined the getPaginatorViewData() function return view('users.list', getPaginatorViewData($paginator));

PS: try to avoid constructor with $options array. It really is better to name all the parameters separately, even omit default value, let the user or a factory (if there is one) decide. Default values usualy just complicate things even if it seems the opposite at first glance... oh and PS2: although this should have been a big warning on the beginning. Dont use eval(), avoid references, definitely dont return references, avoid instantiation of classes by string containing class name. The Pagination::factory is completly wrong. Immediately get rid of that file and never write anything like it again!!! • wont$this->items = \$item; consume memory in ArrayPageProvider ?. I added update with some code on how i am slicing array – shorif2000 Nov 4 '19 at 11:56
• @shorif2000 well, that's the array implementation. that one serves the purpose when you have all items in memory and want to serve pages of it. If you want something else, you need different implementation. Dont check if they provided array or ArrayObject and then construct ArrayData object from it. Let them pass the ArrayData to you (which is kinda the analogue to my ArrayPageProvider, so better let them pass the PageProviderInterface which is the abstraction)... you seem to be quite unfamiliar with the concept of IoC, generaly the overdose of new operator is not good... – slepic Nov 4 '19 at 12:54
• @shorif2000 diving deeper into your code, I am very confused. It is all twisted inside out in a weird way. Naming is bad. Like why PaginatorInterface is implemented by Pagination when there is also PaginationInterface. Then there is DefaultPaginator which implements neither of the two. Collection class that is actualy some weird kind of array with fakable attributes, which gets sliced at some point... PaginatorInterface should tell you number of items and pages, not you tell it what those things are... I think you should hardly rethink what you are actualy doing. – slepic Nov 4 '19 at 13:13
• I have updated it slightly. what do you think now? I added some update above for db support – shorif2000 Nov 4 '19 at 17:18
• @shorif2000 I have added example usage. You should not do the slicing outside the ArrayPagePRovider as in you edit in your answer... The arrayPageProvider is designed to do just that. But dont get too stuck with that specific implementation, i dont really see a real usage for it... it is just the simplest implementation i could think of and also you were talking about arrays... but in real world you want to avoid array implementations probably... – slepic Nov 5 '19 at 17:14