Redux pagination transducer

Question

Motivation

In my web app (React, Redux, TypeScript) we have a common need to write paginated API calls, with various UI interactions (infinite scroll, prefetch pages in the background, or manual page navigation controls). Rather than re-write pagination handling in every case, I wanted to create a re-usable method to use pagination anywhere, as well as make it easy to update non-paginated data into paginated data.

Pagination is repeatedly cited as a specific example where the features of Redux, like reducer composition, are meant to shine. So I gave it a shot!

Because I'm using TypeScript a parallel goal is to make this type-safe and self-documenting.

Paginated Server API

All paginated API endpoints take requests like this:

{
  offset: 0,
  limit: 100
}

And return like this:

{
  items: [ ...100 items ],
  size: 300
}

In this example we are asking for the first page (offset=0) with 100 per page, and receiving the first page items along with the info that there are 300 total items (ie 3 pages). The next page would be { offset: 100, limit: 100 }.

Approach

• paginate(config, reducer) is a a "transducer" (aka "enhancer" or "higher-order reducer") that takes a reducer meant for items (array) and wraps it in handling of page request, success, and error action cases to update a paginated state object.
• paginated(actions, onFetch, getPaginatedState) is an action creator that will dispatch actions which compliments paginate() with page request, success, and error actions. It only requires implementing how a specific page fetch request is made via onFetch(offset, limit), and what slice of the store to pull pagination state from via getPaginatedState(state).
• <InfinateScroller>, <PaginationControls> and <InfinitePreloader> are components that deal with user interactivity and when to request pages from the API.
• Connected components pass the relevant pagination store state slice and page fetch action to one of the above components.

Code

Paginated<T> type that represents a store state slice of paginated data:

export interface Paginated<T> {
  /**
   * True if a request for a page is in progress.
   */
  fetching: boolean;
  /**
   * True if a request for a page resulted in an error, which could be an API error or UI processing error.
   */
  error: boolean;
  /**
   * The configured page size to use in requests.
   */
  pageSize: number;
  /**
   * The current requested page index (0 = first page), or null if no request has been made yet.
   */
  currentPage: number | null;
  /**
   * The loaded items, or null if no request has been completed yet.
   */
  items: T[] | null;
  /**
   * The total number of items available from the server, or null if no request has been completed yet.
   */
  totalItems: number | null;
}

Example usage of Paginated<T> the store state type:

interface StoreState {
    paginatedProjects: Paginated<Project>;
}

interface Project {
  id: string;
  name: string;
  // ...
}

PaginatedActionTypes type used for both paginate() transducer and paginated() action creator to describe the actions during a pagination API call:

interface PaginatedActionTypes {
  requestType: string;
  successType: string;
  errorType: string;
}

The paginate() transducer implementation:

import { Action, Reducer } from "redux";

interface PaginateConfig extends PaginatedActionTypes {
  /**
   * The number of items requested with each page.
   */
  pageSize: number;
  /**
   * Optional list of action types that should reset the state to default.
   * For example, use this to have certain UI actions like filtering and sorting
   * clear the existing data and down-stream UIs will re-fetch the first page.
   */
  invalidateTypes?: string[];
}

export interface PaginatedRequestAction extends Action {
  requestedPage: number;
}

export interface PaginatedSuccessAction<T> extends Action {
  items: T[];
  size: number;
  fetchedPage: number;
}

export interface PaginatedErrorAction extends Action {
  failedPage: number;
}

type PaginatedAction<T> =
  PaginatedRequestAction
  | PaginatedSuccessAction<T>
  | PaginatedErrorAction;

/**
 * This function can enhance a reducer to handle a paginated API collection.
 *
 * The wrapped reducer can be written without knowledge of pagination.
 *
 * The correlated state will be a Paginated wrapper that holds the pagination state and the loaded items.
 */
export function paginate<T>(config: PaginateConfig, reducer: Reducer<T[]>) {
  const { requestType, successType, errorType, invalidateTypes, pageSize } = config;
  const defaultState: Paginated<T> = {
    fetching: false,
    error: false,
    pageSize,
    // Use `null` to indicate "never loaded" vs "loaded empty collection", used by down-stream components to show loading state.
    totalItems: null,
    items: null,
    currentPage: null
  };

  const paginatedReducer = (state: Paginated<T> = defaultState, action: PaginatedAction<T>): Paginated<T> => {
    switch (action.type) {

      case requestType:
        const request = action as PaginatedRequestAction;
        return {
          ...state,
          fetching: true,
          currentPage: request.requestedPage
        };

      case successType:
        const success = action as PaginatedSuccessAction<T>;
        if (state.currentPage !== success.fetchedPage)
          throw new Error(`Received page ${ success.fetchedPage } but expecting page ${ state.currentPage }`);

        const existingItems = state.items || [];

        return {
          ...state,
          fetching: false,
          error: false,
          items: [...existingItems, ...success.items],
          totalItems: success.size
        };

      case errorType:
        const error = action as PaginatedErrorAction;
        return {
          ...state,
          fetching: false,
          error: true
        };

      default:
        // Actions that invalidate the loaded data, resets to initial state
        if (invalidateTypes && invalidateTypes.includes(action.type)) {
          return defaultState;
        }

        // Pass unhandled actions through to inner reducer
        if (state.items) {
          const nextItems = reducer(state.items, action);

          // Update items if the inner reducer returned a new state
          if (nextItems !== state.items) {
            // Update the size by delta if it changed by the inner reducer
            const nextTotalItems = state.totalItems != null && state.items
              ? state.totalItems + (nextItems.length - state.items.length)
              : state.totalItems;
            return {
              ...state,
              items: nextItems,
              totalItems: nextTotalItems
            };
          }
        }

        return state;
    }
  };

  return paginatedReducer;
}

The paginated() action creator implementation, which uses thunk:

import { StoreState, ThunkDispatch } from "../../store/configureStore";

export interface Collection<T> {
  items: T[];
  size: number;
}

/**
 * This is a helper function to create action creators for the paginate() enhancer.
 *
 * You give it the same action types for { request, success, fail } and callbacks to call
 * the actual API and extract the paginated state, and it handles the fetching of
 * the next page of data and dispatching the actions during the life-cycle of a call.
 */
export function paginated<T>(
  { requestType, successType, errorType }: PaginatedActionTypes,
  onFetch: (offset: number, limit: number, getState: () => StoreState) => Promise<Collection<T>>,
  getPaginatedState: (state: StoreState) => Paginated<T>
) {
  return async (dispatch: ThunkDispatch, getState: () => StoreState) => {
    const { pageSize, currentPage, fetching } = getPaginatedState(getState());

    if (fetching) {
      console.error(`Already fetching ${ requestType } page -- ignoring duplicate call. Components should not try to fetch another page while fetching a page is in progress.`);
      return;
    }

    const nextPage = currentPage == null ? 0 : currentPage + 1;

    const request: PaginatedRequestAction = {
      type: requestType,
      requestedPage: nextPage
    };
    dispatch(request);

    try {
      const nextPageOffset = nextPage * pageSize;
      const response = await onFetch(nextPageOffset, pageSize, getState);

      if (response.size == null)
        throw new Error(`Unexpected paginated response: "size" is missing`);

      const action: PaginatedSuccessAction<T> = {
        type: successType,
        items: response.items || [],
        size: response.size,
        fetchedPage: nextPage
      };
      dispatch(action);

    } catch (e) {
      console.error(`Error in paginated.onFetch call:`, e);
      const error: PaginatedErrorAction = {
        type: errorType,
        failedPage: nextPage
      };
      dispatch(error);
    }
  };
}

<InfiniteScroller> component, which uses react-waypoint to request the next page when the user scrolls to the bottom of the loaded data.

import * as React from "react";
import { PureComponent } from "react";
import Waypoint from "react-waypoint";
import WaypointEvent = ReactWaypoint.WaypointEvent;
import { Loader } from "./Loader";
import { LoadingError } from "./LoadingError";

interface InfiniteScrollerProps {
  /**
   * Whether the initial page has been loaded. If false an initial onLoadMore() will be called when first rendered.
   */
  initialLoaded: boolean;

  /**
   * Whether a page is currently being loaded. Prevents onLoadMore() from being called until false.
   */
  isLoading: boolean;

  /**
   * Whether the requested page resulted in an error.
   */
  hasError: boolean;

  /**
   * Called when the next (or initial) page needs to be loaded due to scrolling (or initial render).
   */
  onLoadMore(): void;

  /**
   * How many items are currently loaded.
   */
  itemsLoaded: number;

  /**
   * Total items that can be loaded.
   */
  totalItems: number;
}

export class InfiniteScroller extends PureComponent<InfiniteScrollerProps, {}> {

  handleEnterBottomWaypoint = (e: WaypointEvent): void =>
    this.props.onLoadMore();

  componentDidUpdate(prevProps: Readonly<InfiniteScrollerProps>, prevState: Readonly<{}>, snapshot?: any): void {
    if (!this.props.initialLoaded && prevProps.initialLoaded !== this.props.initialLoaded) {
      this.props.onLoadMore();
    }
  }

  render() {
    const { initialLoaded, itemsLoaded, totalItems, className, isLoading, hasError, children } = this.props;

    const hasMoreItems = itemsLoaded < totalItems;
    const needsToLoadMore = (!initialLoaded || hasMoreItems) && !hasError;
    return (
      <div className={ className }>
        { children }
        { isLoading ? <Loader/> : needsToLoadMore && <Waypoint onEnter={ this.handleEnterBottomWaypoint }/> }
        { hasError && <LoadingError/> }
      </div>
    );
  }
}

<InfinitePreloader> and <PaginationControls> work similarly.

Example

(Some irrelevant details trimmed.)

paginatedProjectsReducer which wraps a projectsReducer using the paginate() transducer:

// This is an example of a reducer that deals with an array and has no knowledge of pagination:
export type ProjectsAction =
  UpdateProjectSuccessAction
  | UpdateProjectsSuccessAction
  | CreateProjectSuccessAction
  | DeleteProjectSuccessAction
  // etc..

export const projectsReducer = (state: Project[], action: ProjectsAction): Project[] => {
  switch (action.type) {
    case UPDATE_PROJECT_SUCCESS: // ...
    case UPDATE_PROJECTS_SUCCESS: // ...
    case CREATE_PROJECT_SUCCESS: // ...
    case DELETE_PROJECT_SUCCESS: // ...

    default:
      const _exhaustiveCheck: never = action;
      return state;
  }
};

// This is an example of a reducer wrapped with pagination:
export const paginatedProjectsReducer = paginate<Project>(
  {
    requestType: FETCH_PROJECTS_REQUEST,
    successType: FETCH_PROJECTS_SUCCESS,
    errorType: FETCH_PROJECTS_ERROR,
    invalidateTypes: [
      RELOAD_PROJECTS,
      LEAVE_PROJECTS_CONTEXT,
      FILTER_PROJECTS,
      SORT_PROJECTS
    ],
    pageSize: 100
  },
  projectsReducer
);

paginatedProjectsReducer is used in the store's root reducer:

import { paginatedProjectsReducer as paginatedProjects } from "./projectsReducer";
export const rootReducer = combineReducers({
  paginatedProjects
});

fetchProjects action creator using the paginated() action creator to fetch the next page of projects:

import { api } from "../api";

export const FETCH_PROJECTS_REQUEST = "FETCH_PROJECTS_REQUEST";
export const FETCH_PROJECTS_SUCCESS = "FETCH_PROJECTS_SUCCESS";
export const FETCH_PROJECTS_ERROR = "FETCH_PROJECTS_ERROR";

export const fetchProjects = (planId: string) => (
  paginated<Project>(
    {
      requestType: FETCH_PROJECTS_REQUEST,
      successType: FETCH_PROJECTS_SUCCESS,
      errorType: FETCH_PROJECTS_ERROR
    },
    async (offset, limit, getState) => (
      // Simple API call, but can be more complicated as needed
      await api.getProjectsForPlan({ id: planId, limit, offset })
    ),
    state => state.paginatedProjects
  )
);

Usage of <InfiniteScroller> from a connected component:

  render() {
    const { paginatedProjects, fetchProjects, planId } = this.props;

    return (
      <InfiniteScroller initialLoaded={ !!paginatedProjects.items }
                        isLoading={ paginatedProjects.fetching }
                        hasError={ paginatedProjects.error }
                        onLoadMore={ () => fetchProjects(planId) }
                        itemsLoaded={ paginatedProjects.items?.length ?? 0 }
                        totalItems={ paginatedProjects.totalItems ?? 0 }>
        { items && items.map(project => <ProjectRow key={ project.id } {...project} />) }
      </InfiniteScroller>
    )

Summary

That's a lot of code, thanks to anyone who takes the time to look at it!

This ended up fairly similar to redux-pagination and probably other implementations, one notable difference being I opted to keep the fetching delegated outside the pagination implementation, so that pagination is not opinionated about how to fetch pages. I also opted not to use middleware like the Redux example, because frankly I find the indirection hard to follow, and didn't see a benefit to it. Tell me how I'm wrong!

I've been using this for a little while now, so it's somewhat battle-tested, but I'd love to get any feedback. It seems that pagination is a common use-case in Redux but I found it surprisingly hard to find an unopinionated implementation I could fit into my existing app, so I built this one. Cheers!