Pagination, filtering and sorting done right

Pagination, filtering and sorting are part of the API contract. Treat them as first-class design choices, because small inconsistencies in collection endpoints quickly become expensive client bugs.

Paginate every growing collection

Any collection that can grow beyond a small bounded size should be paginated from the start. Pagination protects the service from unbounded reads and protects clients from large responses, timeouts, and memory pressure.

Do not add pagination only after the first large customer arrives. Retrofitting pagination is a breaking change when clients already expect a complete list from one request.

Prefer cursor pagination for changing data

Offset pagination is simple: the client asks for offset=100&limit=50 or page=3&pageSize=50. It works for small, stable collections and admin screens, but it can produce duplicates or gaps when records are inserted or removed while the client is paging.

Cursor pagination uses an opaque token that represents a position in a stable ordering. It is better for active datasets, event streams, feeds, and high-volume collections. The token should be treated as an implementation detail. Clients should store and send it, not parse it.

Use deterministic ordering

Pagination is only reliable when ordering is deterministic. If the client sorts by a non-unique field, add a stable tie-breaker internally, such as the resource identifier or creation timestamp plus identifier. Without a tie-breaker, two rows with the same sort value can move between pages.

Document the default ordering. Do not change it silently. A default sort change can break clients that process pages incrementally.

Keep filters explicit and bounded

Filtering should use documented fields and operators. Avoid accepting arbitrary database expressions, raw SQL fragments, or implementation-specific field names. Those designs increase injection risk, leak internals, and make index planning difficult.

Support the filters clients actually need. Common examples are equality filters, date ranges, status values, ownership, and prefix search. For each filter, document the field, type, allowed operators, case sensitivity, time zone handling, and whether multiple values mean AND or OR.

Return 400 for unsupported filters rather than ignoring them. Silent ignore behaviour makes clients believe they are seeing a narrowed result when they are not.

Make sorting predictable

Use a documented sort syntax and keep it consistent across collection endpoints. A compact approach is sort=createdAt,-id, where a leading hyphen means descending order. Another approach is separate fields such as orderBy=createdAt desc,id desc. Either can work. The important part is that it is documented, validated, and consistent.

Reject unsupported sort fields. Do not pass sort input directly to a database query. Map public sort names to approved internal columns or expressions.

Return navigation information

A paginated response should tell the client how to get the next page. Cursor APIs can return nextPageToken in the body, a Link header with rel="next", or both. Header links are standard HTTP, while body tokens are often easier for SDKs and application developers. Whichever approach you choose, make it consistent.

Avoid requiring clients to construct the next URL by copying all filters, sort fields, and cursor state themselves. The server knows the correct next position. Give it back to the client.

Be careful with totals

A total count can be useful, but it is not always cheap or stable. Counting a large filtered dataset can be more expensive than returning a page. In active datasets, the count can change before the client reaches the next page.

If totals are provided, document whether they are exact, approximate, or omitted for expensive queries. Do not make every page wait for an exact count unless the product requirement justifies the cost.

Preserve query consistency across pages

The filters, sort order, page size, and authorisation context must remain consistent across page requests. If a cursor token encodes query state, reject attempts to reuse it with different filters or sort options. If it does not encode query state, clients must resend the same query parameters with the cursor.

Do not let clients change sort order halfway through a cursor sequence. That creates unclear boundaries and duplicate processing.

Design for limits and failure

Set a default page size and a maximum page size. Return clear validation errors when the requested size is too large. Use stable error formats so clients can distinguish invalid query parameters from temporary server failures.

For expensive filters, prefer an explicit 400 or 422 validation error over timing out. If a query shape is unsupported at scale, say so in the contract.

Conclusion

Good collection design is not just limit and offset. It requires bounded page sizes, stable ordering, documented filters, validated sorting, reliable navigation, and clear behaviour around totals. Cursor pagination is usually the safer default for changing data, while offset pagination is best reserved for small or stable datasets.