目录
about#
django3.2 + djanngorestframework3.12.4 + python3.9 + win10
Django默认的分页器适用于前后端不分离的项目使用,那么针对于前后端分离的项目,我们就要学习restframework提供的分页器来进行分页处理了。
所谓的分页器就是restframework写好的分页类,我们这里只需要学习分页器类的使用以及如何局部配置和全局配置,包括如何自定义分页器类,再加上注意事项也就这些了。
分页器是在有了查询结果(可能经过过滤之后):
- 如果有配置分页器,那么就走分页器进行分页,再将分页后的数据进行序列化返回给客户端。
- 如果没有配置分页器,那么直接序列化后返回给客户端。
首先说注意事项:
- 记得分页前我们应该主动加上排序,对一个有序的结果进行分页才是恰当的。否则你可能会遇到warning提示
UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list
,当你看拿到这个提示,你就要想着去加排序。 - 在全局进行分页配置之后,它只对使继承了
mixins.ListModelMixin
或generics.GenericAPIView
这些有list方法的视图类生效。对于继承了APIView的视图类,我们可以手动的去自定义分页类来进行分页处理,否则不生效。
主要的类:
# 分页基类,实现公共的功能,被下面那几个不同的分页类继承
BasePagination
# 主要的几个分页类
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
# 根据url上的page参数来进行翻页
PageNumberPagination
# 根据limit和offset参数进行翻页
# LimitOffsetPagination
# 根据游标进行翻页
# CursorPagination
# app分页/大数据量分页
必要的准备
models.py
,有203条数据。
from django.db import models
class UserInfo(models.Model):
user = models.CharField(verbose_name='用户名', max_length=32)
pwd = models.CharField(verbose_name='密码', max_length=32)
token = models.CharField(verbose_name='token', max_length=64, null=True, blank=True)
def __str__(self):
return self.user
class Meta:
ordering = ['id', ] # 我这里提前加上排序了
api/urls.py
:
from django.urls import path, include, re_path
from rest_framework.routers import DefaultRouter,SimpleRouter
from .views import views, auth_views, page_views,jwt_view
router = SimpleRouter()
router.register(prefix='pg', viewset=page_views.IndexView, basename='pg')
urlpatterns = []
urlpatterns += router.urls
"""
主路由代码长这样:
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('api/', include("api.urls")),
]
"""
settings.py
中的关于REST_FRAMEWORK
的配置是空的:
REST_FRAMEWORK = {}
如果有变动,后续会列出来,没列出来的,就没有变动。
PageNumberPagination#
先来看全局配置。
全局配置1(不推荐)#
主要是settings.py
文件:
REST_FRAMEWORK = {
# 使用PageNumberPagination分页类
"DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
# 每页的数据条数,注意,必须指定PAGE_SIZE,并且必须是大写的PAGE_SIZE,否则不生效
"PAGE_SIZE": 10
}
此时的views.py
长这样:
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class IndexView(ModelViewSet): # 这个视图类的返回会自动应用上分页
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
这样配置之后,我们访问:
get请求:http://127.0.0.1:8000/api/pg/
响应结果:
{
"count": 203, # 数据的总条数
"next": "http://127.0.0.1:8000/api/pg/?page=2", # 下一页地址
"previous": null, # 上一页地址,因为我们访问的是首页,所以没有上一页
"results": [ # 当前的页的数据集
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
........省略........
{
"id": 10,
"user": "zhangkai10",
"pwd": "123"
}
]
}
局部配置方式1(不推荐)#
这种是结合settings.py和视图类搭配使用。
settings.py
文件中只指定page_size,分页类在视图类中,谁用谁配置:
REST_FRAMEWORK = {
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
"PAGE_SIZE": 2,
}
此时的views.py
长这样:
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.decorators import action
from rest_framework.response import Response
from api.models import UserInfo
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
# 使用上分页类,page_size自动回去settings中找,如果setting中同样不指定page_size,分页器不会起作用
pagination_class = PageNumberPagination
请求也是正常发送就行了:
get请求:http://127.0.0.1:8000/api/pg/?page=2
{
"count": 203,
"next": "http://127.0.0.1:8000/api/pg/?page=3",
"previous": "http://127.0.0.1:8000/api/pg/",
"results": [
{
"id": 3,
"user": "zhangkai3",
"pwd": "123"
},
{
"id": 4,
"user": "zhangkai4",
"pwd": "123"
}
]
}
不过,不建议这种方式,因为单独在配置文件中,配置了PAGE_SIZE
参数,而没有配置DEFAULT_PAGINATION_CLASS
参数,你Django运行会报WARNINGS提示:
Watching for file changes with StatReloader
Performing system checks...
System check identified some issues:
WARNINGS:
?: (rest_framework.W001) You have specified a default PAGE_SIZE pagination rest_framework setting, without specifying also a DEFAULT_PAGINATION_CLASS.
HINT: The default for DEFAULT_PAGINATION_CLASS is None. In previous versions this was PageNumberPagination. If you wish to define PAGE_SIZE globally whilst defining pagination_class on a per-view basis you may silence this check.
System check identified 1 issue (0 silenced).
February 21, 2023 - 21:27:43
Django version 3.2, using settings 'drf.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
局部配置方式21(推荐)#
这种就是重写PageNumberPagination
类,这样的话,我们有些参数就可以自己定制了。
settings.py
此时关于分页的啥都不用写了:
REST_FRAMEWORK = {
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
# "PAGE_SIZE": 10,
}
我将自定义的分页类也省事儿写到views.py
中,比较便于观察,后期可以单独找个文件存放。
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.decorators import action
from rest_framework.response import Response
from api.models import UserInfo
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyPageNumberPagination(PageNumberPagination):
# 默认每页显示的数据条数
page_size = 2
# 自定义分页参数名称,由默认的page改为下面定义的pg
page_query_param = 'pg'
# 如果没有下面的参数,那么默认显示的条数由上面的page_size值决定
# 如果指定了下面的参数,可以指定每页显示的条数
page_size_query_param = 'pg_size'
# 虽然我们可以自己定义每页显示的条数了,但是也不能想写多少写多少,它必须小于等于max_page_size
# 如果大于max_page_size值则每页按照max_page_size值来决定
max_page_size = 10
# 当url上 pg=last,则返回最后一页,默认值就是last
last_page_strings = ('last', ) # 如果自定义,则必须是元组的形式,否则不生效
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
pagination_class = MyPageNumberPagination
现在请求中,分页就可以有几种方式了:
# 返回第二页,每页2条数据
http://127.0.0.1:8000/api/pg/?pg=2
# 返回第二页,每页5条数据
http://127.0.0.1:8000/api/pg/?pg=2&pg_size=5
# 返回第二页,每页10条数据,pg_size值大于max_page_size,按max_page_size值返回,即10条数据
http://127.0.0.1:8000/api/pg/?pg=2&pg_size=100
# 返回最后一页
http://127.0.0.1:8000/api/pg/?pg=last
自定义分页器响应结果#
如果客户端输入的页码有问题,比如输入的不是数字,或者输入0,或者输入其它字符,我们后端应该进行提示并且返回首页内容。
看看怎么处理,这里的示例用的是PageNumberPagination
分页器。
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.views import APIView
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import (
PageNumberPagination, LimitOffsetPagination,
CursorPagination, _positive_int, _divide_with_ceil
)
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyPageNumberPagination(PageNumberPagination):
page_size = 2
page_query_param = 'pg'
page_size_query_param = 'pg_size'
max_page_size = 10
last_page_strings = ('last',)
msg = {}
def get_page_number(self, request, paginator):
page_number = request.query_params.get(self.page_query_param, 1)
# 如果输入的是last,表示要访问最后一页
if page_number in self.last_page_strings:
return paginator.num_pages
if not page_number.isdecimal():
self.msg['msg'] = "输入的页码不是数字,已为你返回首页内容"
return 1
page_number = int(page_number)
if page_number < 1:
self.msg['msg'] = '输入的页码必须大于等于1,已为你返回首页内容'
return 1
if page_number > paginator.num_pages:
self.msg['msg'] = "页码超范围了,没取到值,已为你返回首页内容"
return 1
return page_number
def paginate_queryset(self, queryset, request, view=None):
"""
重写分页方法
"""
self.msg.clear()
page_size = self.get_page_size(request)
if not page_size:
return None
paginator = self.django_paginator_class(queryset, page_size)
page_number = self.get_page_number(request, paginator)
try:
# 我们在get_page_number方法中做了一些判断
self.page = paginator.page(page_number)
except InvalidPage as exc:
# 然后再这个try中可能有其他的没考虑到的,都让它返回首页
self.msg['msg'] = str(exc)
self.page = paginator.page(1)
if paginator.num_pages > 1 and self.template is not None:
self.display_page_controls = True
self.request = request
return list(self.page)
def get_paginated_response(self, data):
""" 重写响应结果 """
return Response(OrderedDict([
('total_page', self.page.paginator.num_pages), # 经过分页后的总页数
('total_data', self.page.paginator.count), # 总的记录条数
('current_page', self.page.number), # 当前页的页码
('next_page_url', self.get_next_link()),
('previous_page_url', self.get_previous_link()),
('results', data), # 分页后的当前页的数据
('msg', self.msg), # 页码输入有问题,在这里展示错误信息提示
('code', 0), # 返回状态码
]))
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
pagination_class = MyPageNumberPagination
settings.py
不用配置分页相关内的参数:
REST_FRAMEWORK = {}
来个测试,比如用户瞎输入:
# 请求
http://127.0.0.1:8000/api/pg/?pg=1aaa
# 响应
{
"total_page": 102,
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?pg=2",
"previous_page_url": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
],
"msg": {
"msg": "输入的页码不是数字,已为你返回首页内容"
},
"code": 0
}
LimitOffsetPagination#
全局配置1(不推荐)#
settings.py
文件:
REST_FRAMEWORK = {
# 使用LimitOffsetPagination分页类
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
"DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.LimitOffsetPagination",
# 每页的数据条数,注意,必须指定PAGE_SIZE,并且必须是大写的PAGE_SIZE,否则不生效
"PAGE_SIZE": 2
}
此时的views.py
长这样:
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class IndexView(ModelViewSet): # 这个视图类的返回会自动应用上分页
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
这样配置之后,我们访问:
get请求:http://127.0.0.1:8000/api/pg/
响应结果:
{
"count": 203,
"next": "http://127.0.0.1:8000/api/pg/?limit=2&offset=2",
"previous": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
]
}
# limit是每页展示的条数,这个可以动态调整,offset是查询数据的开始偏移量,相当于上面PageNumberPagination的page参数,默认 offset
get请求:http://127.0.0.1:8000/api/pg/?limit=2&offset=2
{
"count": 203,
"next": "http://127.0.0.1:8000/api/pg/?limit=2&offset=4",
"previous": "http://127.0.0.1:8000/api/pg/?limit=2",
"results": [
{
"id": 3,
"user": "zhangkai3",
"pwd": "123"
},
{
"id": 4,
"user": "zhangkai4",
"pwd": "123"
}
]
}
全局配置一看就行了,用的不多,还是看看怎么自己封装,在局部使用。
局部配置方式1(不推荐)#
这种是结合settings.py和视图类搭配使用。
settings.py
文件中只指定page_size,分页类在视图类中,谁用谁配置:
REST_FRAMEWORK = {
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.LimitOffsetPagination",
"PAGE_SIZE": 2
}
此时的views.py
长这样:
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.decorators import action
from rest_framework.response import Response
from api.models import UserInfo
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
# 使用上分页类,page_size自动回去settings中找,如果setting中同样不指定page_size,分页器不会起作用
pagination_class = PageNumberPagination
局部配置方式2(推荐)#
这种就是重写PageNumberPagination
类,这样的话,我们有些参数就可以自己定制了。
settings.py
此时关于分页的啥都不用写了:
REST_FRAMEWORK = {
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
# "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.LimitOffsetPagination",
# "PAGE_SIZE": 2
}
我将自定义的分页类也省事儿写到views.py
中,比较便于观察,后期可以单独找个文件存放。
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyLimitOffsetPagination(LimitOffsetPagination):
# 从offset开始,展示limit的条数
default_limit = 2
# 指定地址栏的limit参数名,默认是limit http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 例如指定为 lt 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&offset=4
limit_query_param = 'lt'
# 指定偏移量的名字,默认是 offset http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 如果指定为 ot 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&ot=4
offset_query_param = 'ot'
# limit_query_param参数值范围是小于等于max_limit参数值,否则就以max_limit值为准
max_limit = 10 # 默认为None
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
pagination_class = MyLimitOffsetPagination
现在请求中,分页就可以有几种方式了:
# 从offset为4开始,返回2条数据
GET http://127.0.0.1:8000/api/pg/?lt=2&ot=4
{
"count": 203,
"next": "http://127.0.0.1:8000/api/pg/?lt=2&ot=6",
"previous": "http://127.0.0.1:8000/api/pg/?lt=2&ot=2",
"results": [
{
"id": 5,
"user": "zhangkai5",
"pwd": "123"
},
{
"id": 6,
"user": "zhangkai6",
"pwd": "123"
}
]
}
自定义分页器响应结果#
如果客户端输入的页码有问题,比如输入的不是数字,或者输入0,或者输入其它字符,我们后端应该进行提示并且返回首页内容。
看看LimitOffsetPagination
分页类中怎么搞,当然了,思路就是下面的思路,可能不是最优解,可能有bug,毕竟是我一拍脑门子自己搞出来的。
思路还是老套路,重写某些方法,或者增加一些判断。
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination, _positive_int
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyLimitOffsetPagination(LimitOffsetPagination):
# 从offset开始,展示limit的条数
default_limit = 2
# 指定地址栏的limit参数名,默认是limit http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 例如指定为 lt 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&offset=4
limit_query_param = 'limit'
# 指定偏移量的名字,默认是 offset http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 如果指定为 ot 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&ot=4
offset_query_param = 'offset'
# limit_query_param参数值范围是小于等于max_limit参数值,否则就以max_limit值为准
max_limit = 10 # 默认为None
# 自定义分页的错误提示
msg = {"offset": "", "limit": ""}
def get_offset(self, request):
"""
对于offset值的各种处理都在这搞定就行了,我想到的就是下面那些判断,如果有bug,自行调整
"""
# 首先获取用户输入的offset
offset = request.query_params.get(self.offset_query_param, None)
# 如果客户端访问的是这个地址:http://127.0.0.1:8000/api/pg/?limit=2
# 要么是就是首页,要么是用户手动调整了参数,没传offset参数,那么就给它返回首页就行了
# 也返回首页,因为不带offset
if offset is None:
self.msg['offset'] = "首页"
return 0
if not offset.isdecimal(): # 输入的不是正整数
self.msg['offset'] = "输入的offset值非法,返回首页"
return 0
offset = int(offset)
if offset >= self.count:
self.msg['offset'] = "查询页码超范围,已为你返回首页"
return 0
if offset == 0:
self.msg['offset'] = "当前输入的offset为0,已为你返回首页"
return 0
# 其它情况就是用户的正常输入了,正常返回即可
return offset
def paginate_queryset(self, queryset, request, view=None):
"""
这种查询效率也是不高,queryset是所有的记录数,然后通过切片将该当前页的数据切出来,再进行序列化传递给客户端
"""
self.msg.clear() # 清空自定义的消息提示
self.request = request
self._queryset = queryset
self.limit = self.get_limit(request)
if self.limit is None: # 没有limit直接不走分页了
return None
self.count = self.get_count(queryset)
# -------------- 接下来,就是对关于分页的各种情况的判断处理了 -------------
# 表中没有数据
if self.count == 0:
self.msg['offset'] = "查询结果为空"
return []
# 上面源码 self.limit = self.get_limit(request)
# 中已经处理好了用户输入limit值大于max_limit的情况,这里只是做下提示
if int(request.query_params.get(self.limit_query_param, self.default_limit)) > self.max_limit:
self.msg['limit'] = f'limit值不能大于设定值:{self.max_limit}'
self.offset = self.get_offset(request)
if self.count > self.limit and self.template is not None:
self.display_page_controls = True
return list(queryset[self.offset:self.offset + self.limit])
@property
def get_current_page(self):
# 用户的offset参数输入0、瞎输入、或者本就是首页那么当前页就是第一页
if self.offset <= 1:
return 1
else:
# 正常页码,就应该是总记录数除以每页条数再加一
# return self.offset // self.limit + 1
return _divide_with_ceil(self.offset, self.limit)
def get_paginated_response(self, data):
""" 重写响应结果 """
return Response(OrderedDict([
('total_data', self.count), # 总的记录条数
('current_page', self.get_current_page), # 当前页的页码
('next_page_url', self.get_next_link()),
('previous_page_url', self.get_previous_link()),
('results', data), # 分页后的当前页的数据
('msg', self.msg), # 页码输入有问题,在这里展示错误信息提示
('code', 0), # 返回状态码
]))
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
pagination_class = MyLimitOffsetPagination
settings.py
不用配置分页相关内的参数:
REST_FRAMEWORK = {}
来个测试,比如用户瞎输入:
# 正常的
GET http://127.0.0.1:8000/api/pg/?limit=2&offset=2
# 响应
{
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?limit=2&offset=4",
"previous_page_url": "http://127.0.0.1:8000/api/pg/?limit=2",
"results": [
{
"id": 3,
"user": "zhangkai3",
"pwd": "123"
},
{
"id": 4,
"user": "zhangkai4",
"pwd": "123"
}
],
"msg": {},
"code": 0
}
# 其它情况
GET http://127.0.0.1:8000/api/pg/?limit=2&offset=ada
{
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?limit=2&offset=2",
"previous_page_url": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
],
"msg": {
"offset": "输入的offset值非法,返回首页"
},
"code": 0
}
# 其他情况
GET http://127.0.0.1:8000/api/pg/?limit=2&offset=1111111111111
{
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?limit=2&offset=2",
"previous_page_url": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
],
"msg": {
"offset": "查询页码超范围,已为你返回首页"
},
"code": 0
}
CursorPagination#
基于cursor的分页提供了一个不透明的“游标”指示器,客户端可以使用它对结果集进行分页。这种分页样式只显示正向和反向控件,而不允许客户端导航到任意位置。
基于cursor的分页比其他方案更复杂。它还要求结果集呈现固定的顺序,并且不允许客户端任意索引结果集。但是,它确实提供了以下好处:
- 提供一致的分页视图。当恰当地使用时,
CursorPagination
可以确保客户端在翻阅记录时永远不会看到同一个项目两次,即使在分页过程中其他客户端正在插入新的项目时亦是如此。 - 支持使用非常大的数据集。对于极其庞大的数据集,使用基于offset的分页样式进行分页可能会变得效率低下或无法使用。基于cursor的分页方案具有固定时间的属性,并且不会随着数据集大小的增加而减慢。
正确使用基于cursor的分页需要注意一些细节。你需要考虑你希望该方案适用于什么样的顺序。默认值是按 "-created"
排序。这假设模型实例上必须有一个“created”时间戳字段,并且将呈现一个“timeline”样式的,最新添加的项在前面的分页视图。
可以通过重写分页类中的 'ordering'
属性,或将 OrderingFilter
筛选器类与 CursorPagination
一起使用来修改排序。当与 OrderingFilter
一起使用时,您应该考虑严格限制用户可以按其排序的字段。
正确使用cursor分页应使用有满足以下条件的排序字段:
- 应当是一个不变的值,例如时间戳、slug或其他在创建时只设置一次的字段。
- 应当是独一无二的,或者几乎是独一无二的。毫秒精度的时间戳就是一个很好的例子。cursor分页的这种实现使用了一种智能的"position plus offset" 样式,允许它正确地支持不严格唯一的值作为排序。
- 应当为可强制转化为字符串的不可为null的值。
- 不应当是浮点数。精度误差容易导致不正确的结果。提示:改用小数。
- 字段应具有数据库索引。
概括起来一句话:性能强大;只能上下翻页;数据集必须是有序的,排序字段最好加上索引;
所以,这个分页器就不方便全局配置了。
接下来,看下局部使用,也就是我们直接自定义分页器类。
settings.py
不用配置分页相关内的参数:
REST_FRAMEWORK = {}
然后来看views.py
中怎么搞:
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination, CursorPagination, _positive_int, _divide_with_ceil
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyCursorPagination(CursorPagination):
# 必须指定这个参数,默认值是 created
ordering = "id"
# page_size指定每页的数据条数
# 默认值是api_settings.PAGE_SIZE 也就是None
# 如果我们setting指定了PAGE_SIZE,就用那个值,但我们settings中关于分页都不写
page_size = 2
# 指定 cursor 查询参数的名称。默认为 'cursor'
# 这是一个加密的游标参数,我们一般没必要改动这个参数和参数值
# http://127.0.0.1:8000/api/pg/?cursor=cD0y&size=2
cursor_query_param = 'cursor'
# 指定分页数据条数的参数名
# http://127.0.0.1:8000/api/pg/?cursor=cD0y&size=2
page_size_query_param = 'size'
# page_size参数值范围是小于等于 max_page_size 参数值,否则就以 max_page_size 值为准
max_page_size = 10
# 想搞自定义的话,也是从这里入手,我就不展开说了,参考上面那个MyLimitOffsetPagination中的代码逻辑
def paginate_queryset(self, queryset, request, view=None):
# 因为父类的queryset是数据的总数量,我们这里想要获取这个值,可以简单的拿一下,然后剩下的逻辑还是让其父类的方法执行
self._queryset_size = len(queryset)
return super(MyCursorPagination, self).paginate_queryset(queryset, request, view=None)
@property
def get_current_page(self):
if self.cursor is None: # 为None时,是首页
return 1
else:
return (int(self.cursor[2]) // self.page_size) + 1
# 自定义响应结果,也是从这里入手
def get_paginated_response(self, data):
return Response(OrderedDict([
('total_data', self._queryset_size), # 数据总条数
('total_page', (self._queryset_size // self.page_size) + 1), # 根据数据总条数和每页数量算出来的总页数
('current_page', self.get_current_page), # 当前页的页码
('next', self.get_next_link()), # 下一页,如果为null表示没有下一页了
('previous', self.get_previous_link()), # 上一页,如果为null表示没有上一页了
('results', data), # 当前页码的数据
('code', 0), # 自定义返回状态码
]))
class IndexView(ModelViewSet):
queryset = UserInfo.objects.all()
serializer_class = UserInfoSerializer
pagination_class = MyCursorPagination
来个效果吧:
GET http://127.0.0.1:8000/api/pg/
{
"total_data": 203,
"total_page": 102,
"current_page": 1,
"next": "http://127.0.0.1:8000/api/pg/?cursor=cD0y",
"previous": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
]
}
# 可以不传size参数
GET http://127.0.0.1:8000/api/pg/?cursor=cD00
{
"total_data": 203,
"total_page": 102,
"current_page": 3,
"next": "http://127.0.0.1:8000/api/pg/?cursor=cD02",
"previous": "http://127.0.0.1:8000/api/pg/?cursor=cj0xJnA9NQ%3D%3D",
"results": [
{
"id": 5,
"user": "zhangkai5",
"pwd": "123"
},
{
"id": 6,
"user": "zhangkai6",
"pwd": "123"
}
],
"code": 0
}
# 可以传size参数
http://127.0.0.1:8000/api/pg/?cursor=cD00&size=3
{
"total_data": 203,
"total_page": 68,
"current_page": 2,
"next": "http://127.0.0.1:8000/api/pg/?cursor=cD03&size=3",
"previous": "http://127.0.0.1:8000/api/pg/?cursor=cj0xJnA9NQ%3D%3D&size=3",
"results": [
{
"id": 5,
"user": "zhangkai5",
"pwd": "123"
},
{
"id": 6,
"user": "zhangkai6",
"pwd": "123"
},
{
"id": 7,
"user": "zhangkai7",
"pwd": "123"
}
],
"code": 0
}
APIView中使用分页#
如果视图类继承了APIView,我们自己写逻辑,该如何使用分页呢,这里我们一起来看下。
这里还是选择settings.py
中不写分页相关的配置:
REST_FRAMEWORK = {}
urls.py
也要变动下:
from django.urls import path, include, re_path
from .views import views, auth_views, page_views,jwt_view
urlpatterns = [
path('pg/', page_views.IndexView.as_view()),
]
"""
主路由代码长这样:
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('api/', include("api.urls")),
]
"""
其它的都是视图类的代码了,我们分开来看。
PageNumberPagination#
还是使用我们自定义的分页器类。
代码主要是视图类views.py
:
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.views import APIView
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import (
PageNumberPagination, LimitOffsetPagination,
CursorPagination, _positive_int, _divide_with_ceil
)
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyPageNumberPagination(PageNumberPagination):
page_size = 2
page_query_param = 'pg'
page_size_query_param = 'pg_size'
max_page_size = 10
last_page_strings = ('last',)
msg = {}
def get_page_number(self, request, paginator):
page_number = request.query_params.get(self.page_query_param, 1)
# 如果输入的是last,表示要访问最后一页
if page_number in self.last_page_strings:
return paginator.num_pages
if not page_number.isdecimal():
self.msg['msg'] = "输入的页码不是数字,已为你返回首页内容"
return 1
page_number = int(page_number)
if page_number < 1:
self.msg['msg'] = '输入的页码必须大于等于1,已为你返回首页内容'
return 1
if page_number > paginator.num_pages:
self.msg['msg'] = "页码超范围了,没取到值,已为你返回首页内容"
return 1
return page_number
def paginate_queryset(self, queryset, request, view=None):
"""
重写分页方法
"""
self.msg.clear()
page_size = self.get_page_size(request)
if not page_size:
return None
paginator = self.django_paginator_class(queryset, page_size)
page_number = self.get_page_number(request, paginator)
try:
# 我们在get_page_number方法中做了一些判断
self.page = paginator.page(page_number)
except InvalidPage as exc:
# 然后再这个try中可能有其他的没考虑到的,都让它返回首页
self.msg['msg'] = str(exc)
self.page = paginator.page(1)
if paginator.num_pages > 1 and self.template is not None:
self.display_page_controls = True
self.request = request
return list(self.page)
def get_paginated_response(self, data):
""" 重写响应结果 """
return Response(OrderedDict([
('total_page', self.page.paginator.num_pages), # 经过分页后的总页数
('total_data', self.page.paginator.count), # 总的记录条数
('current_page', self.page.number), # 当前页的页码
('next_page_url', self.get_next_link()),
('previous_page_url', self.get_previous_link()),
('results', data), # 分页后的当前页的数据
('msg', self.msg), # 页码输入有问题,在这里展示错误信息提示
('code', 0), # 返回状态码
]))
class IndexView(APIView):
def get(self, request):
# 获取queryset数据集
queryset = UserInfo.objects.all()
# 实例化分页对象
page_obj = MyCursorPagination()
# 将queryset传进去进行分页处理
page_queryset = page_obj.paginate_queryset(queryset, request, self)
# 将分页后的数据进行序列化
serializer = UserInfoSerializer(instance=page_queryset, many=True)
# # 如果这么返回,只有当前页的数据
# return Response(serializer.data)
# 调用咱们自己的自定义分页方法
return page_obj.get_paginated_response(serializer.data)
访问也是一切如常:
GET http://127.0.0.1:8000/api/pg/?pg=1111
{
"total_page": 102,
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?pg=2",
"previous_page_url": null,
"results": [
{
"id": 1,
"user": "zhangkai1",
"pwd": "123"
},
{
"id": 2,
"user": "zhangkai2",
"pwd": "123"
}
],
"msg": {
"msg": "页码超范围了,没取到值,已为你返回首页内容"
},
"code": 0
}
LimitOffsetPagination#
还是使用我们自定义的分页器类。
代码主要是视图类views.py
:
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.views import APIView
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import (
PageNumberPagination, LimitOffsetPagination,
CursorPagination, _positive_int, _divide_with_ceil
)
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyLimitOffsetPagination(LimitOffsetPagination):
# 从offset开始,展示limit的条数
default_limit = 2
# 指定地址栏的limit参数名,默认是limit http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 例如指定为 lt 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&offset=4
limit_query_param = 'limit'
# 指定偏移量的名字,默认是 offset http://127.0.0.1:8000/api/pg/?limit=2&offset=4
# 如果指定为 ot 那么地址栏就是 http://127.0.0.1:8000/api/pg/?lt=2&ot=4
offset_query_param = 'offset'
# limit_query_param参数值范围是小于等于max_limit参数值,否则就以max_limit值为准
max_limit = 10 # 默认为None
# 自定义分页的错误提示
msg = {"offset": "", "limit": ""}
def get_offset(self, request):
"""
对于offset值的各种处理都在这搞定就行了,我想到的就是下面那些判断,如果有bug,自行调整
"""
# 首先获取用户输入的offset
offset = request.query_params.get(self.offset_query_param, None)
# 如果客户端访问的是这个地址:http://127.0.0.1:8000/api/pg/?limit=2
# 要么是就是首页,要么是用户手动调整了参数,没传offset参数,那么就给它返回首页就行了
# 也返回首页,因为不带offset
if offset is None:
self.msg['offset'] = "首页"
return 0
if not offset.isdecimal(): # 输入的不是正整数
self.msg['offset'] = "输入的offset值非法,返回首页"
return 0
offset = int(offset)
if offset >= self.count:
self.msg['offset'] = "查询页码超范围,已为你返回首页"
return 0
if offset == 0:
self.msg['offset'] = "当前输入的offset为0,已为你返回首页"
return 0
# 其它情况就是用户的正常输入了,正常返回即可
return offset
def paginate_queryset(self, queryset, request, view=None):
"""
这种查询效率也是不高,queryset是所有的记录数,然后通过切片将该当前页的数据切出来,再进行序列化传递给客户端
"""
self.msg.clear() # 清空自定义的消息提示
self.request = request
self._queryset = queryset
self.limit = self.get_limit(request)
if self.limit is None: # 没有limit直接不走分页了
return None
self.count = self.get_count(queryset)
# -------------- 接下来,就是对关于分页的各种情况的判断处理了 -------------
# 表中没有数据
if self.count == 0:
self.msg['offset'] = "查询结果为空"
return []
# 上面源码 self.limit = self.get_limit(request)
# 中已经处理好了用户输入limit值大于max_limit的情况,这里只是做下提示
if int(request.query_params.get(self.limit_query_param, self.default_limit)) > self.max_limit:
self.msg['limit'] = f'limit值不能大于设定值:{self.max_limit}'
self.offset = self.get_offset(request)
if self.count > self.limit and self.template is not None:
self.display_page_controls = True
return list(queryset[self.offset:self.offset + self.limit])
@property
def get_current_page(self):
# 用户的offset参数输入0、瞎输入、或者本就是首页那么当前页就是第一页
if self.offset <= 1:
return 1
else:
# 正常页码,就应该是总记录数除以每页条数再加一
# return self.offset // self.limit + 1
return _divide_with_ceil(self.offset, self.limit)
def get_paginated_response(self, data):
""" 重写响应结果 """
return Response(OrderedDict([
('total_data', self.count), # 总的记录条数
('current_page', self.get_current_page), # 当前页的页码
('next_page_url', self.get_next_link()),
('previous_page_url', self.get_previous_link()),
('results', data), # 分页后的当前页的数据
('msg', self.msg), # 页码输入有问题,在这里展示错误信息提示
('code', 0), # 返回状态码
]))
class IndexView(APIView):
def get(self, request):
# 获取queryset数据集
queryset = UserInfo.objects.all()
# 实例化分页对象
page_obj = MyLimitOffsetPagination()
# 将queryset传进去进行分页处理
page_queryset = page_obj.paginate_queryset(queryset, request, self)
# 将分页后的数据进行序列化
serializer = UserInfoSerializer(instance=page_queryset, many=True)
# # 如果这么返回,只有当前页的数据
# return Response(serializer.data)
# 调用咱们自己的自定义分页方法
return page_obj.get_paginated_response(serializer.data)
访问也没问题:
GET http://127.0.0.1:8000/api/pg/?limit=2&offset=2
{
"total_data": 203,
"current_page": 1,
"next_page_url": "http://127.0.0.1:8000/api/pg/?limit=2&offset=4",
"previous_page_url": "http://127.0.0.1:8000/api/pg/?limit=2",
"results": [
{
"id": 3,
"user": "zhangkai3",
"pwd": "123"
},
{
"id": 4,
"user": "zhangkai4",
"pwd": "123"
}
],
"msg": {},
"code": 0
}
CursorPagination#
还是使用我们自定义的分页器类。
代码主要是视图类views.py
:
from collections import OrderedDict
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.views import APIView
from rest_framework.serializers import ModelSerializer
from rest_framework.response import Response
from django.core.paginator import InvalidPage
from rest_framework.pagination import (
PageNumberPagination, LimitOffsetPagination,
CursorPagination, _positive_int, _divide_with_ceil
)
from api.models import UserInfo
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class MyCursorPagination(CursorPagination):
# 必须指定这个参数,默认值是 created
ordering = "id"
# page_size指定每页的数据条数
# 默认值是api_settings.PAGE_SIZE 也就是None
# 如果我们setting指定了PAGE_SIZE,就用那个值,但我们settings中关于分页都不写
page_size = 2
# 指定 cursor 查询参数的名称。默认为 'cursor'
# 这是一个加密的游标参数,我们一般没必要改动这个参数和参数值
# http://127.0.0.1:8000/api/pg/?cursor=cD0y&size=2
cursor_query_param = 'cursor'
# 指定分页数据条数的参数名
# http://127.0.0.1:8000/api/pg/?cursor=cD0y&size=2
page_size_query_param = 'size'
# page_size参数值范围是小于等于 max_page_size 参数值,否则就以 max_page_size 值为准
max_page_size = 10
# 想搞自定义的话,也是从这里入手,我就不展开说了,参考上面那个MyLimitOffsetPagination中的代码逻辑
def paginate_queryset(self, queryset, request, view=None):
# 因为父类的queryset是数据的总数量,我们这里想要获取这个值,可以简单的拿一下,然后剩下的逻辑还是让其父类的方法执行
self._queryset_size = len(queryset)
return super(MyCursorPagination, self).paginate_queryset(queryset, request, view=None)
@property
def get_current_page(self):
if self.cursor is None: # 为None时,是首页
return 1
else:
return (int(self.cursor[2]) // self.page_size) + 1
# 自定义响应结果,也是从这里入手
def get_paginated_response(self, data):
return Response(OrderedDict([
('total_data', self._queryset_size), # 数据总条数
('total_page', (self._queryset_size // self.page_size) + 1), # 根据数据总条数和每页数量算出来的总页数
('current_page', self.get_current_page), # 当前页的页码
('next', self.get_next_link()), # 下一页,如果为null表示没有下一页了
('previous', self.get_previous_link()), # 上一页,如果为null表示没有上一页了
('results', data), # 当前页码的数据
('code', 0), # 返回状态码
]))
class IndexView(APIView):
def get(self, request):
# 获取queryset数据集
queryset = UserInfo.objects.all()
# 实例化分页对象
page_obj = MyCursorPagination()
# 将queryset传进去进行分页处理
page_queryset = page_obj.paginate_queryset(queryset, request, self)
# 将分页后的数据进行序列化
serializer = UserInfoSerializer(instance=page_queryset, many=True)
# # 如果这么返回,只有当前页的数据
# return Response(serializer.data)
# 调用咱们自己的自定义分页方法
return page_obj.get_paginated_response(serializer.data)
访问下:
GET http://127.0.0.1:8000/api/pg/?cursor=cD0y
{
"total_data": 203,
"total_page": 102,
"current_page": 2,
"next": "http://127.0.0.1:8000/api/pg/?cursor=cD00",
"previous": "http://127.0.0.1:8000/api/pg/?cursor=cj0xJnA9Mw%3D%3D",
"results": [
{
"id": 3,
"user": "zhangkai3",
"pwd": "123"
},
{
"id": 4,
"user": "zhangkai4",
"pwd": "123"
}
],
"code": 0
}
常见报错#
UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list:#
这个不算报错,只是算是提示,因为在分页时,我们应该对数据进行排序,这样保证分页数据不会出现紊乱问题,所以,如果没有进行排序,就会提示这个Warning。
那么解决办法也是非常简单,加上排序呗。而加排序可以有两种方案。
方案1:在orm中进行排序
这个方案也就是在视图类中,加上排序。
# views.py
from rest_framework.viewsets import ModelViewSet, GenericViewSet
from rest_framework.serializers import ModelSerializer
from rest_framework.decorators import action
from rest_framework.response import Response
from api.models import UserInfo
from rest_framework.pagination import PageNumberPagination
class UserInfoSerializer(ModelSerializer):
class Meta:
model = UserInfo
exclude = ['token']
class IndexView(ModelViewSet):
# queryset = UserInfo.objects.all()
# 加上order_by排序防止出现warning UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list:
queryset = UserInfo.objects.all().order_by('id')
serializer_class = UserInfoSerializer
方案2:就是在模型类中加上排序
# models.py
from django.db import models
class UserInfo(models.Model):
user = models.CharField(verbose_name='用户名', max_length=32)
pwd = models.CharField(verbose_name='密码', max_length=32)
token = models.CharField(verbose_name='token', max_length=64, null=True, blank=True)
def __str__(self):
return self.user
class Meta:
# 就下面这行
ordering = ['id', ]
标签:分页,self,django,limit,pg,offset,import,page
From: https://www.cnblogs.com/dev-maintain-tester/p/17180900.html