引入进程池
在学习线程池之前，我们先看一个例子
from multiprocessing import Process
import time
def task(name):
print("name",name)
time.sleep(1)
if __name__ == "__main__":
start = time.time()
p1 = Process(target=task,args=("safly1",))
p2 = Process(target=task, args=("safly2",))
p3 = Process(target=task, args=("safly3",))
p1.start()
p2.start()
p3.start()
p1.join()
p2.join()
p3.join()
print("main")
end = time.time()
print(end- start)
输出如下：
name safly1 name safly2 name safly3 main 1.2071197032928467
以上的方式是一个个创建进程，这样的耗费时间才1秒多，虽然高效，但是有什么弊端呢？
如果并发很大的话，会给服务器带来很大的压力，所以引入了进程池的概念
使用ProcessPoolExecutor进程池
Python3.2开始，标准库为我们提供了concurrent.futures模块，它提供了ThreadPoolExecutor和ProcessPoolExecutor两个类，实现了对threading和multiprocessing的进一步抽象，对编写线程池/进程池提供了直接的支持。
通过ProcessPoolExecutor 来做示例。
我们来看一个最简单的进程池
from concurrent.futures import ProcessPoolExecutor
import time
def task(name):
print("name",name)
time.sleep(1)
if __name__ == "__main__":
start = time.time()
ex = ProcessPoolExecutor(2)
for i in range(5):
ex.submit(task,"safly%d"%i)
ex.shutdown(wait=True)
print("main")
end = time.time()
print(end - start)
输出如下：
E:\python\python_sdk\python.exe "E:/python/py_pro/4 进程池.py" name safly0 name safly1 name safly2 name safly3 name safly4 main 3.212218999862671
简单解释下：
ProcessPoolExecutor（2）创建一个进程池，容量为2，循环submit出5个进程，然后就在线程池队列里面，执行多个进程，ex.shutdown(wait=True)意思是进程都执行完毕，在执行主进程的内容
使用shutdown
ex.shutdown(wait=True)是进程池内部的进程都执行完毕，才会关闭，然后执行后续代码
如果改成false呢？看如下代码
from concurrent.futures import ProcessPoolExecutor
import time
def task(name):
print("name",name)
time.sleep(1)
if __name__ == "__main__":
start = time.time()
ex = ProcessPoolExecutor(2)
for i in range(5):
ex.submit(task,"safly%d"%i)
ex.shutdown(wait=False)
print("main")
end = time.time()
print(end - start)
输出如下：
main 0.01500844955444336 name safly0 name safly1 name safly2 name safly3 name safly4
使用submit同步调用
同步调用：提交/调用一个任务，然后就在原地等着，等到该任务执行完毕拿到结果，再执行下一行代码
from concurrent.futures import ProcessPoolExecutor
import time, random, os
def piao(name, n):
print('%s is piaoing %s' % (name, os.getpid()))
time.sleep(1)
return n ** 2
if __name__ == '__main__':
p = ProcessPoolExecutor(2)
start = time.time()
for i in range(5):
res=p.submit(piao,'safly %s' %i,i).result() #同步调用
print(res)
p.shutdown(wait=True)
print('主', os.getpid())
stop = time.time()
print(stop - start)
输出如下：
E:\python\python_sdk\python.exe "E:/python/py_pro/4 进程池.py" safly 0 is piaoing 12996 0 safly 1 is piaoing 14044 1 safly 2 is piaoing 12996 4 safly 3 is piaoing 14044 9 safly 4 is piaoing 12996 16 主 12932 5.202786684036255 Process finished with exit code 0
使用submit异步调用
异步调用: 提交/调用一个任务，不在原地等着，直接执行下一行代码
# from multiprocessing import Process,Pool
from concurrent.futures import ProcessPoolExecutor
import time, random, os
def piao(name, n):
print('%s is piaoing %s' % (name, os.getpid()))
time.sleep(1)
return n ** 2
if __name__ == '__main__':
p = ProcessPoolExecutor(2)
objs = []
start = time.time()
for i in range(5):
obj = p.submit(piao, 'safly %s' % i, i) # 异步调用
objs.append(obj)
p.shutdown(wait=True)
print('主', os.getpid())
for obj in objs:
print(obj.result())
stop = time.time()
print(stop - start)
输出如下：
E:\python\python_sdk\python.exe "E:/python/py_pro/4 进程池.py" safly 0 is piaoing 1548 safly 1 is piaoing 7872 safly 2 is piaoing 1548 safly 3 is piaoing 7872 safly 4 is piaoing 1548 主 7808 0 1 4 9 16 3.202626943588257
输出信息的换行是我标识有输出停顿的
简单说下执行流程：
由于进程池容量是容纳2个进程，所以会2+2+1 三次进入线程池执行，花费3秒
如果我们改下上面的代码，修改的代码如下：
from concurrent.futures import ProcessPoolExecutor
import time, random, os
def piao(name, n):
print('%s is piaoing %s' % (name, os.getpid()))
time.sleep(1)
return n ** 2
if __name__ == '__main__':
p = ProcessPoolExecutor(2)
objs = []
start = time.time()
for i in range(5):
obj = p.submit(piao, 'safly %s' % i, i) # 异步调用
objs.append(obj)
for obj in objs:
print(obj.result())
p.shutdown(wait=True)
print('主', os.getpid())
stop = time.time()
print(stop - start)
输出如下：（同样我用换行，标识出输出的时间段了）
E:\python\python_sdk\python.exe "E:/python/py_pro/4 进程池.py" safly 0 is piaoing 7852 safly 1 is piaoing 8484 safly 2 is piaoing 7852 0 safly 3 is piaoing 8484 1 safly 4 is piaoing 7852 4 9 16 主 6816 3.178352117538452
进程池实现ftp
服务端：
from socket import *
from concurrent.futures import ProcessPoolExecutor
import os
server=socket(AF_INET,SOCK_STREAM)
server.setsockopt(SOL_SOCKET,SO_REUSEADDR,1)
server.bind(('127.0.0.1',8080))
server.listen(5)
def talk(conn,client_addr):
print('进程pid: %s' %os.getpid())
while True:
try:
msg=conn.recv(1024)
if not msg:break
conn.send(msg.upper())
except Exception:
break
if __name__ == '__main__':
p=ProcessPoolExecutor(5)
while True:
conn,client_addr=server.accept()
p.submit(talk,conn,client_addr)
客户端：
from socket import *
client=socket(AF_INET,SOCK_STREAM)
client.connect(('127.0.0.1',8080))
while True:
msg=input('>>: ').strip()
if not msg:continue
client.send(msg.encode('utf-8'))
msg=client.recv(1024)
print(msg.decode('utf-8'))
标签：__,ProcessPoolExecutor,name,python,print,time,进程 From： https://www.cnblogs.com/zhiminyu/p/17418465.html