concurrent.futures模块提供了一种高级别的接口,用于异步执行可调用对象。它包含两个主要的类:ThreadPoolExecutor和ProcessPoolExecutor,分别用于线程池和进程池的并发执行。这两个类实现了对concurrent.futures.Executor接口的支持。
以下是concurrent.futures的基本用法:
ThreadPoolExecutor的用法:from concurrent.futures import ThreadPoolExecutor
def my_function(x):
return x * x
# 创建线程池
with ThreadPoolExecutor(max_workers=3) as executor:
# 提交任务到线程池
future = executor.submit(my_function, 5)
# 获取任务结果(阻塞直到任务完成)
result = future.result()
print("Result:", result)
ProcessPoolExecutor的用法:from concurrent.futures import ProcessPoolExecutor
def my_function(x):
return x * x
# 创建进程池
with ProcessPoolExecutor(max_workers=3) as executor:
# 提交任务到进程池
future = executor.submit(my_function, 5)
# 获取任务结果(阻塞直到任务完成)
result = future.result()
print("Result:", result)
map方法:ThreadPoolExecutor和ProcessPoolExecutor都提供了map方法,用于将可调用对象应用于可迭代的参数集合,返回一个结果的迭代器。
from concurrent.futures import ThreadPoolExecutor
def my_function(x):
return x * x
# 创建线程池
with ThreadPoolExecutor(max_workers=3) as executor:
# 使用map方法执行任务
results = executor.map(my_function, [1, 2, 3, 4, 5])
# 获取结果
for result in results:
print(result)
submit和as_completed实现并发:import concurrent.futures
import time
def my_function(x):
time.sleep(1) # 模拟耗时操作
return x * x
# 创建线程池
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
# 提交多个任务
futures = [executor.submit(my_function, i) for i in range(5)]
# 使用as_completed按完成顺序获取结果
for future in concurrent.futures.as_completed(futures):
result = future.result()
print(result)
concurrent.futures模块可以方便地实现并发执行,但对于涉及到全局解释器锁(GIL)的CPU密集型任务,它并不能充分发挥多核处理器的优势。concurrent.futures通常是一种简单而有效的并发模型。submit方法提交任务,并通过as_completed按完成的顺序获取结果。总体而言,concurrent.futures提供了一种高层次、易用的并发编程接口,适用于多种并发场景。