脚本宝典收集整理的这篇文章主要介绍了Python中单线程、多线程和多进程的效率对比实验,脚本宝典觉得挺不错的,现在分享给大家,也给大家做个参考。
tITle: Python中单线程、多线程与多进程的效率对比实验
date: 2016-09-30 07:05:47
tags: [多线程,多进程,Python]
categories: [Python]
meta: Python中多线程和多进程的对比
Python是运行在解释器中的语言,查找资料知道,python中有一个全局锁(GIL),在使用多进程(Thread)的情况下,不能发挥多核的优势。而使用多进程(MultiPRocess),则可以发挥多核的优势真正地提高效率。
对比实验
资料显示,如果多线程的进程是CPU密集型的,那多线程并不能有多少效率上的提升,相反还可能会因为线程的频繁切换,导致效率下降,推荐使用多进程;如果是IO密集型,多线程进程可以利用IO阻塞等待时的空闲时间执行其他线程,提升效率。所以我们根据实验对比不同场景的效率
操作系统
CPU
内存
硬盘
Windows 10
双核
8GB
机械硬盘
<!--more-->
(1)引入所需要的模块
@H_360_58@import requests
import time
From threading import Thread
from multiprocessing import Process
def count(x, y):
# 使程序完成50万计算
c = 0
while c < 500000:
c += 1
x += x
y += y
def write():
f = open("test.txt", "w")
for x in range(5000000):
f.write("testwriten")
f.close()
def read():
f = open("test.txt", "r")
lines = f.readlines()
f.close()
_head = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) Applewebkit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36'}
url = "http://www.tieba.COM"
def http_request():
try:
webPage = requests.get(url, headers=_head)
html = webPage.text
return {"context": html}
except Exception as e:
return {"error": e}
# CPU密集操作
t = time.time()
for x in range(10):
count(1, 1)
print("Line cpu", time.time() - t)
# IO密集操作
t = time.time()
for x in range(10):
write()
read()
print("Line IO", time.time() - t)
# 网络请求密集型操作
t = time.time()
for x in range(10):
http_request()
print("Line Http Request", time.time() - t)
输出
CPU密集 | 95.6059999466 | 91.57099986076355 | 92.52800011634827 | 99.96799993515015 |
IO密集 | 24.25 | 21.76699995994568 | 21.769999980926514 | 22.060999870300293 |
网络请求密集型 | 4.519999980926514 | 8.563999891281128 | 4.371000051498413 | 14.671000003814697 |
counts = []
t = time.time()
for x in range(10):
thread = Thread(target=count, args=(1,1))
counts.append(thread)
thread.start()
while True:
e = len(counts)
for th in counts:
if not th.is_alive():
e -= 1
if e <= 0:
break
print(time.time() - t)
output |
---|
99.9240000248 |
101.26400017738342 |
102.32200002670288 |
def io():
write()
read()
t = time.time()
ios = []
t = time.time()
for x in range(10):
thread = Thread(target=count, args=(1,1))
ios.append(thread)
thread.start()
while True:
e = len(ios)
for th in ios:
if not th.is_alive():
e -= 1
if e <= 0:
break
print(time.time() - t)
Output |
---|
25.69700002670288 |
24.02400016784668 |
t = time.time()
ios = []
t = time.time()
for x in range(10):
thread = Thread(target=http_request)
ios.append(thread)
thread.start()
while True:
e = len(ios)
for th in ios:
if not th.is_alive():
e -= 1
if e <= 0:
break
print("Thread Http Request", time.time() - t)
Output |
---|
0.7419998645782471 |
0.3839998245239258 |
0.3900001049041748 |
counts = []
t = time.time()
for x in range(10):
process = Process(target=count, args=(1,1))
counts.append(process)
process.start()
while True:
e = len(counts)
for th in counts:
if not th.is_alive():
e -= 1
if e <= 0:
break
print("Multiprocess cpu", time.time() - t)
Output |
---|
54.342000007629395 |
53.437999963760376 |
t = time.time()
ios = []
t = time.time()
for x in range(10):
process = Process(target=io)
ios.append(process)
process.start()
while True:
e = len(ios)
for th in ios:
if not th.is_alive():
e -= 1
if e <= 0:
break
print("Multiprocess IO", time.time() - t)
Output |
---|
12.509000062942505 |
13.059000015258789 |
t = time.time()
httprs = []
t = time.time()
for x in range(10):
process = Process(target=http_request)
ios.append(process)
process.start()
while True:
e = len(httprs)
for th in httprs:
if not th.is_alive():
e -= 1
if e <= 0:
break
print("Multiprocess Http Request", time.time() - t)
|Output|
|0.5329999923706055|
|0.4760000705718994|
CPU密集型操作 | IO密集型操作 | 网络请求密集型操作 | |
---|---|---|---|
线性操作 | 94.91824996469 | 22.46199995279 | 7.3296000004 |
多线程操作 | 101.1700000762 | 24.8605000973 | 0.5053332647 |
多进程操作 | 53.8899999857 | 12.7840000391 | 0.5045000315 |
通过上面的结果,我们可以看到:
此文为1年随手写的,多谢评论区指出谬误,因数据是平均数,影响不大,故未做修改
以上是脚本宝典为你收集整理的Python中单线程、多线程和多进程的效率对比实验全部内容,希望文章能够帮你解决Python中单线程、多线程和多进程的效率对比实验所遇到的问题。
本图文内容来源于网友网络收集整理提供,作为学习参考使用,版权属于原作者。
如您有任何意见或建议可联系处理。小编QQ:384754419,请注明来意。