最新消息:

比JSON快10倍的序列化包:msgpack Python

未分类 admin 12765浏览 0评论

MessagePack
MessagePack简称msgpack,官方网站是http://msgpack.org/ ,代码可以在github上查看https://github.com/msgpack

官方介绍

“Extremely efficient object serialization library for cross-language communication.It’s like JSON, but very fast and small.”

从官方的性能测试结果来看,它能够比google protocol buffers快4倍,比json快10倍多。

官方的测试结果是C++的代码,那么Python下的性能又怎么样呢?
python下使用msgpack需要安装

sudo pip install msgpack-python

当然,你也可以通过源码安装,源码地址:https://github.com/msgpack/msgpack-python
msgpack python性能测试

测试代码:

# -*- coding: utf-8 -*-
#
# Copyright 2012 rebill.info
#

"""
:Author: rebill
"""
import msgpack
import json
from timeit import Timer

class MsgpackTest(object):
    """
    msgpack performance test
    """

    def __init__(self):
        self.small = {'key':'value'}
        self.big = []
        for i in range(1000):
            self.big.append({'str':'val_%d'%i, 'num':i})

    def test_msgpack_small(self):
        d = msgpack.dumps(self.small)
        msgpack.loads(d)

    def test_msgpack_big(self):
        d = msgpack.dumps(self.big)
        msgpack.loads(d)

    def test_json_small(self):
        d = json.dumps(self.small)
        json.loads(d)

    def test_json_big(self):
        d = json.dumps(self.big)
        json.loads(d)

if __name__ == '__main__':
    number = 10000
    mtest = MsgpackTest()

    t = Timer(mtest.test_msgpack_small)
    print 'test_msgpack_small run %s sec' % t.timeit(number)

    t = Timer(mtest.test_json_small)
    print 'test_json_small run %s sec' % t.timeit(number)

    t = Timer(mtest.test_msgpack_big)
    print 'test_msgpack_big run %s sec' % t.timeit(number)

    t = Timer(mtest.test_json_big)
    print 'test_json_big run %s sec' % t.timeit(number)

运行环境1:

Asus eeepc(AMD C-50 800 x 2, 2G RAM)
ArchLinux 3.4.2-2-ARCH x86_64

运行结果1:

test_msgpack_small run 0.0982592105865 sec
test_json_small run 0.780169963837 sec
test_msgpack_big run 55.8631219864 sec
test_json_big run 351.957278013 sec

运行环境2:

SamSung R458 (Intel(R) Core(TM)2 Duo CPU T6400 @ 2.00GHz x 2, 2G RAM x 2)
Ubuntu 3.2.0-25-generic-pae x86

运行结果2:

test_msgpack_small run 0.138391017914 sec
test_json_small run 0.168692827225 sec
test_msgpack_big run 9.05132102966 sec
test_json_big run 42.3488309383 sec

从结果看,随着CPU的增强,序列化速度有明显提升。
小数据的情况下,JSON在低端CPU上性能表现不佳,在高端CPU上的表现跟msgpack接近。
大数据的情况下,msgpack的性能比JSON快5-6倍。
PS. 如果是涉及大量序列化运算的应用,建议选择高频的CPU。

转载请注明:爱开源 » 比JSON快10倍的序列化包:msgpack Python

您必须 登录 才能发表评论!