详解python操作hbase数据的方法介绍

配置 thriftpython使用的包 thrift个人使用的python 编译器是pycharm community edition. 在工程中设置中,找到project interpreter, 在相应的工程下,找到package,然后选择 “+” 添加, 搜索 hbase-thrift (python client for hbase thrift interface),然后安装包。安装服务器端thrift。参考官网,同时也可以在本机上安装以终端使用。thrift getting started也可以参考安装方法 python 调用hbase 范例首先,安装thrift下载thrift,这里,我用的是thrift-0.7.0-dev.tar.gz 这个版本tar xzf thrift-0.7.0-dev.tar.gz cd thrift-0.7.0-dev sudo ./configure –with-cpp=no –with-ruby=no sudo make sudo make install然后,到hbase的源码包里,找到src/main/resources/org/apache/hadoop/hbase/thrift/执行thrift –gen py hbase.thrift mv gen-py/hbase/ /usr/lib/python2.4/site-packages/ (根据python版本可能有不同)获取数据示例 1

# coding:utf-8
from thrift import thrift
from thrift.transport import tsocket
from thrift.transport import ttransport
from thrift.protocol import tbinaryprotocol
from hbase import hbase
# from hbase.ttypes import columndescriptor, mutation, batchmutation
from hbase.ttypes import *
import csv
def client_conn():
# make socket
transport = tsocket.tsocket(‘hostname,like:localhost’, port)
# buffering is critical. raw sockets are very slow
transport = ttransport.tbufferedtransport(transport)
# wrap in a protocol
protocol = tbinaryprotocol.tbinaryprotocol(transport)
# create a client to use the protocol encoder
client = hbase.client(protocol)
# connect!
transport.open()
return client
if __name__ == “__main__”:
client = client_conn()
# r = client.getrowwithcolumns(‘table name’, ‘row name’, [‘column name’])
# print(r[0].columns.get(‘column name’)), type((r[0].columns.get(‘column name’)))
result = client.getrow(“table name”,”row name”)
data_simple =[]
# print result[0].columns.items()
for k, v in result[0].columns.items(): #.keys()
#data.append((k,v))
# print type(k),type(v),v.value,,v.timestamp
data_simple.append((v.timestamp, v.value))
writer.writerows(data)
csvfile.close()
csvfile_simple = open(“data_xy_simple.csv”, “wb”)
writer_simple = csv.writer(csvfile_simple)
writer_simple.writerow([“timestamp”, “value”])
writer_simple.writerows(data_simple)
csvfile_simple.close()
print “finished”

会基础的python应该知道result是个list,result[0].columns.items()是一个dict 的键值对。可以查询相关资料。或者通过输出变量,观察变量的值与类型。说明:上面程序中 transport.open()进行链接,在执行完后,还需要断开transport.close()目前只涉及到读数据,之后还会继续更新其他dbase操作。

以上就是详解python操作hbase数据的方法介绍的详细内容,更多请关注 第一php社区 其它相关文章!

Posted in 未分类

发表评论