论文部分内容阅读
在线视频服务是互联网服务的重要内容,存储是在线视频服务提供的基础.HDFS作为面向通用文件的云存储系统,被很多视频服务网站采用,但其负载均衡工具没有考虑利用视频文件在线播放时的带宽消耗特性使集群的带宽资源得到更充分的利用.为解决这一问题,提出视频存储场景下的负载均衡方法 VOBM,它对视频文件在线播放时的带宽消耗与视频文件的码率、数据块大小和访问热度的关系进行了分析并建立了新的负载评估模型,在此基础上它在负载方案生成和负载调度两个环节中加入了对带宽消耗因素的考虑.在HDFS原有负载均衡工具的基础上实现了该方法,实验证明该方法能够有效避免高带宽消耗数据块的聚集,在高带宽消耗视频文件作为服务访问热点的实验场景中,该方法在90%的场景中优于原有负载均衡方法,最高能使数据节点集群中瓶颈节点的带宽峰值降低20%.
Online video service is an important part of Internet service, and storage is the basis of online video service.HDFS is used by many video service websites as cloud storage system for common files, but its load balancing tools do not consider the use of video files when playing online In order to solve this problem, this paper puts forward the VOBM, which is based on the bandwidth consumption of video files and the rate of video files, Size and visit the heat of the relationship between the analysis and the establishment of a new load evaluation model, on the basis of which in the load plan generation and load scheduling two links into the consideration of the bandwidth consumption factor in the original HDFS load balancing tool The experimental results show that the proposed method can effectively avoid the aggregation of high-bandwidth data blocks. In the experimental scenario of high-bandwidth consuming video files as the hotspot for service access, this method is superior to the original Load balancing method, the maximum can make the data node cluster bottleneck node bandwidth peak 20 %.