Applying MapReduce frameworks to a virtualization platform for Deep Web data source discovery by XIN Jie, CUI Zhi-ming, ZHAO Peng-peng, ZHANG Guang-ming, XIAN Xue-feng

Subjects: Get full text

Replication-Based Query Management for Resource Allocation Using Hadoop and MapReduce over Big Data by Ankit Kumar, Neeraj Varshney, Surbhi Bhatiya, Kamred Udham Singh

“…Data generation rates are so scary, creating pressure to implement costly and straightforward data storage and recovery processes. MapReduce model functionality is used for creating a cluster parallel, distributed algorithm, and large datasets. …”
Get full text

An AkNN Algorithm for High-Dimensional Big Data by Zhongwei Wang, Yefang Chen, Siyou Xiao, Jiangbo Qian

Subjects: Get full text

An Adaptive Subspace Similarity Search Approach by Jianxin Ren, Huahui Chen

Subjects: Get full text

Research on real-time fusion method of multi-source heterogeneous flight trajectory data stream by Zhuxi ZHANG, Wang TIAN, Shaochuan ZHU, Hongyan LIU, Xi ZHU

Subjects: Get full text

Research and application of prediction model based on ensemble BP neural network by Huimin ZHAO, Jiangtao LUO, Junchao YANG, Zheng XU, Xiao LEI, Lin LUO

Subjects: Get full text

DSMC: A Novel Distributed Store-Retrieve Approach of Internet Data Using MapReduce Model and Community Detection in Big Data by Xu Xu, Jia Zhao, Gaochao Xu, Yan Ding, Yunmeng Dong

“…The traditional single-host web spider and data store approaches have some problems such as low efficiency and large memory requirement, so this paper proposes a big data store-retrieve approach DSMC (distributed store-retrieve approach using MapReduce model and community detection) based on distributed processing. …”
Get full text

Summary of Large-Scale Grapb Partitioning Algoritbms by Jinfeng Xu, Yihong Dong, Shiyi Wang, Xianmang He, Huahui Chen

Subjects: Get full text

Design of Mobile Internet Big Data User Behavior Analysis Engine Based on Cloud Computing by Caixia Tao, Xiaojun Xie, Kang Chen, Lirong, Guo, Chun Liu

Subjects: Get full text

Survey of Distributed Computing Frameworks for Supporting Big Data Analysis by Xudong Sun, Yulin He, Dingming Wu, Joshua Zhexue Huang

“…In addition, we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.…”
Get full text

Parallel association rules incremental mining algorithm based on information entropy and genetic algorithm by Yimin MAO, Qianhu DENG, Zhigang CHEN

“…Aiming at the problems that in the big data environment, the Can-tree based incremental association rule algorithm had problems such as too much space occupation of the tree structure, inability to dynamically set the support threshold, and too much time consumption during the data transfer process between the Map and Reduce stages, the Map Reduce-based parallel association rules incremental mining algorithm using information entropy and genetic algorithm (MR-PARIMIEG)was proposed.Firstly, a similar items merging based on information entropy (SIM-IE) was designed to merge similar data items, and a Can tree based on the merged data set was constructed, thereby reducing the space occupation of the tree structure.Secondly, the dynamic support threshold obtaining using genetic algorithm (DST-GA) was proposed to obtain the relatively optimal dynamic support threshold in the big data environment, and frequent itemset mining was performed according to this threshold to avoid the unnecessary time consumption caused by mining redundant frequent patterns.Finally, in the process of MapReduce parallel operation, the parallel LZO data compression algorithm was used to compress the output data of the Map stage, thereby reducing the size of the transmitted data, and finally improving the running speed of the algorithm.Experimental simulation results show that MR-PARIMIEG has better performance when mining frequent item sets in the big data environment, and it is suitable for parallel processing of larger data sets.…”
Get full text

A Phoenix++ Based New Genetic Algorithm Involving Mechanism of Simulated Annealing by Luokai Hu, Jin Liu, Chao Liang, Fuchuan Ni, Hang Chen

“…Phoenix++ implements the MapReduce programming model that processes and generates large data sets with our parallel, distributed algorithm on a cluster. …”
Get full text

High Performance Frequent Subgraph Mining on Transaction Datasets: A Survey and Performance Comparison by Bismita S. Jena, Cynthia Khan, Rajshekhar Sunderraman

“…Although MapReduce came with many benefits, its disk I/O and non-iterative style model could not help much for FSM domain since subgraph mining process is an iterative approach. …”
Get full text

Maintenance Analysis Method Based on Big Data Processing Technology in Power Telecommunication Network by Zhimin Yang, Bin Wu, Ran Shu

“…The characteristics of power communication network equipment's maintenance,alarm and asset data were analyzed,and then the data preprocessing work was done by adopting attributes reduction and clustering method.The related index and influencing factors required by the analysis of equipment repair work were proposed.Meanwhile,the data mining method which combines the techniques of MapReduce and Apriori algorithms was presented.Finally,the proposed data mining method was applied to mine the actual network data.According to the mined results,the network maintenance work was analyzed from the perspectives of maintenance distribution,maintenance effectiveness on operated service,maintenance time-cost and so forth.…”
Get full text

DRAV: Detection and repair of data availability violations in Internet of Things by Jinlin Wang, Haining Yu, Xing Wang, Hongli Zhang, Binxing Fang, Yuchen Yang, Xiaozhou Zhu

“…DRAV uses algorithms in the MapReduce programming framework, and these include detection and repair algorithms based on enhanced conditional function dependency for data consistency violation, MapJoin, and ReduceJoin algorithms based on master data for k -nearest neighbor–based integrity violation detection, and repair algorithms. …”
Get full text

A discovery strategy for APT anomaly based on homologous behavior analysis by Yihan YU, Yu FU, Xiaoping WU, Hongcheng LI

“…As APT（advanced persistent threat）attacks are increasingly frequently,higher requirements for the detection of APT attacks were proposed.It was an effective method to early discover the attack behavior of APT based on homologous behavior analysis.Aiming at the problem of low efficiency of data authentication caused by excessive data,the historical behavior database with data label technology was established and the database was stored in the cloud.Relying on the Hadoop platform and the aggregate computing ability of MapReduce and the pseudorandom permutation technique,the whole traffic parallel detection of the network was realized.In order to determine whether there was a APT attack behavior,the detection of APT attacks was implemented by comparing the data labels in the database.Test results show that the proposed method can detect the abnormal behavior of APT from the network as soon as possibleand improve the efficiency of the whole data flow detection.…”
Get full text

Data Analysis Method of Intelligent Analysis Platform for Big Data of Film and Television by Youwen Ma, Yi Wan

“…The method selects Hadoop open source cloud platform as the basis, combines the MapReduce distributed programming model and HDFS distributed file storage system and other key cloud computing technologies. …”
Get full text

Supporting Efficient Family Joins for Big Data Tables via Multiple Freedom Family Index by Qiang Zhu, Chao Zhu

“…The Hadoop/MapReduce framework has been widely utilized for processing big data. …”
Get full text

An Improved Clustering Algorithm of Tunnel Monitoring Data for Cloud Computing by Luo Zhong, KunHao Tang, Lin Li, Guang Yang, JingJing Ye

“…It is a clustering algorithm using the MapReduce within cloud computing that deals with data. …”
Get full text

A Binary Cuckoo Search Big Data Algorithm Applied to Large-Scale Crew Scheduling Problems by José García, Francisco Altimiras, Alvaro Peña, Gino Astorga, Oscar Peredo

“…In this article, we propose a cuckoo search binary algorithm using the MapReduce programming paradigm implemented in the Apache Spark tool. …”
Get full text