Analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment
The aim of this paper is the analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment. The analysis was based on comparison between both mentioned tools with use of large data set, represented by 28 million records. Research was provided with use of script...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Lublin University of Technology
2024-03-01
|
| Series: | Journal of Computer Sciences Institute |
| Subjects: | |
| Online Access: | https://ph.pollub.pl/index.php/jcsi/article/view/4060 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The aim of this paper is the analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment. The analysis was based on comparison between both mentioned tools with use of large data set, represented by 28 million records. Research was provided with use of scripts and queries destined for Apache Hive and Apache Pig, and then executed 10 times on environment brought by created virtual machine. Those methods were performed on the same data sets for 16 times according to previously prepared research scenarios. As the conclusion, authors had observed that Apache Hive is more efficient tool, than Apache Pig.
|
|---|---|
| ISSN: | 2544-0764 |