Issuu

International Research Journal of Engineering and Technology (IRJET) Volume: 10 Issue: 05 | May 2023

www.irjet.net

e-ISSN: 2395-0056 p-ISSN: 2395-0072

Multi-Cloud Services Digvijay Gore

Aniruddha Vaze

CSE Department Rajarambapu Institute of Technology, Rajaramnagar Sangli, India

Pruthvi Belgaonkar

Rushikesh Suryanwanshi

CSE Department Rajarambapu Institute of Technology, Rajaramnagar Sangli, India

---------------------------------------------------------------------***---------------------------------------------------------------------Abstract— Data management has a major and important role in the systematic working of an organization and institute. There are a lot of problems when it comes to data management and storage. Many organizations don’t have a proper way to store, organize and handle data, and also can’t afford to have that storage facility in the early stage of a startup, but now with help of new technology like cloud computing, it is possible to store, handle, organize and retrieve easily at any place and any time with an additional benefit of paying off what we use and how much we use, But this also has a drawback, many news organizations don’t know how to use this facility a do whole big time consuming manual process, But this problem can also be solved by doing this whole process of data storage automatically, This can be done with the new side of technology such as Ansible, Terraform etc.

Hadoop is a Data Distribution tool which is generally used in many industries because it is majorly used for managing large amounts of data and also configuring higher computational power with MapReduce Cluster. This Clustering process is very complicated in nature and also very difficult to Handle so Ansible Automation Tool comes into place to configure Hadoop Cluster automatically also it can be managed. Ansible is used to launch multiple processes very quickly and efficiently also main use cases of Ansible are provisioning, application deployment, software management, continuous deployment of applications, automation etc. Ansible is used to provide the underlying hosts and network devices also hypervisors, and computer hosts. It can install services, and add computer hosts, services and applications inside any environment.

II. LITERATURE REWIEW

This paper presents the design and implementation of an automation framework for data management and storage on a cloud platform with help of automation tools like Ansible. This system will be useful for everyone who wants to store and handle their data easily and autonomously in just one click without any long procedure.

Iqbaldeep Kaur, et al. [1], According to author, "Big Data" refers to methods and tools for quickly storing, distributing, managing, and analysing massive datasets. Big data can be structured, unstructured, or semi-structured, making it impossible for traditional data management techniques to handle it. Hadoop is the main platform for organising Big data, which also addresses the issue of how to make it useful for analytics. With a relatively high level of fault tolerance, Hadoop is an open-source software project that enables the distributed processing of enormous data collections.

Keywords – Hadoop, Ansible, Cloud Computing, Terraform

I. INTRODUCTION In this, today’s rapidly growing world data management and storage play the most important role in one’s life. Speaking of data storage, management and handling when it comes to organizations, institutes and startups it plays a very crucial role. Organizations face problems with data storage and management without having a proper way or platform to store their data. When it comes to data storage on cloud platforms it is a very long and difficult process. To overcome these we need a smart and efficient way to which we can solve the problem of data storage and management autonomously.

Impact Factor value: 8.226

Mansaf Alam and Kashish Ara Shakil, et al. [2], Describes the Big Data in this category in terms of its quantity, worth, variety, and speed. On the other hand, bulk data is consumed and possibly produced by longrunning analytical and decision support queries employing Hadoop-based systems. Harshawardhan S. Bhosale, et al. [3], According to the possibility for faster scientific discipline advancements due to the analysis of massive amounts of data, these technical hurdles must be overcome for effective and quick processing of Big Data. at all phases of the analysis pipeline, from data gathering through

ISO 9001:2008 Certified Journal

Page 1029