Understanding Big Data File Systems - HDFS and DBFS HD

Before getting into Data Ingestion using NiFi as well as Data Processing using Spark let us get an overview of File Systems used in the Big Data ecosystem. Join this channel to get access to perks: https://www.youtube.com/channel/UCakdSIPsJqiOLqylgoYmwQg/join Let us get an overview of File Systems that are used as part of Big Data Clusters, both on-prem as well as the cloud. Here is the link for the material covered as part of the session - https://github.com/dgadiraju/itversity-books/blob/master/Data%20Engineering%20Bootcamp/40%20Big%20Data%20ecosystem%20-%20Overview/02%20Understanding%20Big%20Data%20File%20Systems%20-%20HDFS%20and%20DBFS.ipynb * Understanding Storage Servers * List of File Systems * Understanding Hadoop Storage (HDFS) * HDFS Architecture * HDFS Commands – Overview * Customizing Properties * Overview of DBFS Commands Similar to HDFS and DBFS we use AWS or Azure specific commands to manage files on S3 or Azure Blob respectively. They provide web interfaces as well for the same. Here is the complete playlist about Spark for Certifications: https://www.youtube.com/playlist?list=PLf0swTFhTI8rMmW7GZv1-z4iu_-TAv3bi Here is the complete playlist about Free Data Engineering Bootcamp: https://www.youtube.com/playlist?list=PLf0swTFhTI8pBe2Vr2neQV7shh9Rus8rl * Join our Meetup group - https://www.meetup.com/itversityin/ * Enroll for our labs - https://labs.itversity.com/plans * Subscribe to our YouTube Channel for Videos - http://youtube.com/itversityin/?sub_confirmation=1 * Access Content via our GitHub - https://github.com/dgadiraju/itversity-books * Lab and Content Support using Slack