ETL using Python: from MySQL to BigQuery

Description

This is a direct and to the point course that will get you quickly ETL’ing data from MySQL to BigQuery.

The lessons in this course are broken out into short How-Tos. Therefore you can take this course over the weekend and be ready to show off your skills on Monday morning!

Things that we will cover:

  • Setup
    • Setting up a GCP Account
    • Credential and Authentication for security
    • Python Environment Setup
  • Extract
    • Use Python to connect to MySQL
    • Use Python’s pandas to export data
    • Python library usage for saving files to file paths
  • Transform
    • Use Python functions to transform data
    • Use Python pandas to transform data
    • Use inline SQL during Extract for data transformation
  • Load
    • Use the BigQuery Python library
    • Connect to BigQuery
    • Load data to BigQuery
    • Incremental Loads vs Truncate and Load
    • Other data handling options during Load

After taking this course, you’ll be comfortable with the following pretty cool things:

  • Connect to MySQL using Python
  • Learn how to obscure your database credentials so you’re not exposing them in your code
  • Usage of the os module for the purpose of saving files and hard coding fewer things.
  • Use both Python and the pandas library to transform data on the fly during the Transformation phase of your ETL
  • Learn how to use GBQ’s modules/libraries to make the loading of the data a very easy, straightforward task

Have fun, enjoy and keep growing!

Who this course is for:

  • Business Intelligence Analysts
  • Data Analysts
  • Beginner Data Engineers
  • Beginner Software Developers
  • Data Power Users

Requirements

  • Python installed (e.g. virtual environment, anaconda, etc…)
  • Familiarity with SQL
  • Familiarity with Python
  • GCP Account for BigQuery Access
  • An IDE like VS Code or PyCharm

Last Updated 9/2021

Download Links

Direct Download

ETL using Python: from MySQL to BigQuery.zip (1.1 GB) | Mirror

Leave a Reply

Your email address will not be published. Required fields are marked *