Description
This is a direct and to the point course that will get you quickly ETL’ing data from MySQL to BigQuery.
The lessons in this course are broken out into short How-Tos. Therefore you can take this course over the weekend and be ready to show off your skills on Monday morning!
Things that we will cover:
- Setup
- Setting up a GCP Account
- Credential and Authentication for security
- Python Environment Setup
- Extract
- Use Python to connect to MySQL
- Use Python’s pandas to export data
- Python library usage for saving files to file paths
- Transform
- Use Python functions to transform data
- Use Python pandas to transform data
- Use inline SQL during Extract for data transformation
- Load
- Use the BigQuery Python library
- Connect to BigQuery
- Load data to BigQuery
- Incremental Loads vs Truncate and Load
- Other data handling options during Load
After taking this course, you’ll be comfortable with the following pretty cool things:
- Connect to MySQL using Python
- Learn how to obscure your database credentials so you’re not exposing them in your code
- Usage of the os module for the purpose of saving files and hard coding fewer things.
- Use both Python and the pandas library to transform data on the fly during the Transformation phase of your ETL
- Learn how to use GBQ’s modules/libraries to make the loading of the data a very easy, straightforward task
Have fun, enjoy and keep growing!
Who this course is for:
- Business Intelligence Analysts
- Data Analysts
- Beginner Data Engineers
- Beginner Software Developers
- Data Power Users
Requirements
- Python installed (e.g. virtual environment, anaconda, etc…)
- Familiarity with SQL
- Familiarity with Python
- GCP Account for BigQuery Access
- An IDE like VS Code or PyCharm
Last Updated 9/2021
Download Links
Direct Download
ETL using Python: from MySQL to BigQuery.zip (1.1 GB) | Mirror
