How to construct queries in SQL to confirm data accuracy. — My most time-consuming task at work is not creating ingestion pipelines, writing complex queries or AirFlow orchestration. Each of these tasks results in a clear, observable output generated by Python or a similar engine. Given a reasonable amount of clarity, technical knowledge and domain awareness, these tasks can be completed…

How understanding dependencies can make the difference in a failed run and a successful GitHub Actions deployment. — A few weeks ago I faced a persistent issue at work while writing, committing and deploying a Google Cloud Platform cloud function. For those curious about the details, it was a function that would generate alerts based on input from Google Cloud Logging, a process I covered a few weeks…

In a field dominated by the self-taught and self-assured, it’s important to overcome the stigma of asking for help. — I typically don’t have a problem writing emails. However, there is one time when I second-guess my ability to communicate errors, frame concepts and explain use cases. It’s when I have to ask for help. As I’m typing and backspacing a question mark my mind races: Am I communicating this…

Why side projects remain essential for building technical proficiency and nurturing professional development. — My first weekend of my first week in my current role I vividly remember writing queries while watching NFL football. That Sunday afternoon as the game stagnated, between Python and SQL, I wrote a few hundred lines of code. After I tested and tweaked my script, I uploaded it to…

How to use Google Cloud’s Logs API and Logging Query Language in Python to obtain real-time data on active GCP instances. — Using LQL in Python Logging query language (LQL) enables users to query Cloud Logging data, yet integrating LQL with Python allows for more dynamic querying than building and running queries in the UI. If you’re like me, up until a certain point, you may have only employed LQL queries in the GCP UI to…

Aggregating my career, job hunting and education content for data professionals. — Last updated May 2022 How to Prepare for Your First Day as a Data Engineer How to effectively refine skills and reduce the possibility of burning out before you even begin a data engineering…medium.com Unglamorous Data Engineering Projects That Can Make An Impact Although unassuming tasks, revising code and periodically rethinking infrastructure can make a big impact.medium.com

Aggregating my data science and technology content. — Last updated May 2022 Data Science Must-Reads That Aren’t Textbooks Books to help you understand data science concepts — without feeling like assigned reading.medium.com The Data Vice No One Talks About: Data Hoarding. How long-term data retention could violate the law… and not even yield business results.medium.com Should Data Scientists Have Agents? Why representing data scientists would be a low-risk, high-reward investment for both talent agents and data…medium.com

Aggregating my Python content. — Last Updated: May 2022. Ingesting Malformed Text Files in Python: Common Errors and Fixes How data engineers can identify, reconfigure and address malformed CSV files in batch pipelines.medium.com Multi-Column Transformation Using Config Files in Python How to optimize multi-column transformation operations using Python’s config library.medium.com How I Used Python and SQL to Find The Most Expensive Watches in the World How to Build an End-to-End Data Analysis Project Using Python/SQLtowardsdatascience.com