Published in Learning SQL·PinnedMember-only5 SQL Bad Habits You Need to BreakFrom beginners to professionals, chances are we’re all guilty of at least one of the following bad SQL habits. — Unlike a bad habit such as smoking, no nicotine gum exists to wean programmers off bad habits in SQL or any other language. However, unlike smoking, these SQL habits aren’t hurting anyone, so this begs a somewhat obvious question. Why stop? You need to stop writing good queries badly for…Sql7 min read
PinnedMember-onlyWelcome! Scroll down for latest work.Collections of my most popular work by niche: Python SQL Data Science/Technology Data Career/Job Hunting/Education1 min readWelcome! Scroll down for latest work.Collections of my most popular work by niche:PythonSQLData Science/TechnologyData Career/Job Hunting/Education----
Published in Pipeline: A Data Engineering Resource·3 days agoMember-onlyWhy Your Data Pipeline Didn’t Get UsedUnderstanding why some builds never make it to production can help you conceptualize and deploy better pipelines. — More Data Please While I’ve lost count of how many data pipelines I’ve built, I will always remember the pipelines that, through no fault of my own, got tossed aside. Working for weeks or months on a build that doesn’t make it to production is a characteristic of data science work. While something…Data Engineering5 min read
Published in Pipeline: A Data Engineering Resource·5 days agoMember-onlycURLing: Converting cURL to Python with CurlConverterEasily translate your pesky API’s cURL commands to Python. — Note: I am not affiliated with CurlConverter and the following does not represent an endorsement. I am simply a fan who would like to share his findings. cURL -> Python Often API documents are one of two things: Way too long or way too short. Regardless of length, complexity and clarity, one of…Data Engineering4 min read
Published in Pipeline: A Data Engineering Resource·Jul 22Member-onlyPrettify Your Python Logs with LoguruHow an easy-to-use logger can format your log data output. — Note: The following is an objective evaluation of a Python library. I am not associated with nor do I receive any form of compensation from any entity associated with Loguru. If you’re a data engineer, chances are, you will likely agree with two fundamental truths I’ve discovered about our craft: …Data Engineering5 min read
Published in Learning SQL·Jul 20Member-onlyCriteria for Choosing a SQL Course You’ll Actually Want to FinishBeyond reviews, price and personal judgement, here are criteria for how you should be evaluating SQL courses before you enroll. — Even if you excitedly begin an online SQL course created by providers like edX, Coursera or Udemy, you must know that, according to Massive Open Online Course (MOOC) data, you are only 5% likely to complete the course — even if you pay for it. Since there is an ever-growing…Data Engineering6 min read
Published in Learning SQL·Jul 13Member-onlyWhat Beginners Need to Know About SQL Query OptimizationHow even beginners can write optimized queries. — Even though I’m perfectly aware that you, as a beginner in SQL, are being overwhelmed with hours of content, lists of book recommendations and hundreds of lines of code, I’m going to make one more recommendation: Learn how to write performant SQL queries as a beginner. The good news is…Sql5 min read
Published in Pipeline: A Data Engineering Resource·Jul 11Member-only3 Ways Junior Data Engineers Can Make An Impact In Their First YearSubtle initiatives to positively impact your team and credibility. — When I began as a data engineer I developed a misconception that I had to be involved in high-visibility, revenue-generating projects to properly contribute. Reading about how companies salivate at the thought of hiring, (or, after mass layoffs) poaching, data engineering talent and then not being immediately assigned to a…Data Engineering4 min read
Published in Pipeline: A Data Engineering Resource·Jul 5Member-onlyUsing Python to Extract Datasets, Tables From SQL Query StringsHow to use Python to extract datasets and table names from plaintext SQL queries. — When writing SQL there is often an out-of-sight, out-of-mind mentality. Once a query is executed, as long as it isn’t a recurring query or otherwise scheduled job, there is not much reason to save query text. After all, once you run a query in many environments, including Google Cloud Platform’s…Data Engineering6 min read
Published in Learning SQL·Jun 27Member-onlySQL Quality Assurance QueriesHow to construct queries in SQL to confirm data accuracy. — My most time-consuming task at work is not creating ingestion pipelines, writing complex queries or AirFlow orchestration. Each of these tasks results in a clear, observable output generated by Python or a similar engine. Given a reasonable amount of clarity, technical knowledge and domain awareness, these tasks can be completed…Sql6 min read
