Basics to Remember

Before getting into the weeds of using SQL...

Before diving too far into code, it's important to remember the role SQL now plays in many ETL pipelines.

In the olden days, a pipeline was by default made nearly entirely of pure SQL building sets from one carefully-maintained local server to another, all under the watchful gaze of a local sysadmin. Since the rise of SaaS, particularly for financial and sales operations, ETL functions often start with an external product, and usually pipe through or to other off-site products.

This is reflected heavily in some best practices (particularly those developed by the SaaS themselves), but not in other works. Flipping between the two can cause a bit of whiplash. Keeping the context of the documentation in mind while reading is an important step.