this post was submitted on 19 Oct 2023
1716 points (98.8% liked)

Programmer Humor

32560 readers
280 users here now

Post funny things about programming here! (Or just rant about your favourite programming language.)

Rules:

founded 5 years ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[–] oce@jlai.lu 41 points 1 year ago (1 children)

Your knowledge of data engineering may be limited. SQL is predominant in data processing nowadays. FOSS tools such as DBT allows to write efficient data processing pipelines with SQL and some YAML config without the need for a general purpose coding language.
Why would anyone want that? Because SQL has the interesting property of describing the result you want rather than describing how to compute it. So you can put inside the database, a query engine with decades of optimizations, that will make a much better job at finding the best execution plan than the average developer.
It also means it's easier to train people for data processing nowadays.

[–] NOT_RICK@lemmy.world 3 points 1 year ago

Learning DBT was pretty easy for me as a data analyst. Now I’m contributing to my company’s data warehouse instead of just pulling existing data.