Published inData Engineer ThingsHow to Simplify Data Pipeline Deployment with CI/CD in Data EngineeringDeploying code to production is a critical but often tedious task for data engineers. In traditional workflows, a data engineer must access…May 4May 4
Published inData Engineer ThingsThe Key to Providing Business Value Through DataOpsHave you ever found yourself reviewing code that you don’t understand? Or perhaps you’re a developer who’s been frustrated by business…Apr 17Apr 17
Published inData Engineer ThingsThe Biggest Mistake I Made as a Data EngineerWhen I joined as a senior data engineer, I was tasked with creating a data platform from scratch. I used powerful tools like Apache…Apr 15Apr 15
Published inData Engineer ThingsBulk Jobs vs. Delta Jobs in Data Engineering: Choosing the Right ApproachData engineering often involves extracting data from various sources and integrating it into a data platform, whether it’s a data…Mar 181Mar 181
Published inPython in Plain EnglishHow to Easily Extract Salesforce Data Using PySpark in PythonExtracting Salesforce data with PySpark can be a daunting task, especially when you’re faced with outdated modules and tricky dependencies…Mar 4Mar 4
Published inData Engineer ThingsMetrics for Data Engineering: Why, How, and Which Metrics to TrackData engineering can feel like a thankless job. You’re maintaining pipelines, handling data storage, and fielding endless requests without…Feb 23Feb 23
Published inData Engineer ThingsEnhancing Code Reusability with Python Packages in AWS GlueCode reusability is key to building efficient, maintainable, and scalable ETL pipelines. If you’re working with AWS Glue, leveraging Python…Feb 20Feb 20
Published inArt of Data EngineeringHow to Configure the GlueJobOperator in Apache AirflowData engineering often requires setting up workflows that seamlessly connect multiple tools. One common challenge is integrating Apache…Feb 9Feb 9
Published inCodeXWhy Apache Airflow Shouldn’t Be Used as a Data Processing EngineWhen working with Apache Airflow, it can be tempting to use its tasks and Python operators to extract, transform, and load (ETL) data…Jan 12Jan 12
Published inPython in Plain EnglishEasy to Implement Speech-To-Text ToolTranscribing with high-quality free tool using pythonSep 21, 20221Sep 21, 20221
