close

DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS
danie_rozin profile

Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS

#ai #webdev #dataengineering #tutorial
Comments
4 min read
Apache Data Lakehouse Weekly: April 3–9, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: April 3–9, 2026

#news #data #dataengineering #opensource
Comments
7 min read
ETL vs ELT: Which One Should You Use and Why?
robert_njuguna profile

ETL vs ELT: Which One Should You Use and Why?

#dataengineering #database #data #luxdev
Image 1
Comments
7 min read
AWS Lake Formation: Why Your Data Lake Permissions Are Probably a Mess (And How to Fix That)
soumyadeep_basu_3101d3ac1 profile

AWS Lake Formation: Why Your Data Lake Permissions Are Probably a Mess (And How to Fix That)

#dataengineering #awsdatalake #aws
Comments
3 min read
Airflow vs Prefect vs Dagster: Picking the Right Orchestrator in 2026
datastackx profile

Airflow vs Prefect vs Dagster: Picking the Right Orchestrator in 2026

#dataengineering #python #airflow #dagster
Comments
6 min read
Your Customer Table Has Duplicates You Can't See With SQL How I Built a Cross-Platform Identity Resolution Layer for a Dark Kitchen Data Platform
nerdbossstm profile

Your Customer Table Has Duplicates You Can't See With SQL How I Built a Cross-Platform Identity Resolution Layer for a Dark Kitchen Data Platform

#dataengineering #apachespark #kafka #deltalake
Image Image Image 3
Comments
8 min read
How to Bypass the Pandas "Object Tax": Building an 8x Faster CSV Engine in C
nareshcn2 profile

How to Bypass the Pandas "Object Tax": Building an 8x Faster CSV Engine in C

#python #performance #dataengineering #datascience
Comments
2 min read
PostgreSQL Foreign Data Wrappers: Cross-Database Queries Explained
philip_mcclarence_2ef9475 profile

PostgreSQL Foreign Data Wrappers: Cross-Database Queries Explained

#database #dataengineering #postgres #sql
Comments
4 min read
How Google Maps Predicts Traffic in Real Time: Live Data and ETA Explained
helloashish99 profile

How Google Maps Predicts Traffic in Real Time: Live Data and ETA Explained

#googlemaps #traffic #gps #dataengineering
Comments
3 min read
ETL vs ELT: Two Paradigms, One Goal
edmund_eryuba profile

ETL vs ELT: Two Paradigms, One Goal

#dataengineering #datascience #database #cicd
Comments
5 min read
How I cut Python JSON memory overhead from 1.9GB to ~0MB (11x Speedup)
nareshcn2 profile

How I cut Python JSON memory overhead from 1.9GB to ~0MB (11x Speedup)

#python #c #performance #dataengineering
Comments
2 min read
Building Market Data Pipelines Without REST Boilerplate
infoway_api profile

Building Market Data Pipelines Without REST Boilerplate

#api #architecture #backend #dataengineering
Image 1
Comments 1
4 min read
I Got a Teams Message: 'Why Do We Need a Semantic Layer?'
datatechbridge profile

I Got a Teams Message: 'Why Do We Need a Semantic Layer?'

#analytics #architecture #data #dataengineering
Comments
7 min read
ETL vs ELT: Which One Should You Use and Why?
rachel_muriuki_c5062dd89a profile

ETL vs ELT: Which One Should You Use and Why?

#dataengineering #etl #elt #bigdata
Image Image 3
Comments
5 min read
Python was too slow for 10M rows—So I built a C-Bridge (and found the hidden data loss)
nareshcn2 profile

Python was too slow for 10M rows—So I built a C-Bridge (and found the hidden data loss)

#python #cpp #performance #dataengineering
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.