Neon Databases Now Support Compute Prewarming

Adding read replicas nearly crashed our production database. Everyone says to add read replicas to scale a read-heavy database. It’s the standard playbook. We followed it, adding two PostgreSQL replicas to offload queries from our primary instance. Instead of seeing performance gains, our primary DB's CPU usage spiked. We started getting alerts for replication lag that grew from seconds to minutes. During peak traffic, writes to the primary began to slow down as it struggled to keep up with the replication stream. The issue wasn't the reads; it was the nature of our writes. The application performed frequent, large bulk updates. This created a firehose of changes that the primary had to serialize and ship to the replicas, overwhelming its resources. We had scaled our read capacity but created a new, more dangerous bottleneck on the write path. The solution was to move away from replicas and instead implement application-level caching with Redis for the most common queries. This reduced the overall load far more effectively than replicas ever could. What's a 'standard' scaling practice that backfired on your team? Let's connect — I share lessons like this regularly. #SystemDesign #Database #Scalability

Key-Value Store is one of the simplest yet most powerful data storage models, where data is stored as a key and its corresponding value. It forms the backbone of many large-scale systems like caching layers, distributed databases, and real-time applications due to its high performance, scalability, and simplicity. As part of Low-Level Design (LLD), I implemented a Distributed Key-Value Store in C++ from scratch. The design uses consistent hashing for partitioning, replication for fault tolerance, and quorum-based reads/writes (R/W/N) to balance consistency and availability. It includes a Coordinator for request routing, a Replication Manager, and thread-safe storage nodes, along with handling basic failure scenarios and performance considerations. This helped me gain deeper insights into distributed systems and scalable design. Github repository: https://lnkd.in/dJ_ksrAA #cplusplus #systemdesign #lowleveldesign #distributedsystems #scalability #redis #distributedkeyvalue

Applications must execute as one or more stateless, share-nothing processes. A process must not rely on local memory or filesystem state to function correctly across requests. Instead, all persistent or shared data must live in backing services (databases, caches, object storage). ✅ Correct Pattern (Good) Use Redis / DB for sessions Use S3 / object storage for files Treat every process as disposable Think of each process as: A pure function: output = f(request, external_state)

The database benchmarking work from Alex Padula at TidesDB is fascinating: https://lnkd.in/eyt9SJTg https://lnkd.in/eSHA5QK7 On Sysbench, TidesDB showed a 5x to 7x performance advantage.  HammerDB TPROC-C narrowed that gap to 1.02x, with TidesDB still ahead. Similar engines on the same hardware, but very different results. That’s the workload.  Sysbench is a narrow test. HammerDB TPROC-C brings contention and scaling. Is it a coincidence that this same workload exposed the path to a 2.5x performance gain in MariaDB? https://lnkd.in/eCjJMbzc If you want to know how a system behaves under real pressure, you need a benchmark that applies that pressure. That’s the point of HammerDB. #databases #benchmarking #mysql #mariadb #performance #opensource #hammerdb

Great breakdown from Steve, this is exactly why benchmark tooling matters! Sysbench can tell you how fast your engine runs in a straight line, but HammerDB TPROC-C tells you how it handles real traffic. If you're making database decisions based on production-like workloads, HammerDB is the tool to reach for. Running workloads, profiling, tracing and patching, then repeat. #benchmarking #sql #databases #analysis #mariadb #tidesdb #hammerdb

Stop storing tokens in your database. It's 2026, and some systems are still doing this. Every request hitting the database leads to more queries, increased latency, and added complexity for no good reason. We moved tokens to Redis, implementing the following strategies: - Set expiration on creation - Auto-cleanup (no cron, no hacks) - Zero unnecessary database load The result? - Faster performance - Cleaner architecture - More scalability Your database should store data, not temporary garbage.

The three PostgreSQL metrics I check first on any incident call. Not CPU. Not disk. Not even query latency. Those are symptoms. I want root causes in the first 60 seconds. 𝟭. 𝗔𝗰𝘁𝗶𝘃𝗲 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀 + 𝘄𝗮𝗶𝘁 𝗲𝘃𝗲𝗻𝘁𝘀. What's happening right now? Are queries waiting on locks, I/O, or CPU? Wait events tell you what PostgreSQL is actually blocked on -- not what it's doing, but what it's waiting for. 𝟮. 𝗟𝗼𝗰𝗸 𝗰𝗵𝗮𝗶𝗻𝘀. Not just "are there locks" -- what's the chain? A single long-running ALTER TABLE can cascade and block 50 queries, each of which blocks 5 more. Finding the root blocker is the fastest path to resolution. 𝟯. 𝗥𝗲𝗰𝗲𝗻𝘁 𝗾𝘂𝗲𝗿𝘆 𝗽𝗹𝗮𝗻 𝗰𝗵𝗮𝗻𝗴𝗲𝘀. Did a query plan regress in the last 24 hours? A deploy, an ANALYZE, or a data threshold can silently change how PostgreSQL executes a query. The timing often correlates perfectly with when the incident started. These three checks narrow the problem space faster than any dashboard. Wait events tell you what the database is stuck on. Lock chains tell you who's blocking whom. Plan changes tell you if the optimizer made a bad decision. Between the three, you've usually identified the root cause before you've finished your first coffee. What's your first check when the database is on fire? I'm curious if others have a different playbook. #PostgreSQL #IncidentResponse #Database #DevOps #DBA

💥 Our system crashed at 2 AM… and it wasn’t because of traffic. We had: ✔ Good APIs ✔ Clean code ✔ Scalable infrastructure But still… everything slowed down. 👉 The real problem? Bad Database Design. At peak load: Queries took 3–5 seconds CPU usage spiked 🔥 Users started dropping off We thought it was a scaling issue… But it wasn’t. 🔍 After digging deep, we found: ❌ No proper indexing ❌ Too many joins across tables ❌ Poorly designed relationships ❌ Over-normalized schema 👉 In short: The database couldn’t handle real-world usage. ⚡ What we changed: ✔ Added indexes on high-frequency queries ✔ Reduced joins using smart denormalization ✔ Optimized schema design ✔ Introduced caching (Redis) 📈 The result? API latency dropped from 3s → 200ms System handled 5x more traffic Zero downtime during peak 🚀 💡 Big Lesson: “Your backend is only as strong as your database design.” #SystemDesign #DatabaseDesign #BackendEngineering #SoftwareArchitecture #TechLessons #Scalability

System Design Fact #3: Fast systems aren't built on one database. They are built on the right mix. Not all data needs the same latency, and choosing the wrong layer is the fastest way to kill your performance. Here is how modern architectures layer their storage: 1) The Cache Layer (Redis / Memcached) Latency: Microseconds Best for: Hot data, active sessions, auth tokens. 2) The Relational Layer (Postgres / MySQL/Oracle) Latency: Low Milliseconds Best for: Transactions, ACID compliance, complex user data. 3) The Scalable Layer (DynamoDB / Cassandra) Latency: Consistent low-ms at massive scale. Best for: Events, logs, time-series writes. 4) The Object Layer (S3 / GCS) Latency: High (Hundreds of ms). Best for: Images, videos, backups, and "infinite" scale. The Bottom Line: Engineers don't just pick a database; they design a data flow. Match your storage to your speed requirements. #SystemDesign #Backend #SoftwareEngineering #Scalability #CloudComputing

💡 MongoDB Quick Tip #199 – The Replication Bottleneck Nobody Noticed Experiencing a gradual drop in write performance… but no visible CPU or disk issues? 🤔 This is a common scenario many teams overlook. 👉 The hidden culprit? Replication lag. As highlighted in this quick tip : ⚠️ Replication lag can silently impact write throughput 🌐 Network or disk bottlenecks often slow down replication speed 📊 Monitoring replication metrics helps detect issues early ⚖️ Balanced hardware across replica members ensures better stability 💡 Key Insight: Even when your primary looks healthy, replication inefficiencies can degrade overall performance without obvious signs. 📌 Don’t just monitor system resources — always keep an eye on replication health. #MongoDB #DatabasePerformance #DBA #Replication #NoSQL #TechTips #GenexDBS #DataEngineering