Two Ways to Process Data
When you have a lot of data to process—logs, events, analytics—you can handle it in batches (chunks at a time, on a schedule) or as a stream (continuously, as it arrives). Each approach has tradeoffs. Here’s the high-level comparison.
Batch processing
Batch processing means processing data in large, discrete blocks (batches). You run the job at scheduled intervals (e.g. every hour, every night) or after accumulating a certain amount of data (e.g. every 10,000 records). Examples: nightly ETL jobs, daily report generation, bulk image resizing. | Pros | Cons |