Bartosz - Streaming Data Engineer AI Skill

Senior Streaming Data Engineer who treats real-time pipelines as a distinct discipline, not batch processing running faster.

Bartosz covers the full real-time pipeline surface: Kafka and Flink cluster and topic design, exactly-once versus at-least-once processing tradeoffs, schema registry and schema evolution (Avro/Protobuf), consumer group and partition strategy, backpressure and consumer lag monitoring, streaming-to-warehouse landing patterns (CDC, Kafka Connect, streaming inserts), and stream processing job design (windowing, watermarks, stateful aggregation).

Who it's for

Data engineers, platform leads, and streaming-pipeline owners who need Kafka/Flink topic design, processing guarantees, and lag monitoring tied to real freshness and ordering requirements, not a copy-pasted default.

Key capabilities

• Kafka/Flink topic, partition, and processing-guarantee design mapped to real duplicate/drop cost
• Schema registry compatibility checks that catch a downstream consumer break, not just a registry pass
• Per-partition consumer lag and backpressure monitoring, not misleading aggregate lag
• CDC, Kafka Connect, or streaming-insert landing pattern chosen per freshness and volume need
• Windowing, watermark, and checkpoint/savepoint recovery design verified before go-live

How to use it

Paste Bartosz's SKILL.md into your Claude Project Instructions (or any AI system prompt), then describe your Streaming Data Engineer problem. Works with Claude, ChatGPT, and any AI chat. Under 2 minutes to install.