Event driven architecture is very powerful and well suited for a distributed microservices environment. By introducing a broker intermediary, event driven architecture offers a decoupled architecture, easier scalability, and a much higher degree of resiliency. But it’s much harder to set up correctly…

In 2021, my team worked on migrating Wix’s 2000 microservices from self-hosted Kafka clusters to a multi-cluster Managed cloud platform (Confluent Cloud) in a seamless way that did not need to involve the owners of these services. The migration was done during regular traffic without any downtime. This was the…

Choosing which database to use is one of the most important decisions you can make when working on a new microservice. If you realize down the line that you’ve made the wrong choice, migrating to another database is a very costly and risky procedure. Each database technology (and type) has…

Unlike with UI frameworks, or software industry buzzwords, data storage technologies, i.e. databases are famously slow to change, and new innovation takes longer to be adopted. For a large organization, switching to a different OLTP database as your main workhorse happens very rarely. In recent years, with the advent of…

Fibers are the backbone of the highly performant, asynchronous and concurrent abilities of ZIO. They are lightweight “green threads” implemented by the ZIO runtime system. Forking any IO[E, A] effect means it will immediately run on a new fiber. The ZIO documentation recommends that non-advanced programmers avoid fibers and instead…

Wix’s distributed system of 2000 clustered microservices is required to process billions of business events every day with very high speed in a highly concurrent fashion. There is a need to balance the load between the various cluster nodes, such that no bottlenecks are created. For serving HTTP requests, this…

Wix has a huge scale of traffic. more than 500 billion HTTP requests and more than 1.5 billion Kafka business events per day. Caching the responses and events is critical for having the best performance and resilience for Wix’s 1500 microservices. As traffic to your service grows, customers can face…

Recently I have learned a lot about what qualities make for a user friendly (and non-user friendly) client library. Over the last 1.5 years I have been part of a team at Wix that’s developing event-driven data streaming infra on top of the Kafka message broker. The most important part…

Recently I had the privilege of sitting down with Confluent’s Tim Berglund to discuss how Wix developers use Apache Kafka at scale in a unique way. Kafka consumers and producers are used as the event-driven messaging backbone between Wix’s various microservices — 1500 in total. …