How to do event-streaming on the cloud at La Mobilière
Situation & Goal:
La Mobilière is in the middle of their cloud transformation journey. The entire IT infrastructure will be moved to the cloud. This is a huge undertaking and event-streaming is also a central part. Event-streaming is especially important because it provides data connectivity and is the connection between the legacy and the cloud world. From a business perspective, La Mobilière wants to become quicker and more flexible by moving to the cloud. With Kafka as a Service, they are decoupling their data streams in time and space, enabling them to integrate and deploy applications faster and with more flexibility, as well as analyze and evaluate data efficiently.
About La Mobilière:
La Mobilière aims to offer its customers optimal and holistic advice on insurance solutions. For this reason, they are pioneers in digitalization. It was founded in Berne in 1826 and has been headquartered there ever since.
La Mobilière has an on-prem Kafka. This instance needs to be replaced by Kafka as a Service to properly run in the cloud. Before everything can be moved to the cloud, there’s a transition period, where some data-streams are on-prem, while others are in the cloud. During the transition, the infrastructure on the cloud must still be able to exchange data with applications on-prem (“backward-compatibility”) as well as with new applications in the cloud (“forward-compatibility”). This hybrid cloud approach ensures that during the transition phase, the two worlds are still able to exchange data where needed.
La Mobilière’s sophisticated cloud strategy also presents a few key technical challenges. Especially since the event-streaming infrastructure will operate on the cloud, governance policies and zero-trust security models must be considered when evaluating a possible implementation. Security demands on the cloud and on-premises are quite different, adding an extra layer of complexity. So, the main technical challenges to be considered, are:
- Enterprise governance rules that require certain safety, legal and organizational constraints to be fulfilled
- Integration of roles and security patterns
- Integrating zero-trust environments on-premises and on the cloud
- Automation (infrastructure as code) with e. g. HashiCorp’s Terraform
- Integrating monitoring with tools on the Azure Cloud
- Self-service provisioning of service accounts, topics and acls
- Replication of data and schemas from on-prem
- Replication of data from the cloud to be accessed on-prem
But the question remained: How can this even be done? The engineers at La Mobilière and at SPOUD evaluated the situation and came to the conclusion that Confluent Cloud could solve most of their needs. To test their hypothesis, they set up a PoC.
Solution & Benefit:
SPOUD consulted the team at La Mobilière with their knowledge about Kafka, Kafka as a Service and Confluent Cloud. By setting up the PoC, SPOUD illustrated where there was room for improvement regarding security and interoperability. SPOUD built the basis to estimate the effort to build Kafka as a Service and thus laid the foundation for the project.
Our goal was to kickstart the Kafka cloud migration at La Mobilière. In 3 months, we evaluated the best possible solution to do event-streaming on the cloud and set up a PoC to test our evaluation.
In detail, we:
In detail, we did:
- Set up an architecture for zero-trust hybrid Confluent Kafka as a Service Cloud
- Built a PoC cluster, to prove cloud integration possibilities – using the latest and greatest APIs
- Integrated Kafka with new Confluent Governance
- Automated setup of Kafka with an “Infrastructure as Code” approach
- Integrated Confluent Cloud monitoring API with the existing observability system on the Azure Cloud
Right now, la Mobilière and SPOUD are working on building the complete Kafka as a Service infrastructure – stay tuned for an update on that front.
Interested in event-streaming on the cloud?
If you want to get an even deeper understanding of the project, you can reach out to us. We’re more than happy to elaborate!
Chief Product Officer