How to do event-streaming on the cloud at La Mobilière

1000

topics

days

Hybrid Cloud

Environment

Zero-trust

Security model

Kafka Confluent Cloud Terraform Microsoft Azure Cloud Monitoring CI/CD

Situation & Goal:

La Mobilière is in the middle of their cloud transformation journey. The entire IT infrastructure will be moved to the cloud. This is a huge undertaking and event-streaming is also a central part. Event-streaming is especially important because it provides data connectivity and is the connection between the legacy and the cloud world. From a business perspective, La Mobilière wants to become quicker and more flexible by moving to the cloud. With Kafka as a Service, they are decoupling their data streams in time and space, enabling them to integrate and deploy applications faster and with more flexibility, as well as analyze and evaluate data efficiently.

“With the help of SPOUD we were able to quickly evaluate the different options for event-streaming on the cloud. Thanks to their experience, we could assess, test and build a PoC in only three months. The PoC is now the foundation for the bigger scoped project.”

Tirone Gigandet, Team Lead, La Mobilière

Their current on-prem Kafka Connect architecture introduced by SPOUD is used extensively. For this reason, La Mobilière was faced with the question, “How can we do event-streaming and messaging on the cloud?” There are numerous solutions for this but finding the best one for the desired outcomes and needs is hard and requires a great deal of experience with event-streaming to properly evaluate. To solve the challenge with event streaming in the cloud, SPOUD came into play. The goal was to answer the question “How can La Mobilière do event-streaming in the cloud and, more specifically, can we cover all the needs using Confluent Cloud (SaaS)?” The question was answered by evaluating the current architecture, defining the technical requirements, and building a PoC around the assumptions.

About La Mobilière:

La Mobilière aims to offer its customers optimal and holistic advice on insurance solutions. For this reason, they are pioneers in digitalization. It was founded in Berne in 1826 and has been headquartered there ever since.

Challenges:

La Mobilière has an on-prem Kafka. This instance needs to be replaced by Kafka as a Service to properly run in the cloud. Before everything can be moved to the cloud, there’s a transition period, where some data-streams are on-prem, while others are in the cloud. During the transition, the infrastructure on the cloud must still be able to exchange data with applications on-prem (“backward-compatibility”) as well as with new applications in the cloud (“forward-compatibility”). This hybrid cloud approach ensures that during the transition phase, the two worlds are still able to exchange data where needed.

La Mobilière’s sophisticated cloud strategy also presents a few key technical challenges. Especially since the event-streaming infrastructure will operate on the cloud, governance policies and zero-trust security models must be considered when evaluating a possible implementation. Security demands on the cloud and on-premises are quite different, adding an extra layer of complexity. So, the main technical challenges to be considered, are:

Enterprise governance rules that require certain safety, legal and organizational constraints to be fulfilled
Integration of roles and security patterns
Integrating zero-trust environments on-premises and on the cloud
Automation (infrastructure as code) with e. g. HashiCorp’s Terraform
Integrating monitoring with tools on the Azure Cloud
Self-service provisioning of service accounts, topics and acls
Replication of data and schemas from on-prem
Replication of data from the cloud to be accessed on-prem

But the question remained: How can this even be done? The engineers at La Mobilière and at SPOUD evaluated the situation and came to the conclusion that Confluent Cloud could solve most of their needs. To test their hypothesis, they set up a PoC.

1000

topics

3

Months

Hybrid Cloud

Environment

Zero-trust

Security model

Automation

Of processes

Solution & Benefit:

SPOUD consulted the team at La Mobilière with their knowledge about Kafka, Kafka as a Service and Confluent Cloud. By setting up the PoC, SPOUD illustrated where there was room for improvement regarding security and interoperability. SPOUD built the basis to estimate the effort to build Kafka as a Service and thus laid the foundation for the project.

Our goal was to kickstart the Kafka cloud migration at La Mobilière. In 3 months, we evaluated the best possible solution to do event-streaming on the cloud and set up a PoC to test our evaluation.

In detail, we:

In detail, we did:

Set up an architecture for zero-trust hybrid Confluent Kafka as a Service Cloud
Built a PoC cluster, to prove cloud integration possibilities – using the latest and greatest APIs
Integrated Kafka with new Confluent Governance
Automated setup of Kafka with an “Infrastructure as Code” approach
Integrated Confluent Cloud monitoring API with the existing observability system on the Azure Cloud

Right now, la Mobilière and SPOUD are working on building the complete Kafka as a Service infrastructure – stay tuned for an update on that front.

Interested in event-streaming on the cloud?

If you want to get an even deeper understanding of the project, you can reach out to us. We’re more than happy to elaborate!

Simon Hofer

Chief Product Officer