IOT down

Major incident Access Control IOT
2024-11-07 07:30 CET · 1 hour, 20 minutes

Updates

Post-mortem

Dear Customers,

This morning, one of the master servers in our cluster where IOT is running experienced a high CPU load, which caused the cluster to become unresponsive. We were notified of this incident through our monitoring and began investigating as soon as possible.

We soon noticed that we would be unable to recover it promptly, so we decided to fail over IOT to our other data center, which already had the infrastructure ready but required some manual steps to get it up and running.

We are pressing our partner, who enables the hosting of this cluster, to identify the issue that caused the high CPU load and to understand why the secondary master node did not take over.

Rest assured, we are taking all necessary steps to ensure this will not happen again. We apologize for any inconvenience this may have caused you.

Kind regards,
Team Vintia

November 7, 2024 · 12:59 CET
Resolved

Dear Customers,

The issue have been solved and we will try to provide a post mortem as soon as possible.

Kind regards,
Team Vintia

November 7, 2024 · 09:44 CET
Monitoring

Dear customers,

The problem has been identified and resolved, we are monitoring the status and will keep you posted once we determined the underlying cause of this issue.

Our apologies for any inconvenience this may have caused.

Kind regards,
Willem

November 7, 2024 · 09:41 CET
Update

We are currently in the process of restoring services.
New update will be provided when all services are up & running.

November 7, 2024 · 09:12 CET
Issue

Dear Customers,

We are reaching out to inform you that IOT is currently down. We are investigating the root cause and we’ll keep you posted as soon there is an update.

Kind regards,
Team Vintia

November 7, 2024 · 08:25 CET

← Back