VPN Outage - authentication and Confluence issues

Incident Report for Uhub

Resolved

This incident is now resolved. We are awaiting a root cause analysis from out IT provider for the P1 outage.
Posted Sep 25, 2019 - 17:06 UTC

Update

The underlying issue has been resolved and we are now reverting from our workaround solution (which has just started failing due to another issue). During the revert process, some user access may disappear for a short period of time. We are monitoring as we go.
Posted Sep 25, 2019 - 14:57 UTC

Update

We are continuing to monitor for any further issues.
Posted Sep 25, 2019 - 14:56 UTC

Monitoring

A temporary workaround has been implemented and we are monitoring the platform.
Posted Sep 24, 2019 - 22:44 UTC

Identified

Our IT Provider has begun investigating the VPN outage. In the meantime we have enabled a backup authentication service. We expect that authentication services should now be somewhat stable but possibly slow for a little while.
Posted Sep 24, 2019 - 22:32 UTC

Update

Jira is now also unstable due to the VPN failure. A P1 has been raised with our IT provider who manage the VPN.
Posted Sep 24, 2019 - 21:51 UTC

Investigating

One of the VPN connections that Uhub relies on for a portion of the user base is currently down. Some users will not be able to authenticate. We are noticing some instability in Confluence as a result of the outage and working to minify impact.
Posted Sep 24, 2019 - 21:15 UTC
This incident affected: Uhub Authentication Services, Uhub Confluence, and Uhub Jira and Service Management.