Resolved -
The backlog of extractions has been processed and the issue has been resolved.
Feb 9, 08:56 UTC
Monitoring -
The systems are back to normal and have been processing for the last couple of hours. We restored the data processing pipelines in a phased manner starting 8:30 AM UTC and all stages were restored by 10:30 AM UTC. The extractions are being processed as usual. We will continue to monitor the applications to ensure that the backlog is cleared in the next 24-36 hours.
We identified the root cause was a data storage limitation in one of our backend tables that required an urgent fix. To safely perform this correction, data processing was temporarily paused and migration was performed.
Feb 5, 12:39 UTC
Update -
We are continuing to investigate this issue.
Feb 4, 13:37 UTC
Investigating -
We are observing slowness in completing extraction processing in the backend servers and as a result the backlog is increasing. This is due to a DB issue and it may cause some random slowness in certain UI pages like Analyse ACE/Maintainability .We are currently investigating the issue.
Feb 4, 13:36 UTC