2.0 Syncs Running Behind
Incident Report for Context.IO
Resolved
The syncs cleared overnight and are back to operating normally.
Posted 3 months ago. Aug 12, 2017 - 09:20 CDT
Identified
We've temporarily disabled the message counts (`nb_messages`) for the `2.0/accounts` endpoints to help with response times. We'll enable those again once the sync queue is cleared.
Posted 3 months ago. Aug 11, 2017 - 19:23 CDT
Monitoring
The backlog of messages continues to drain at an increasing pace. We estimate the queue will be flushed this evening. We'll resolve this incident as soon as the syncs have caught up completely.
Posted 3 months ago. Aug 11, 2017 - 16:33 CDT
Update
The backlog of syncs is putting a higher load than usual on our systems, which is causing some requests to 2.0 to time out. This should start to improve as the sync backlog clears.
Posted 3 months ago. Aug 11, 2017 - 13:02 CDT
Identified
We have identified the cause of the sync queue and have deployed a fix. The backlog of syncs is now going down.
Posted 3 months ago. Aug 11, 2017 - 12:32 CDT
Update
The engineers are working to isolate the root cause of the sync delays. We will provide an ETA for the resolution once we have full confirmation on the cause and identify the best solution.
Posted 3 months ago. Aug 11, 2017 - 11:27 CDT
Investigating
We're investigating a spike in our 2.0 sync queue which is also affecting the performance of 2.0 webhooks.
Posted 3 months ago. Aug 11, 2017 - 08:47 CDT