Noticing an issue where a few collections have lost entries
Incident Report for Quintype
Postmortem

Problem
A DB update script for updating status for few collections was run, which unranked the stories belonging to those collections

Resolution

  1. The ranked order was picked from the DB backup taken just before the execution of the update script
  2. The rank order and the related collection metadata was restored from the backup

Measures to prevent this in future

  1. We will be prioritizing, building versions for collections and the ability to revert to a previous version of collection
  2. We believe this issue could have been caught with more rigorous testing on a UAT environment.
  3. We will also be investigating adding alerts on empty home pages
Posted Jul 11, 2019 - 21:00 IST

Resolved
This incident was resolved. The team has manually updated a few collections where were broken. This issue affected only 2 publishers on the system.
Posted Jul 11, 2019 - 14:26 IST
Identified
We are currently looking at an issue that caused a few collections to lose the entries in the collection. We are restoring this data from an earlier backup.
Posted Jul 11, 2019 - 12:05 IST
This incident affected: API.