Yotpo Service Disruption notice
Incident Report for Yotpo
Resolved
The Yotpo service disruption has been fully resolved and the following services have been restored:

- Widget performance is back to normal.
- Q&A section on-site and in the Yotpo admin has been re-enabled.
- Caching and Content Delivery method reverted back to original state for all accounts.
- Widgets pagination has been re-enabled.
- Widgets payload no longer returns a 500 Internal Server Error.
- Facebook Reviews Tab.
- Reviews and order creation back to normal (no data loss).

We apologize for the inconvenience caused and greatly appreciate your continued patience during this investigation.
Posted Aug 24, 2017 - 01:23 IDT
Update
Yotpo on-site Widgets disruption is still intermittently on going, Widgets might return a 500 Internal Server Error upon calling their payload.

In order minimize the previously reported on-site performance issues (slowness & time-outs), we have:
1. Temporarily disabled the Q&A section on-site and in the Yotpo admin.
2. Temporarily disabled The Reviews Widget pagination option (only first page will be displayed).

To summarize, the following Yotpo services are still impacted:
1. Widgets payload might return a 500 Internal Server Error.
2. Facebook Reviews Tab returning a 500 error page.
3. Orders / reviews creation and updates may experience delays but loss is not expected.

We apologize for the inconvenience and will provide further updates as soon as possible.
Posted Aug 23, 2017 - 22:08 IDT
Update
Yotpo on-site widgets disruption is still on going, and you may encounter long loading times or cases where widgets fail to load.

In order to minimize on-site performance issues, we will temporary set the Yotpo widget performance setting to "Better Performance" for all accounts, which will load the widget using CDN and caching.

Due to this change, modifications made in the Yotpo Admin, such as un-publishing a review or changing the look and feel will take up to an hour to be reflected on your site.

To summarize, the following Yotpo services are still impacted:
1. Slowness / timeouts in Yotpo's on-site widgets and Widget section in Yotpo admin.
2. Facebook Reviews Tab returning a 500 error page
3. Orders / reviews creation and updates may experience delays but loss is not expected.

We apologize for the inconvenience and will continue to provide updates regularly.

The Yotpo Team
Posted Aug 23, 2017 - 16:42 IDT
Update
UPDATE:

Yotpo Admin and Live Moderation are back up. Yotpo's Reviews.me URLs are also accessible

Yotpo On-site widgets disruption has aggravated, and you may encounter long loading times or case where widgets fail to load.

To summarize, the following Yotpo services are still impacted:
1. Slowness / timeouts in Yotpo's on-site widgets
2. Facebook Reviews Tab returning a 500 error page
3. Orders / reviews creation and updates may experience delays but loss is not expected.

We Apologize for the inconvenience and appreciate your patience while we work to restore full service.

The Yotpo Team
Posted Aug 23, 2017 - 13:43 IDT
Identified
UPDATE:

Root cause identified in our caching mechanism . The Yotpo Ops team is working on mitigation and getting services back to a fully functional state.

At the moment the following Yotpo services are impacted:
1. Inability to access Yotpo Admin and Live Moderation
2. Slowness in Yotpo's on-site widgets
3. Facebook Reviews Tab returning a 500 error page
4. Reviews.me URLs inaccessible
5. Orders / reviews creation and updates may experience delays but loss is not expected.

More to come in the next Status Update.

The Yotpo Team
Posted Aug 23, 2017 - 11:28 IDT
Investigating
We are investigating an interruption in our API service, causing slowness in Yotpo's on-site widgets, as well as issues logging into the Yotpo Admin and Live Moderation Interface.

We apologize for the temporary inconvenience. More details to follow in the next status update

The Yotpo Support Team
Posted Aug 23, 2017 - 10:43 IDT