Community Platform Issues

[Drew_C]

Hello everyone,
As you have surely noticed, our community has been impacted by an issue our platform provider has been working on since yesterday morning. I posted about this yesterday but the announcement has been lost (presumably because of database corruption on their side).

I wanted to share the announcement sent to us a few minutes ago by GetSatisfaction (our community host).

Dear Valued Clients,

We are currently experiencing a few issues in the platform that may be disruptive to your experience. Our Engineers are aware of these issues, and are working to correct them as soon as possible. Below are the issues we are working to address:

1. While attempting to access some articles on Community, you may see a 404 Page Not Found Errors. Most of these errors were addressed yesterday, but we are seeing a few outlier cases. We are working to resolve these.

2. When attempting to pull up the Analytics page, many of the reports are coming up blank or with no value.

3. Many users have been receiving emails from other GetSatisfaction Communities for which they are not part of, some of which are private communities.

In order to address these issues, there will be some data loss on yesterday's posts. We apologize for the inconvenience!

Thank you for your patience while our Engineering team continues to work to resolve these issues. It is the team's top priority.

We will update you as soon as we have more information regarding the fixes for these issues. We appreciate your patience.

Thank you,
GetSat Support Team

[Erik_Auerswald]

Hi,

just for information, at least one post of me was lost during those incidents. 😞

Br,
Erik

[Drew_C]

Another update:

Posts between 4:45AM and 7AM ET, and then again starting *around 10:30AM to 5:30PM ET were impacted by the events yesterday. In order to address the database issues the engineers were seeing, the database servers had to be moved, which removed the posts created during these timeframes.

New users created during these times would also be lost, and need to be re-created. I've asked the engineers to see if they can identify clients impacted with lost users.

The specific start point of the second incident is not known, but it is close to 10:30AM ET.

[Drew_C]

Latest update from ~20 minutes ago:

Dear Valued Customers,

Our Engineering Team has informed us that the 404 errors, and issues with cross Community emails, have been resolved.

We apologize for this inconvenience, and we've asked the team to assess the root cause in order to prevent this type of disruption going forward.

Unfortunately, due to dependencies in the system, there will be data loss seen with these corrections, as posts that went up yesterday will be lost and unretrievable.

Please let us know if you experience any issues related to these incidents, or if you see anything else out of the ordinary.

Regards,
GetSat Support