[INCIDENT] Elevated API Errors in MDA (Backed Services)

  • 21 October 2016
  • 9 replies
  • 49 views

Badge
SUMMARY



We're experiencing an elevated level of API errors. Initial analysis confirms DNS related issues with our Upstream service provider and they are investigating the same. 



FOR MORE INFORMATION



Please subscribe to this article for regular updates until the issue is resolved. If you have questions about this issue, please open a ticket with us by sending a note to support@gainsight.com.



UPDATE



Friday, October 21 9:20AM CST: Our service provider has resolved the DNS issue. During the outage MDA services including Rules, Outreaches and Data Processor were paused. We've restarted all services and are replaying any jobs that failed during the service incident. 

9 replies

Badge
Our Upstream providers have identified the root cause of the issue causing DNS Resolution errors and are currently working to resolve.
Badge +1
Our service provider has resolved the DNS issue. During the outage MDA services including Rules, Outreaches and Data Processor were paused. We've restarted all services and are replaying any jobs that failed during the service incident. 
A customer rec'd a denial of service error message while attempting to complete a NPS survey yesterday at 2:20pm is this related to the API/MDA DNS issue?
Badge +1
Arnold - I am not sure if this is related. I know you're working with our support team to investigate and they'll be able to help. 
Badge
UPDATE



We are experiencing this issue once again, and seeing errors "Unexpected end of JSON error" in Data Management, Copilot, Connectors tabs.  Will update this thread with the latest information as it comes in.

Userlevel 7
Badge +3
Thanks Mandy...that explains why I was experiencing this:  https://community.gainsight.com/gainsight/topics/timeouts-and-delays-in-gainsight-application



VERY unnerving!
Badge
Might have something to do with this

http://www.wsj.com/articles/denial-of-service-web-attack-affects-amazon-twitter-others-1477056080

Cyberattack Knocks Out Access to Websites
Userlevel 7
Badge +3
Thanks for the info Kristin!
Badge
Resolved - All services and queues are processing normally. 



Please reach out to support@gainsight.com if you are still experiencing any issues.

Reply