SUMMARY
On March 4, 2025 from 17:15 UTC to 18:15 UTC, customers using our Zendesk Support Help Center (support.zendesk.com) experienced “temporarily unavailable” errors and failure to get results when searching.
TIMELINE
March 04, 2025 06:19 PM UTC | March 04, 2025 10:19 AM PT
Between 17:58 and 18:15 UTC March 4, 2025, Zendesk Support Help Center (support.zendesk.com) experienced an issue where searches returned "temporarily unavailable" and no results. The issue has been addressed and search should function as expected at this time.
POST-MORTEM
Root Cause Analysis
This incident was caused by an error while updating the support.zendesk.com IP allow list from an outdated source list. Valid IPs were inadvertently removed from the allow list, leading to connection failures for integrations that relied on those IPs.
Resolution
To resolve the issue, the team promptly updated the IP allow list with the correct values, restoring access for all valid integrations. The outdated IP list was flagged as DNU (Do Not Use) and removed to prevent future errors.
Remediation Items
- Delete legacy IP allow list reference to eliminate confusion and prevent future errors.
- Review and enhance monitoring and alerting systems to detect similar issues more quickly.
- Implement a more stringent validation process for changes to the IP allow list to reduce the risk of human error.
- Consider developing an API/UI enhancement to improve the user interface for IP allowlisting, making it easier to manage and track changes.
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. You can follow this article to be notified when our post-mortem report is published. If you have additional questions about this incident, contact Zendesk customer support.
0 comments
Article is closed for comments.