Investigating an issue with the Segmentation, Registration, Reporting, Static List, and SMS inbound responses components

Incident
September 17, 9:19am

Investigating an issue with the Segmentation, Registration, Reporting, Static List, and SMS inbound responses components

Status: closed
Start: September 16, 12:56am
End: September 17, 9:19am
Duration: 1 day 8 hours 23 minutes
Affected Components:
Mobile Messaging Web Notifications Email Messaging SMS Messaging Performance Analytics
Investigating

September 16, 12:56am

September 16, 12:56am

Since 00:00 AM Pacific, we have been investigating an issue involving our segmentation component.

The following features have been impacted : 
- Attributes 
- Device Events
- Device Properties 

Message deliveries have also been affected. Customers may notice delays in sending messages.
Messages not including the above features are not affected.  

Our engineering team is working to resolve the issue. 

As soon as we have more information, we will update our status page with additional affected components.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 16, 3:18am

September 16, 3:18am

We are continuing to work on a fix for this issue.

Please, feel free to contact Airship Technical Support at https://support.airship.com, you have any questions or concerns.

Update

September 16, 5:32am

September 16, 5:32am

We are still investigating an issue involving our segmentation component.
Our reporting and static list components are also impacted by the issue.

The following features have been impacted : 
- Attributes 
- Device Events
- Device Properties
- Static lists 
- Message reporting & Performance Analytics

Message deliveries have also been affected. 

Customers may notice delays in sending messages.
Customers may notice delays in loading static lists.
Customers may notice delays in loading message reports and Performance Analytics.   

Our engineering team is actively working to resolve the issue. 

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 16, 9:10am

September 16, 9:10am

We are still investigating an issue that is impacting multiple Airship services. Our investigation has found that the issue also affects device registrations and SMS inbound message processing.

The following features have been impacted:

- Attributes
- Device Events
- Device Properties
- Static lists
- Message reporting & Performance Analytics
- Registrations
- Tags
- SMS inbound responses (Opt-ins/outs)

Message deliveries have also been affected.

- Customers may notice delays in sending messages.
- Customers may notice delays in loading static lists.
- Customers may notice delays in loading message reports and Performance Analytics.
- Customers may notice delays with device registrations and the processing of tags and attributes.
- Customers using SMS may see inbound messages, such as opt-ins, opt-outs, and auto-replies, fail to process correctly.

Our engineering team is actively working to resolve the issue.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 16, 1:20pm

September 16, 1:20pm

Our engineering team have stabilized the impacted services while continuing to work to resolve the issue. We will continue to monitor as services recover and will provide an update by 3pm PT.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 16, 2:59pm

September 16, 2:59pm

The impacted services are stable and slowly beginning to recover. Our team is focused on fully resolving the issue and will post another update by 6 PM PT.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 16, 5:01pm

September 16, 5:01pm

The impacted services remain stable, and increased monitoring and alerting remain in place. Our engineering team is working towards a permanent resolution, and we will provide additional updates as necessary.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Update

September 17, 8:01am

September 17, 8:01am

The impacted services still remain stable.
The delayed backlog keeps on recovering 

We are still monitoring our systems.

Our engineering team is focused on working towards a permanent resolution.
We will provide with an update at or before 12pm PT.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.

Resolved

September 17, 9:19am

September 17, 9:19am

This incident has been resolved.

All the impacted services are performing normally.
The delayed backlog has been fully recovered.

Please feel free to contact Airship Technical Support at https://support.airship.com if you have any questions or concerns.