Squarespace Status / Sandy

We have set up a status page at http://status.squarespace.com/ outside of our infrastructure. Please refer to that page for status updates related to Sandy. As of this update, we are hanging on. However, the situation is not stable and we are not yet in the clear. Thank you all for your patience.

Hurricane Sandy Update

Anthony here. Just wanted to share a personal update now that I have a moment.  

At around 9:00 this morning upon hearing about the fuel situation at Peer1, I decided to head out and see if I could lend a hand. The streets of Manhattan near where I live (Soho – not in the evacuation zones) are in not-so-bad shape right now, but the damage left from the flooding in the evacuation zones is significant and real.

I’m sitting in our datacenter NOC at 75 Broad St. Not that it’s been pointed out to me, but there are beds set up on the tiled floor here from the great team at Peer1 who stayed to monitor the situation overnight. These guys have incredible commitment to keeping everything running, and it’s great to see.

Normally, power loss would not be a major problem for our datacenter – Peer1 stayed online during the major Manhattan power outage in 2003 that lasted for days, and we preemptively shifted to backup power around 4:00pm yesterday predicting that Con Edison would be shutting off power in evacuation zones.  Given the nature of the flooding, this situation escalated greatly, submerging our reserve fuel in the basement, shutting off the elevators, and damaging the pumps required to get this fuel to the generator on the 17th floor.

My reasons for coming to the datacenter were twofold: one was simply to help and do whatever I could to help us (and the whole building) stay online. The other was to send our systems team the absolute final signal to perform a clean shutdown of our infrastructure should we be moments away from total power loss. Generally, clean shutdowns are preferable to abrupt halts, since code halting in a known state is better than code halting in an unknown state.

We had an initial warning that 10:45am was going to be the clean shutdown time. To determine how much time we have remaining, engineers are taking readings at particular time intervals to attempt to determine how quickly we are depleting. The tank readings are behaving somewhat erratically, as there is another mechanism replenishing it from a separate fuel header. Some of our recent readings seem more optimistic, but it is impossible to predict how much fuel remains in this header at this time. As of this writing, we have at least 45 minutes left.

Bridges to the island are open right now, and we currently have a fuel truck en route. We have approval from the building to manually carry fuel up in plastic water bottles, and we have a number of our team on-site to carry fuel up the stairs as needed. I do not know if the manual plan will be successful, but we will certainly try.

Unfortunately, I do not have more information on a final resolution to this issue. You should still expect Squarespace to go offline at some point because of the hurricane’s aftermath, but we will do our best to keep that downtime to a minimum. Once we have a reliable stream of fuel to the building, it will go online independently of any other grid issues related to ConEdison and lower Manhattan in general.

We’ll continue to keep you posted. Thank you for your patience.

A

Downtime Expected Due to Sandy

Everyone, I have some unfortunate news to share.

Our primary data center, Peer1, in Lower Manhattan lost power yesterday at about 4:30PM local time. At that time, we smoothly made the transition to generator power and took comfort over the fact that we had enough fuel to last three to four days. (Peer1 stayed online during the last 3 major natural disasters in the area, including a blackout that lasted for days.)

At 8:30PM yesterday, we received reports that the lobby in the data center’s building was beginning to take on water. By 10:30PM, as is sadly the case in most of Lower Manhattan, Peer1’s basement had experienced serious flooding. At 5AM, we learned our data center’s fuel pumps and fuel tanks were completely flooded and unable to deliver any more fuel. At 8AM, they reported that the generators would be able to run for a maximum of four more hours.

Unfortunately, this means that Squarespace will be offline soon (our estimate being at 12PM today). Be assured that while this will impact our availability, there is no chance of data loss or any other permanent effects. We have simply run out of power, backup power, and cannot access our fuel in a flooded basement.

Our teams have been working tirelessly on contingency plans. We are working to bring the Squarespace systems back online as soon as possible. As you have probably read, all bridges and tunnels into and out of Manhattan are closed and large portions of the city remain without power. We will do everything in our power to get Squarespace running as soon as possible, and we will remain online for as long as it is safe.

Squarespace support will remain available 24/7 during this downtime. We will post updates to Twitter via @squarespacehelp and urge you to follow us there.

Our hearts go out to the many people who have lost their lives in this terrible tragedy and also to those who continue to suffer through the consequences of this historic storm.

Thank you for your understanding.

Introducing Squarespace Note

We all have hundreds of fleeting thoughts throughout the day, and the ability to capture and coalesce those ideas is critical to many of our workflows. Yet there’s no easy way to snapshot thoughts as we have them – and that's why we created Squarespace Note.

Squarespace Note is a clean, minimalist, note-taking application that can push your thoughts to email, Evernote, Dropbox, Facebook, Twitter, and, yes, Squarespace. Note is unique from most existing note applications in that it’s not used as a database, but a passthrough. It becomes the first place you go to push text to various services.

The Note app is free and does not require a Squarespace account. Note is already an integral part of our day-to-day workflow here at Squarespace – many of us save thoughts to email and Squarespace a few times a day using Note.  We think it makes us more effective thinkers, doers, and publishers.

We hope you enjoy it as much as we do.

[Please note: We have multiple teams dedicated to iOS application development, and we are continuing to develop updates for the core Squarespace application suite, which is a much larger task.  We will share those updates as soon as we can.]

Squarespace at PhotoPlus

The Squarespace team will be joining over 24,000 professional photographers and enthusiasts in New York City this week for PhotoPlus. If you plan on being at the Javits Center from October 25th to 27th, please visit us! 

Our exhibit at booth #973 will be hard to miss (floor plan). Here’s what you can look forward to if you stop by to meet our staff:

• Demos and Questions – New to Squarespace or want to learn more? Our team will be giving live demos of Squarespace 6 all day long and can answer any questions you may have.
• iOS Portfolio Sneak Peek – Be the first to see our new iOS app, designed especially for photographers.
• Free Setup – Bring a thumb drive of your photos and we’ll get you started with a brand new portfolio site in under 20 minutes. 
• Giveaways – We’ve got t-shirts, field journals, GorillaPods, and more!
  

If you plan on attending PhotoPlus, tweet us @Squarespace with hashtag #photoplusexpo. We look forward to meeting you in person later this week.