A massive tech outage grounded flights and kneecapped hospitals, banks and first responders around the world early Friday, though San Francisco and the rest of the Bay Area seemed to avoid the worst of the disruptions.
The IT meltdown was attributed to an outage on Microsoft cloud platform Azure and a flawed software update by cybersecurity firm CrowdStrike that caused millions of computers worldwide to crash.
The fallout was historic in scope. Many businesses reverted to cash payments, emergency dispatchers in some U.S. cities had to take notes by hand, and entire court systems were knocked offline. Travelers were stranded far from home and got stuck in long lines at major airports worldwide — including in the Bay Area.
SFO reported 87 cancellations and 212 delays as of 11:30 a.m., according to the website FlightAware. The San Jose International Airport reported 13 cancellations and 54 delays. Oakland International Airport recorded eight cancellations, as well as 52 delays.
The Federal Aviation Administration said it was working with airlines to manage the disruptions, which involved 3,835 flight cancellations worldwide and 2,378 into or out of the U.S. as of 11:30 a.m., according to FlightAware. Total delays by late morning reached 33,805, with 7,161 into or out of the U.S.
SFO systems were functioning early Friday, according to airport spokesperson Doug Yakel, who said it seems the airlines themselves bore the brunt of the outage.
But he confirmed that the airport did have to reschedule a groundbreaking event on a $2.6 billion upgrade and seismic retrofit to Terminal 3, which was supposed to take place Friday morning and was expected to include local dignitaries, including San Francisco Mayor London Breed and House Speaker Emerita Nancy Pelosi.
Major U.S. airlines granted waivers for travelers who had to rebook flights because of the outage. Under federal regulations announced this year, passengers qualify for automatic refunds on flights that are significantly delayed or canceled — although the U.S. Department of Transportation is still finalizing details about how to enforce the new standard.
At SFO, suitcases piled up by the hundreds in the baggage claim area as travelers scrambled to check updates about their flights or arrange overnight stays at nearby hotels.
Liz Carone, 32, was at SFO for nearly 12 hours when her United flight to Baltimore was canceled three hours after she boarded the plane. At 10 a.m. Friday, an exhausted Carone was waiting for her baggage, holding her 1-year-old on her hip while her 5-year-old sat on the floor nearby.
“It could’ve been worse,” she said. “There were other kids having meltdowns, but ours were fine.”
Her new flight was rescheduled for 5 a.m. Saturday.
Carone’s husband, Nick, said he had to contact American Express to get a refund on the flight because the line at United’s kiosk was far too long and he was having trouble with the app. He said he even had trouble booking a hotel room because many places said their computer systems were down.
Matt Cooper, a 38-year-old from Ohio, was supposed to stop briefly at SFO on his way to Manila. But at 12:45 a.m. Friday, his United flight was canceled about an hour after it was set to depart, and the airline didn’t retrieve his baggage from the plane for nine hours, attributing the delay to the computer outage.
The new flight won’t take off until around 1 p.m., but Cooper seemed unfazed.
“It happens,” he said.
Hotels around SFO were inundated with bookings by travelers whose flights were rescheduled or canceled.
An employee at the Millwood, a boutique hotel in Millbrae, said its computer system crashed just as it began dealing with the uptick in demand.
“We cannot check in, we cannot check out,” a frustrated hotel clerk told The Standard, adding, “I cannot even email, and I cannot even print.”
The nearby SFO El Rancho was on pace to sell out, according to an employee who attributed the surge to the travel delays.
The Residence Inn San Francisco Airport/San Mateo was full by 11 a.m.
“We have 164 rooms, and they’re all booked,” a clerk confirmed, adding that a bunch of families were waiting in the lobby.
Kevin Sullivan, director of sales at SFO’s Grand Hyatt, said things had calmed down since the hotel saw a surge of panicked travelers late Thursday.
“It was a little bit hectic overnight when everything first went down,” he said.
Sullivan said his staff passed out bottled water to travelers biding their time in the lobby while awaiting newly scheduled flights.
In Burlingame, the town of 30,000 where SFO is based, downtown shops and restaurants braced for a rush of patrons.
“I’m walking up and down talking to restaurant owners, telling them to be ready,” said John Kevranian, president of the Broadway Burlingame Business Improvement District and owner of a local shop called Nuts for Candy & Toys. “I hope people are safe, and you know, that the systems get back online soon. But when there are cancellations and delays, that brings more people to downtown, eating and drinking and using wifi.”
Among the stranded travelers making the most of their time on the Peninsula was a group of coworkers whose connecting flight to a work conference in Australia was grounded SFO.
St. Louis resident Lauren Vernaci, 33, and Kansas City residents Chelsey Ast, 29, and Peter Cajina, 33, said they were boarding their plane late Thursday night when the outage struck.
“They were manually checking our boarding passes and passports,” Ast said, “and then while we were sitting on the plane, the plane rebooted. Everything turned off and turned back on.”
She said she was just grateful the plane wasn’t in the air when it happened.
“We would have been over the ocean the entire time,” Ast said. “And we were thinking of the planes that had already taken off and what happened to them if they were affected by this outage.”
While chaos unfolded on the ground, Cajina said airline representatives told them that planes were stuck in the air, circling overhead until they got clearance to land.
After that, the airline ushered passengers off the plane. Because luggage was held up by the system failure, Ast, Cajina and Vernaci had to wait overnight at the baggage claim.
With flights still delayed the next morning, their company booked the trio a few rooms at the San Francisco Airport Marriott, which was still trying to recover from the same IT outage. After waiting several more hours, they finally got to check in around 11 a.m., shower, change and head out for a bite to eat at Ristorante Rocca in downtown Burlingame.
“We’ve been up for 24 hours,” Ast lamented after their meal.
Instead of going to Australia, she said they’ll head back home on the next available flights, which means spending at least one more night on the San Francisco Peninsula.
Meanwhile, in San Francisco proper, where City Hall doesn’t rely on CrowdStrike, local government services emerged from one of the world’s largest IT outages relatively unscathed.
“While the majority of the city and county of San Francisco IT systems do not utilize this software and therefore were not impacted, some issues were reported and are currently being assessed,” Angela Yip, a spokesperson for the City Administrator’s Office, told The Standard.
The software glitch “most likely” caused some 20 false fire alarms to set off around 2:30 a.m. throughout the city, according to a spokesperson for the San Francisco Fire Department. The Standard was unable to reach the San Francisco Police Department for comment.
The city’s 911 system remained functional throughout the night, according to San Francisco Department of Emergency Management spokesperson Denny Machuca-Grebe. “We saw no major delays or impacts,” he told The Standard by email.
Gallery of 6 photos
Expand photoClick arrows to view
the slideshow
SFO terminals were crowded Friday amid widespread flight cancellations and delays. | Source: Tâm Vũ/The Standard
Arrival and departure status on a monitor at SFO. | Source: Tâm Vũ/The Standard
Passengers go through security at SFO amid a technical outage that grounded flights around the world Friday. | Source: Tâm Vũ/The Standard
Passengers make their way through the terminal at SFO. | Source: Tâm Vũ/The Standard
SFO systems were functioning, an airport spokesman said. | Source: Tâm Vũ/The Standard
Passengers wait by the United Airlines check-in counter. | Source: Tayfun Coskun/Anadolu/Getty Images
So far, the biggest impact seems to be on the airlines, Yip said. Of course, the extent of the fallout remains to be seen.
“The city is working with internal partners and external vendors to further identify and address any residual issues to IT systems,” she added.
San Francisco buses and Muni lines were rolling along as usual, according to MTA spokesperson Erica Kato. A spokesperson for BART said the regional transit service was also “operating as normal.”
The same held true in Oakland, where city officials said emergency services seem unaffected by the outage.
California’s Office of Emergency Services said on X that “all 911, public safety communications and critical infrastructure is functioning as expected.”
A spokesperson for CrowdStrike told The Standard that the firm is “actively working with customers impacted by a defect found in a single content update for Windows hosts.” She said the outage was not caused by a security breach or cyberattack.
“Our customers remain fully protected,” spokesperson Kirsten Speas said in a follow-up email. “We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption. We are working with all impacted customers to ensure that systems are back up and they can deliver the services their customers are counting on.”
The problem, which she said has been fixed, involved a Falcon update for Windows Hosts. She directed users to the company’s blog for the latest updates.
Microsoft 365 posted on X that it was “rerouting the impacted traffic to alternate systems to alleviate impact” and “observing a positive trend in service availability.”
In an email to The Standard Friday afternoon, a Microsoft spokesperson said “a CrowdStrike update was responsible for bringing down a number of IT systems globally” and that Microsoft is “actively supporting customers to assist in their recovery.”
The outage was enough for some people to stop doing business with CrowdStrike — including tech titan Elon Musk.
Though the tech outage roiled companies and entire industries, the U.S. stock markets opened Friday without a hitch and Microsoft shares barely dipped. But CrowdStrike stocks took a double-digit tumble while competitors, like Palo Alto Networks, saw their shares rally.
As companies and agencies scrambled to bring their systems back online, security experts cautioned that hackers have been trying to take advantage of the chaos. Several reports emerged in the wake of the outage about phishing attempts and sites trying to sell bogus patches to CrowdStrike and Microsoft bugs.
The U.S. Cybersecurity and Infrastructure Security Agency issued an advisory Friday urging impacted organizations to “remain vigilant,” to “only follow instructions from legitimate sources” and to “avoid clicking on phishing emails or suspicious links.”
Copyright for syndicated content belongs to the linked Source link
