Ruminations

Home of the Braves

Somethings stick with you for a long time — and sometimes for the rest of your life. There aren’t many of these life experiences that directly translate to a career. For me there are two that stand out: a class field trip to the Solo Cup factory when I was in the 3rd grade, and visiting Atlanta–Fulton County Stadium for a Braves game almost 30 years ago.

The Solo Cup factory tour was eye-opening. It was my first real introduction to manufacturing, assembly lines, workflow and logistics. It all hummed along in harmony. Everything just worked. The warehouse was so organized; I could immediately see the benefits of the unidirectional flow of materials in and finished good out. It’s not an understatement to say that this field trip established the foundation of how I would see workflow and logistics for the rest of my life.

Going to the Braves game was a concrete reminder of how things can and should work. Riding the subway to the stadium was the first indication that I was going to appreciate the trip more than the game itself (and I loved baseball).

The Atlanta transportation authority (MARTA) were ready for the riders going to the game. Announcements to stay in the end cars and which stop to take. As we were exiting the subway there was someone with a megaphone telling us to get our blue ticket so we could ride the bus to the stadium. And a reminder to only give the bus driver half of the ticket if we intended to take the bus back to the subway after the game.

Once we got outside the buses were waiting for us – all lined up and ready to go. Traffic agents filed baseball fans through crosswalks and onto the buses. When the light changed, and a bus departed, more fans were already being sent through the perpendicular crosswalk to another awaiting bus. Other traffic agents were positioned to get the buses out so the rest of the Atlanta traffic could go about their business. It was like clockwork. It was really impressive.

When we got to Atlanta–Fulton County Stadium the buses took advantage of the stadium’s round design. Buses lined up behind each other, the passengers disembarked, and the buses left (presumably to pick up more baseball fans). It was simple and efficient and effective.

To be honest, I don’t remember who the Braves played that night or who won. But I remember getting to the stadium. And I remember the trip back to my hotel. Some would tell you that it’s very "James" that this was my fondest memory of Atlanta. The display of efficiency, and consideration of the other subway passengers and drivers affected by the buses. It just worked.

The Atlanta I remember is not the Atlanta that’s been revealed in the news recently. City management, once so prepared that it would make a Boy Scout blush, was caught flat-footed. An unwitting example of what not to do.

The March 22nd ransomware attack that hit the city of Atlanta crippled the city’s services. And unlike the fans making their way to the stadium almost 30 years ago, Atlanta was not prepared. Not prepared to respond and not prepared to recover. The city's administration was brought to its knees, and nearly four months later it's still trying to learn to walk again.

Let me be clear — I do not endorse paying the ransom in a ransomware attack. I understand that the cost may seem minimal compared to the potential losses. But whether it’s an individual facing a $300 ransom, or the city of Atlanta facing a $51,000 ransom, my stance is the same: do not pay it.

The city of Atlanta was devastated by this ransomware attack. Not only were they ill-prepared, but their backups were lacking to say the least. What was affected? Many core city services, including the police department, municipal courts, department of corrections, water department, bill payment systems, just to name a few. The Atlanta police department lost a great deal of criminal evidence, including the archive of its dash-cam footage. The interim City Attorney office said they lost a decade of legal documents. A councilman said their office lost 16 years' worth of digital records.

The city had to try to conduct business with mobile phones and pieces of paper. Some employees brought in personal laptops (though they did not connect them to the city’s infected network). Even the city's budget proposal was delayed.

The city of Atlanta has more than 400 software systems. Around one-third of them were knocked offline. 30 percent of the affected systems were considered "mission critical". As of last month, many still have not been fully restored.

On June 11th - 11 weeks after the initial infection - the Atlanta Municipal Court was back online. The Court was one of the last city services to recover from the attack. The Court, which handles thousands of traffic tickets and court cases each day, had to start dealing with the backlog generated by the outage.

As of June 20th - three months after the initial infection - the Atlanta police said they had "mostly recovered" from the attack.

So far, this attack has cost Atlanta more than $12 million. And not all of the city's services or systems have been restored. And they have lost years and years' worth of records ... and even criminal evidence. Once the investigations are complete, and the full impact and cost of this infection are known, we will use this as an example of how not to be prepared. At least until the next major municipality or provider of services is infected. When that happens, we will compare it to the Atlanta incident.

Atlanta is not the only municipal victim of infected systems this year, though they have been the most severely impacted. Savanah, GA; Allentown, PA; and Colorado’s DOT (twice) have suffered at the hands of system-wide infections and service outages.

So, the next time you have to deal with your local municipality, and all of its unpleasant bureaucracies, give it a friendly toast. Let it know that the only thing worse than having to endure its eccentric inefficiencies is not being able to. I have a Solo cup ready and waiting, which is more than I can say for Atlanta.

---