How Federal Agencies Can Keep Websites Running Smoothly

The storm caused power outages, canceled flights, closed roads and disrupted holiday plans for millions of Americans. But NWS’s website handled the increased traffic like a soft spring breeze thanks to its web infrastructure, which includes on-premises servers, a private cloud and content delivery networks (CDNs).

“We were ready, and we accommodated it,” Jungbluth says.

Federal agencies must regularly provide fast, reliable access to websites to accommodate citizens’ increasing quests for online information and digital services. But large spikes in web traffic can occur at any time due to events, planned or otherwise.

To ensure websites run smoothly when traffic jumps, some agencies have upgraded network infrastructure to increase bandwidth, deployed load balancers to better distribute traffic, subscribed to CDNs to cache and speed delivery of content, and used real-time monitoring systems to manage performance.

Increasingly, however, they have turned to public cloud services to host and manage their web infrastructure, which automatically scales resources as needed, provides redundancy and is more cost-effective, says Zeus Kerravala, founder and principal analyst with ZK Research.

“A government agency’s web presence in the cloud can be built for massive scale, and most enterprise data centers aren’t built that way,” he says.

NWS’s Hybrid Approach to the Cloud

NWS sees huge traffic spikes on Weather.gov during hurricanes and other major weather events. Visitors are primarily the general public, but also include government officials who need local forecasts to make decisions about protecting their communities, Jungbluth says.

Weather.gov consists of nine national weather centers — including the National Hurricane Center and the Ocean Prediction Center — which cater to advanced users and access specific NWS applications for weather forecasts, warnings and other data, he says. That includes aviation forecasts for commercial airlines and marine forecasts for vessels in U.S. coastal waters and on the high seas.

“We have a massive web space that’s grown organically over the years,” Jungbluth says.  

NWS plans to redesign its website in 18 months and migrate web and application hosting fully to the public cloud. Currently the website, applications and data are hosted on on-premises data centers with a half dozen applications running on Amazon Web Services and Microsoft Azure.

LEARN MORE: How agencies can ensure tech purchases integrate with existing environments.

The agency uses legacy server farms in two data centers in Silver Spring, Md., and Kansas City, Mo., to disseminate website content. CDNs cache the content in points-of-presence in multiple regions throughout the country to ensure users get fast access to weather forecasts, Jungbluth says.

“It allows us to balance the load and maintain the security and stability of the system,” he says. 

NWS has also built a private cloud in two data centers in College Park, Md., and Boulder, Colo., that house integrated dissemination program applications, mission-critical systems that include communications, satellite, radar and flood mapping information. The IDP apps serve data into Weather.gov, he says.

The private cloud, built four years ago with Dell hardware and VMware virtualization software, provides NWS with redundancy and backup. When traffic spikes, NWS splits the applications between the two data centers to better balance the load. Two years ago, the agency also upgraded its Cisco routers and purchased load balancers to improve network speeds and performance.

“It’s stable, but it was a long road to get there,” he says, adding that the IT operations team keeps tabs on the IT infrastructure 24/7 using remote monitoring tools.

Keeping NASA’s Website Up As the Sun Goes Down

In 2017, the first total solar eclipse visible in the continental U.S. since 1979 drew a record number of visitors to NASA’s website. The site hosted between 2 million and 3 million simultaneous viewers at its peak, with 56 million total page views that day.

Despite the massive audience checking in on the eclipse and watching the nearly daylong livestream, NASA.gov never crashed — cloud services and CDNs kept it running.

NASA expects high web traffic later this year during a near-total solar eclipse that will cross the Western U.S. and Central and South America on Oct. 14; and again on April 8, 2024, when a total solar eclipse will cross the U.S., Mexico and Canada.

The 2024 eclipse, which will be the last one visible in the U.S. until 2044, covers a wider path than the 2017 eclipse — meaning it will be visible by more people — and the time when the sun is completely blocked by the moon may be twice as long.

NASA officials say they will be prepared, thanks to planning and the space agency’s Amazon Web Services infrastructure. NASA was an early adopter of cloud hosting when a 2003 spike in website visitors interfered with internal network performance, says Internet Services Manager Brian Dunbar.

EXPLORE: Why agencies’ journey to the cloud requires tackling cybersecurity concerns.

The agency takes advantage of AWS’s ability to load balance and automatically scale cloud resources such as compute, storage and bandwidth to meet sudden increases in demand, says Web and Cloud Services Manager Ian Sturken.

“Because we are a fairly high-traffic site, we need a robust architecture, and that’s where the cloud excels,” Sturken says. “The architecture scales on its own. We don’t have to take any additional action.”

Since the 2017 eclipse, NASA has replaced a third-party CDN with Amazon’s CloudFront CDN service, which caches popular web content or retrieves content from Amazon S3 storage to the edge location that’s closest to individual users, so the content is delivered as fast as possible.

“We can achieve 100 percent availability because AWS autoscales appropriate to the traffic, and it’s regionally distributed,” Sturken says. NASA plans to migrate fully to a Software as a Service offering in the future, so the team no longer needs to manage the web architecture, he adds.

In the meantime, the agency has begun meetings this summer to prepare for both the October annular solar eclipse and the April total eclipse, Dunbar says.

NASA’s content and IT team will work with the agency’s TV team and its Science Mission Directorate to discuss potential new features to view the eclipses on the web, including possibly livestreaming the event in 4K rather than HD as it did seven years ago, he says.

How to Keep Sites Running During Loan Repayment Rush

Last October, the Department of Education beta-tested its student aid debt cancellation website to ensure it could handle an influx of users and to troubleshoot any problems that arose.

Student borrowers seeking information had overwhelmed the StudentAid.gov website and caused it to crash two months earlier when Biden announced plans to forgive up to $20,000 in student loans.

Federal Student Aid, an office within the Education Department, built a debt relief application website with a simple online form. Then it quadrupled the bandwidth to its cloud-hosted website and application infrastructure to handle the load of visitors and process the applications, says Education CIO Luis Lopez.

The weekend beta test was a success. FSA invited about 200,000 people to submit their applications early, but the site attracted 2 million who successfully applied for debt relief.

READ MORE: What are three key threats to cloud cost management?

The public cloud was able to handle the surge of users because of its scalability when demand spiked. “It was great. It was a historical peak for us, and the site remained stable,” Lopez says.

Following the beta test, the Education Department formally launched the website, and it worked seamlessly without any major problems. A total of 22 million borrowers submitted applications for debt relief online that first week.

The department stopped taking forgiveness applications on Nov. 11 because of legal challenges to the debt relief plan, and the Supreme Court in late June blocked the plan from going into effect. But despite the legal and political battle over debt relief, Lopez is proud of how well the FSA handled an unprecedented number of visitors to its StudentAid.gov website.

“It really showed we were more than prepared to be a government data collection site,” Lopez says.