How The Washington Post Built Arc Publishing using Amazon ECS

Arc publishing is a digital tool that helps the content teams to power their business through innovation, better customer experience and storytelling with an integrated approach. It is a suite of engineered tools to meet the needs of modern publishers.

In this video Patrick Cullen, a principal architect provides an executive overview of how Washington post built a new software service business by ECS.

What is the Washington Post?

It is one major news publisher with more than 95 million monthly visitors and 1 billion page views.

At 0:22 Patrick states that Over the last few years Washington post showed tremendous growth and the reason behind this growth is equal focus on excellence in journalism and technology innovation.

What is Arc Publishing?

From 0:32 to 0:56, he explains that because Technology innovation is such an important part in this story, they wanted the same technology and the powers of Washington post to build a business out of it. So they launched a business called Arc publishing.

It is fast and efficient Content Management System (CMS) for modern publishers, speed and innovation for readers, reasoning and development are core goals of publishing.

Migration from old to new Architecture

At 0:57 he tells that many years ago Washington post had an architecture very similar to other major news publishers. They had a Commercial model of CMS and all content was stored in one application in a database app. The problem with this architecture is that it makes innovation difficult because it reduces the flexibility to experiment with new ideas. They gradually migrated to new architecture where each content type is stored in its own micro service.

So In this example, they have a Video story and Photo Service that are all separate to have their own applications in data base stacks.

Benefits of New Architecture

From 1:34 he describes about the benefits of new architecture. The benefits of this architecture is it allows them to independently develop and deploy each of these. This architecture increased the rate of experimentation leading to higher quality product. When they enlarge their publishing, they wanted to bring innovative architecture to our customers because their micro service architecture was designed just for a single tenant to watch imposed.

Major Challenges

At 1:58 he describes that there was no need of supports from multiple tenants, so it is a real problem for them. This meant that each arc publishing customer will need own copy of each micro service.

Although they considered re architecting all of their services to be multi tenants, they realized that would significantly delay launch of their publishing. So instead they decided to clone the environments for each of our publishing customer. This allows them to quickly relate with real users and to launch the business but it makes scaling the business very challenging.

How to overcome Scaling Challenges?

From 2:28 he demonstrates that to put these scaling challenges into contacts, they needed a platform that supported more than 50 deployments in an hour across 100 unique micro services and in their infrastructure, it translated over 150 ECS cluster instances in many thousands of docker containers. Adding to their challenges they had to devote a culture which means there was no dedicated operation team to manage this infrastructure and unlike traditional software companies they had limited engineered resources to focus on the scaling challenges.

From 3:01 he describes that to solve the scaling challenges they built a container platform powered by ECS that they called Nile. Now to start container as unit deployment and sketch those container across multiple ECS Clusters, Nile made easy to port for existing technology to docker containers that can run on ECS. Nile also keeps track of tenant containers so that they can quickly update a particular micro service across the entire environment and keeps the rate of experimentation high.

The Nile platform was also designed for liability because their publishing customers are some of the top media companies in the world with very demanding uptime performance requirements.

Lessons Learned

In conclusion at 3:36 Patrick describes the key benefits of their approach as Building Nile and ECS allowed them to take their limited engineering resources and focus those on building new customer features instead of man engineered infrastructure and they did this without dedicated operation team to manage their environment.