AWS re:Invent 2020: Adrian Cockcroft’s architecture trends and topics for 2021

In this video, Mr.Adrian Cockroft talks about architecture trends for the year 2021. At 0.26, the speaker starts by briefing about Serverless-first. The concept of serverless is very much correlated with microservices.

A microservices is a loosely coupled service-oriented architecture with bounded contexts. People generally opt for microservice architecture when speed and agility are to be achieved.

At 2:43, the speakers list the common concerns revolving around choosing serverless as the first option. Some of the common concerns include portability, scalability, language support, state handling, security, run duration, storage interfacing, event processing, complex configuration, startup, and network latency.

Serverless architecture and Chaos Engineering

At 3:22, Mr. Adrian briefs the latest updates pertaining to the serverless architecture. Originally, one of the limitations with lambda was the 512 megabytes that could be bundled with the function. With this limitation, it was difficult to deploy deep-learning models as a function.

Now, with the latest update, you can use Amazon EFS to Amazon NFS to mount a file system that contains the deep-learning models. This will provide you terabytes per file and up to 25,000 concurrent connections. The other update is the support provided for the common build pipelines. With this update, you can now deliver both serverless and container workloads.

At 4:49, he emphasizes that serverless is the fastest way to build an application

At 5:28, Mr. Adrin talks about the concept of ‘Chaos engineering and failing over without falling over. He defines chaos engineering as an experiment to ensure that the impact of failures is mitigated. Continuous resilience is the ultimate target. Redundancy is built into the system so that there is an alternative in case of failures.

At 7:39, he illustrates the 4 layered chaos engineering architecture. The bottom-most layer is the infrastructure. There must be a way to switch between alternatives. Application and trained people also constitute chaos architecture.

The usefulness of Wardley Maps

At 15:46, the speaker points out how to failover without falling over. Some of the ways include reducing timeouts to drop orphaned requests, routing calls within the same zone, the flood of alerts that need to be reduced to actionable insights, and running regular chaos engineering experiments. In addition, having a high level of automation, symmetries, consistent configuration as a code, consistent instant types, services, versions, zones will help to failover without falling over.

At 17:46, the speaker talks about Wardley Maps. Early adopters use Wardley Maps frequently. Wardley maps allow defining position and movement. In addition, these maps help in defining strategic context, barriers, and actions.

Wardley Maps are generally used to interpret the open-source product strategy, the evolution of technology from custom to utility and make vs buy decisions. At 21:08, the speaker points out some of the results from successful mapping. You get successful products and services due to situational awareness, strategic design making, shared understanding of moves, and application of doctrines.

At 21:42, Mr. Adrin briefs that Big data is data that’s too big for machines to process on one machine. Some ways to take advantage of more memory include in-memory analytics (eg: SAP), very large graph databases, using shared memory to avoid serialization overhead, and using the memory interconnect as a low latency network.

‘Amazon Sustainability Data Initiative’ (ASDI) and ‘OF-Climate’ initiatives

At 26:53, Mr.Adrin talks about how Amazon functions on the environment front. Amazon has committed to power operations with 100% renewable energy by 2030. He also adds that Amazon is on track to achieve this feat 5 years early in the year 2025. Moving to AWS (Amazon Web Services) has helped in reducing carbon to a great extent. There has been a reduction of about 88% in carbon footprint compared to running workloads on the premises.

At 27:22, Mr. Adrin talks about the famous ‘Amazon Sustainability Data Initiative (ASDI)’. This initiative focuses on reducing the cost, time, and technical barriers that are generally associated with analyzing large datasets in order to generate sustainability insights. With this initiative, easy access is provided to petabytes of foundational data. It’s hosted free of charge in public Amazon S3 buckets. It is managed and updated by the data owners.

At 28:14, Mr. Adrin talks about the Linux foundation organization known as the ‘OS-Climate’. This ‘OS-Climate’ Linux foundation is an open-source platform to enable access to critical data and models to support climate risk assessment. Some of the contributions made by AWS towards the OS-Climate foundation include the extension of the ASDI initiative to freely host OS-Climate specific datasets and providing professional services and credits.

It is recommended for organizations to use AWS services and open data programs to architect solutions and achieve their sustainability goals. In this video, Mr.Adrin Cockroft elaborates the prominent architecture trends for the year 2021 with examples.