Today’s lesson was all about Serverless architectures. Serverless is a method where you simply write your code and let Amazon Web Services (AWS) handle all the maintenance, scaling, and heavy lifting automatically. With serverless you can focus purely on creating amazing features and not managing infrastructure. You don’t have to manage complicated servers and pay for expensive computing power that just sits idle. You only pay when your code is actually running.

Serverless computing can be compared to a vending machine. You just pay for the soda (code execution) right when you need it instead of buying a whole store (a dedicated server) and hiring staff. AWS handles the machine maintenance and stocking.

I delved into three core services namely, Lambda, API Gateway, and EventBridge that require zero server management.

I found answers to 3 key questions:

1. Where does my code actually run? (AWS Lambda)

2. How do customers access my application? (API Gateway)

3. How do different parts of my application talk to each other without breaking? (EventBridge)

1. Where does my code actually run? (AWS Lambda)

AWS Lambda is like a professional chef working on call. The chef is not paid to stand around waiting. The chef only shows up and cooks when someone places an order (the trigger). Likewise, Lambda executes your function code when it is triggered, meaning you don’t have to manage any dedicated computers (EC2 instances or containers).

When configuring Lambda, key parameters include Memory, which proportionally affects CPU and network performance, and Timeout, which can be up to 15 minutes.

But there is a trick to increase performance – Provisioned Concurrency. This is akin to keeping the chef on standby. Since the chef might take a minute to grab ingredients (a “cold start”) Provisioned Concurrency means you pay a bit extra to keep the chef ready at the stove so the meal starts instantly. It keeps the function “warm,” reducing the delay caused by initialization (cold starts).

There are other optimization tips like keeping dependencies small and tuning memory settings. Lambda supports various runtimes like Node.js, Python, and Java, as well as custom runtimes. Like a trustworthy employee, each Lambda function is given only the specific permissions it needs to do its job, following the principle of least privilege.

2. How do customers access my application? (API Gateway)

API Gateway is the official entrance for all incoming requests to an application. It manages all the people coming in and directs their orders to the right Lambda chef.  It allows you to build standard REST APIs, faster HTTP APIs, or WebSocket APIs for real-time communication. It helps manage crowds by using Throttling and Quotas to control traffic bursts, so the system isn’t overwhelmed. It can use caching for faster responses  to reduce latency and decrease the number of times a Lambda function needs to be invoked.

It is comparable to The maître d’ at a restaurant. This service stands at the front entrance, managing the traffic, checking IDs (security), setting limits on how many people can enter (throttling), and directing guests (requests) to the right kitchen (Lambda function).

3. How do different parts of my application talk to each other without breaking? (EventBridge)

EventBridge is an invisible communication system. Think of the postal service or mail sorter. When one service (like an Order Service) finishes a job, it drops a letter (the event) into the system. EventBridge reads the address (the rule) and routes it efficiently to any other service (the target, like a Payment Service) that needs to know about it. This makes services independent of each other.

If Lambda is the engine that runs your code and AP Gateway is the front door of the application then Eventbridge can be called The Decoupling Glue. EventBridge creates decoupled microservices, that is, if the Shipping Department breaks, the rest of your system (like the Order Entry system) can keep running.Instead of Service A sending a message directly to Service B (creating a tight connection), Service A sends a message to the Event Bus. EventBridge uses Rules to automatically route that message to Service B (the Target).

If the target service is temporarily unavailable, EventBridge will retry the delivery for up to 24 hours to prevent data loss.

To sum it all up, the three services I studied today are said to form the backbone of modern, event-driven, cost-effective serverless architectures. By linking these tools—API Gateway (front door) to Lambda (task runner) to a database like DynamoDB (storage), and using EventBridge (communicator) it becomes possible to build a complete, powerful system without worrying about server maintenance.

In today’s digital world, customers won’t wait for slow pages, and businesses can’t afford downtime or data loss. The bar of expected speed, uptime, and reliability are at an all-time high.

To that end AWS provides powerful, fully managed tools to help you move data safely, serve users globally, and recover instantly when things go wrong.

This lesson aims at breaking down how AWS makes database migration and global replication simple.

Here are four key questions (and answers) that explain how AWS helps organizations handle data movement, migration, and global access without the headaches.

1. How do we move an existing database to the cloud without interrupting the business?

We can use the “digital moving truck”, AWS Database Migration Service (DMS). How does it work? DMS first copies everything from your existing database (a “Full Load”).

Then it switches to Change Data Capture (CDC) to constantly track new transactions, sign-ups, or orders as they happen. The CDC mode allows the old database to keep running while the new one is built. DMS simply copies every new change and transaction in real-time until you’re ready to switch over.The real-time replication ensures that when you switch to the cloud version, all your data is up to date and downtime is close to zero. It is simple, smooth, and reliable. Think of the moving truck taking the furniture to your new house and then watching the old house and sending over any new mail or package deliveries in near real-time.

DMS  can move data between homogeneous databases ie. the exact same type of database (like moving from one type of MySQL to another) or between heterogeneous databases ie. different types (like moving from an Oracle database to a PostgreSQL database)

2. What if our new cloud database speaks a different “language”?

The AWS Schema Conversion Tool (SCT) handles the translation for heterogeneous migration.SCT converts the structure, code, functions, and rules (the schema) from one type of database to another during a heterogeneous migration. It also gives you a report card on how hard the translation will be. To illustrate, if you move from a foreign country (like Oracle) to a new one (like Aurora PostgreSQL), you don’t just move your furniture; you need to translate your legal documents and instructions.

So, the 2 main tools we need for moving the house (migration) are AWS Database Migration Service (DMS), The Data Mover and AWS Schema Conversion Tool (SCT) The Language Translator. Together, they make cloud migration seamless.

3. How do we make our database fast for users all over the world?

We can use Aurora Global Database, the global relational powerhouse. It is best for relational data (like account records or complex sales transactions). The idea is : One primary region handles all new writes and up to five secondary regions hold live read-only copies. These replicas are updated with sub-second latency, so users in Europe, Asia, or America get lightning-fast responses.

If our main region fails, a secondary one can be promoted in under a minute (RTO < 1 minute), thus keeping global operations alive.

4. What if users need to write data from any region, anytime?

We can use DynamoDB Global Tables. It is best used for high-speed, simple data (like user profiles or gaming scores). How does it work? You set up servers in multiple regions, and all of them can accept new transactions simultaneously (active-active). In other words, every region can handle both reads and writes. Each change made anywhere is instantly copied everywhere.DynamoDB automatically keeps the most recent one, the “last writer wins” rule which is the perfect setup for real-time, global applications like gaming, finance, or user profiles where every millisecond counts.

Expressed differently, If two regions try to change the same record at the exact same moment, the system resolves it automatically by accepting the change that arrived last and there is no manual failover needed for disaster recovery; it’s built-in since all regions can handle writes.

The bottomline is that mastering database migration and global design on AWS is about knowing which tool fits your need. Need to move existing databases with minimal downtime? Use AWS DMS. Need to translate database code and structure? Use AWS SCT Need Global relational performance? Use Aurora Global Database. Need Global NoSQL with multi-region writes? Use DynamoDB Global Tables. Thanks to these tools an organization can move data smoothly and build highly resilient, super-fast applications accessible to anyone, anywhere in the world.

The right choice of digital storage service determines your speed, your ability to share data, and ultimately, your monthly bill so it is a very critical decision. Amazon Web Services (AWS) offers several primary ways to store data and today’s lesson examines how to pick the right one

Here are four key questions that can guide us in selecting and optimizing AWS storage:

Q1: What are the three main types of storage, and when do I use each one?

There are three primary services, differentiated by the kind of data access they offer:

1. EBS (Elastic Block Store): It is like a personal hard drive. It is best for the computer’s operating system (boot volumes) and crucial, constantly changing data, like customer databases. Only one virtual computer can use it at a time

2. EFS (Elastic File System): This is comparable to a shared network folder. It is best for sharing data among many virtual computers simultaneously. It grows and shrinks automatically as you add files. EFS is described as being fully managed and elastic, meaning it handles all the growing and shrinking on its own.

Because it is a regional service, the data is spread across multiple locations (AZs) for high availability.

3. FSx Family: FSx is like a specialized enterprise server. It provides tailored file systems for specific enterprise tasks.It is best for handling complex business needs, like running specific Windows applications, Active Directory integration, or doing heavy-duty data crunching (HPC).

It  offers specialized, fully managed File systems (like SMB or Lustre protocols).

For Windows: FSx for Windows File Server works perfectly if your company uses Microsoft’s protocols (SMB) and Active Directory. It supports high availability across multiple locations (Multi-AZ).

For Supercomputers: FSx for Lustre is built for extreme speed, great for things like machine learning and big data analytics. It can even automatically sync data with S3 storage

Q2: How do these services handle growth and sharing?

The key differences are in scaling and shared access:

EBS generally allows only a single instance to access the volume – athough there is a Multi-Attach feature for io1/io2 volumes in the same AZ. A manual process is required to scale.

EFS is designed for multiple instances and uses automatic, elastic scaling so it can grow and shrink automatically as you add files. It is a regional service so it is not limited to one Availability Zone (AZ).

FSx also supports multiple instances but uses managed scaling.

Q3: How can I save money on my cloud storage?

You can implement lifecycle management and choose cost-effective volume types: gp3 and EFS Infrequent Access.

For EBS, the recommendation is to always choose the gp3 volume type over the older gp2. The gp3 volume provides custom IOPS and throughput, and, for the same performance, using it can be about 20% cheaper.

EFS Lifecycle Management is the biggest cost-saver. You can enable policies to automatically move files that haven’t been touched after a certain number of days (say, 30 days) into the Infrequent Access (IA) storage class. This move can cut your storage costs for that cold data by up to 92%.

General Optimization: Automate EBS snapshot creation using the Data Lifecycle Manager (DLM) and regularly delete unused snapshots.

Q4: Which storage options are best for the absolute highest performance?

To optimize performance, faster drive types like io2 can be used.

For extreme speed for specialized or critical workloads, there are specific options:

For Databases, use the io2 or io2 Block Express types, which are designed for high-performance, latency-critical workloads and support up to 256,000 IOPS.

For Supercomputing, use FSx for Lustre. This file system is purpose-built for high-performance computing (HPC), machine learning (ML), and analytics. It can integrate directly with S3 buckets and automatically sync changes.

To conclude, EBS is block storage and is best used for Databases, OS, Boot Volumes. EFS is file storage and is best used for shared application data. FSx is file storage and is best used for Enterprise Workloads, High Performance Computing (HPC).

How do major online services stay fast even when millions of people log in at once? How do they survive a major regional power outage without skipping a beat? Today’s lesson examines two foundational architectural principles that make it possible, namely, Auto Scaling and High Availability

The key questions I examined today were:

What are the fundamental components of Auto Scaling that enable automatic capacity management?

Which scaling policy is best suited for handling predictable, recurring spikes in user traffic?

How does the system ensure fault tolerance and automatically replace instances that fail?

What is the required architectural pattern for designing highly available applications?

Question 1: What are the fundamental components of Auto Scaling that enable automatic capacity management?

Auto Scaling is the process of automatically adjusting server capacity by adding or removing EC2 instances based on demand. It is like having a smart manager for your fleet of servers who automatically adds servers when you get busy and removes servers to save money when traffic slows down.

It has three core components:

1. The Launch Template. It defines how new instances are configured (e.g., AMI, type, security groups). The recommendation is to  use Launch Templates for modern designs because they support versioning and multiple instance types.

2. The Auto Scaling Group (ASG). This is the logical unit managing the group of EC2 instances. Key ASG parameters include a min size(the smallest number of servers always running), a max size (the absolute capacity limit) and desired capacity (the target number of instances)

3. Scaling Policies. They are the rules which dictate exactly when and how capacity should change.

Question 2: Which scaling policy is best suited for handling predictable, recurring spikes in user traffic?

There are four main scaling policy types; three are critical to remember

1. Predictive Scaling: This uses machine learning to forecast demand so it is ideal for predictable, recurring traffic spikes, such as 9-to-5 office hours or nightly batch jobs. The system scales out capacity before the traffic actually arrives.

2. Target Tracking Scaling is the simplest and most common policy. It is like setting a thermostat. You set the goal eg. keep the server’s CPU utilization exactly at 50%. If the temperature (CPU) goes up, the system automatically adds capacity until the goal is met.

3. Step Scaling: This scales incrementally based on specific threshold breaches.E.g., add 1 instance if CPU hits 60%; add 2 instances if it hits 80%.

Question 3: How does the system ensure fault tolerance and automatically replace instances that fail?

Fault tolerance relies heavily on Health Checks. Health Checks determine if an instance is operational. The ASG is constantly checking the health of servers. If a server fails its check – freezes up or stops responding – the ASG does not try to fix it. It immediately fires that instance and spins up a brand new, healthy replacement automatically.

While the default is the EC2 status check, it is advanced practice to combine this with a check through the load balancer (ELB check) to ensure it’s serving traffic correctly (end-to-end resilience)

Lifecycle Hooks are like a pause button that  allow you to pause an instance briefly during transition to ensure everything is handled gracefully. When a new server starts up or an old one shuts down, it might need a minute for running configuration scripts, attaching necessary monitoring agents, or gracefully deregistering the instance before shutdown. These hooks pause the instance during this phase.

Question 4: What is the required architectural pattern for designing highly available applications?

To ensure service never goes down, redundancy is key. Never put all your eggs in one basket, as the expression goes.

Multi-AZ Architecture: This means you don’t put all your servers in one data center building (Availability Zone). If there is a power outage in that building, the whole application goes down.

Using a Multi-AZ pattern involves spreading servers across two separate geographical areas so that if one fails, the other keeps running, which is required for high availability.This pattern usually uses an Application Load Balancer (ALB) combined with an ASG spanning the AZs to ensure fault-tolerant load balancing and redundancy.

Multi-Region Active-Passive: In this design, a primary region handles traffic, while a secondary region waits while maintaining minimal standby resources, using Route 53 Failover Routing for switchover.

Multi-Region Active-Active: This global strategy requires that both regions serve traffic simultaneously, requiring global data replication and often using Route 53 Latency Routing to send users to the closest region

To sum up, here are a few highlights from the lesson:

The Auto Scaling Group (ASG) manages both automatic EC2 scaling and automated replacement of unhealthy instances

Use Launch Templates over Launch Configurations for modern designs

Choose Predictive Scaling for managing forecasted or cyclical demand.

For high availability, use a Multi-AZ architecture

See you tomorrow!

Today’s lesson revolved around how to set up a network in a way that is scalable, secure, and highly available (HA). These were the big questions I delved into:

1. How do Security Groups (SGs) and NACLs differ in function?

2. Which type of Endpoint should I choose for private AWS connectivity?

3. What is the scalable solution for connecting many VPCs?

4. How should NAT Gateways be deployed for High Availability (HA)?

First, a brief overview of key concepts. Your VPC is your entire network setup in the cloud. We might compare a Virtual Private Cloud (VPC) to a large building or campus. If your VPC is your own private property , you would no doubt organise them into rooms; subnets are the specialized rooms within that building. Continuing that analogy, the public Subnet would be the front door since it has a direct route to the street, which is the Internet Gateway (IGW). Any resource here needs a public address to be accessible from the internet. We might call the Private Subnet the back office- this room has no direct route to the street, or IGW. If resources here need to communicate with the internet (e.g., for updates), they must use a dedicated, shared payphone, or NAT Gateway located in a public subnet. An isolated subnet is the vault – a room that is completely locked down and has no route to the street (IGW) or payphone (the NAT Gateway). It is ideal for highly sensitive data like internal databases and backups .Every room has a Route Table – a map that tells the traffic leaving that room where to go.

1. How do Security Groups (SGs) and NACLs differ in function?

We have two main tools or security layers to protect our resources- SGs and NACLs.

A Security Group (SG) is a bodyguard for a single resource like an EC2 instance or ENI. SGs are stateful: If the bodyguard allows traffic in based on an inbound rule, it automatically remembers and allows the return traffic. They only use allow rules.

A NACL (Network Access Control List) is a fence around the entire subnet. NACLs are stateless: you must explicitly write rules for traffic in and traffic out. The NACL (Fence) is checked before the Security Group (Bodyguard). NACLs are best used for coarse perimeter filtering, especially to implement deny rules (e.g., blocking known malicious IP ranges).

Remember: whereas SGs are applied per resource (instance, ENI), NACLs are applied per subnet.

2. Which type of Endpoint should I choose for private AWS connectivity?

If your private resources need to use AWS services without traversing the internet or NAT Gateway, you use Endpoints. The type depends on the service:

When accessing S3 or DynamoDB, use a Gateway Endpoint . This is a free, dedicated route defined in your route table via a prefix list.

When accessing private service APIs (like KMS, SSM, or third-party SaaS services), use an Interface Endpoint (PrivateLink) . This works by creating a dedicated network card (ENI) inside your subnet.

3. What is the scalable solution for connecting many VPCs?

VPC Peering is like a secret tunnel joining two houses. It is a simple, low-latency, point-to-point connection which is great for connecting two neighboring VPCs. The downside is that it does not scale well. If you have 10 VPCs, you need dozens of connections.  Peering is non-transitive—traffic cannot pass through one peer to reach a third VPC.

Transit Gateway (TGW), on the other hand, is the scalable, centralized hub-and-spoke solution. All VPCs attach to the TGW, thus allowing traffic to flow between any of them. Use TGW for enterprise scale involving many VPCs and connections back to your on-premises network.

4. How should NAT Gateways be deployed for High Availability (HA)?

Remember the payphone analogy? NAT Gateway, the payphone is AZ-specific. So If you only install one payphone in one specific geographic area (Availability Zone), two things happen:

1. If that area fails, all your private rooms in all other areas lose their outbound calling ability.

2. All traffic crossing to use that single payphone costs you extra money.

The solution? For robust design, you need a payphone in every area where you have private rooms. In other words, in each Availability Zone where you have private subnets  you must deploy a separate NAT Gateway to ensure high availability and avoid cross-AZ data transfer costs.

These were my big takeaways from today’s lesson:

1. SGs are stateful and applied at the instance level while NACLs are stateless and checked first at the subnet level, allowing you to use deny rules for blocking traffic.
2. Use the free Gateway Endpoint for S3 and DynamoDB, and use the Interface Endpoint for all other services and APIs.
3. Use Transit Gateway for enterprise scenarios involving many VPCs (hub-and-spoke). VPC peering is non-transitive.
4. For High Availabilty (HA)  in production, deploy one NAT gateway per AZ. Do not assume a NAT Gateway is multi-AZ

See you tomorrow!