Application Native Infrastructure - 123dev #94

Posted on October 18, 2022  •  4 minutes  • 757 words

Comments

Application Native

The traditional box drawn around an “application” has been small. Developers don’t need to learn the other stuff like networking, storage, load balancing is a lie we’ve told ourselves for a long time—especially with traditional, on-prem infrastructure. The dividing line between teams was the boundary where applications ended and ops began.

Ops was everything developers didn’t want to do. In reality, the application always included ops just as much as it included application code. Customers don’t care if you’re on-prem with Cisco switches and NetApp storage or in AWS with VPCs and s3. It’s all part of the application.

Cloud Native came along and made infrastructure API driven. It made it possible for applications to take ownership and declare the requirements of their code, but it was not easy to get right. Infrastructure was still a separate thing managed by a different team with unknown skills.

Now infrastructure is regularly managed by application teams whether they realize it or not. The Lambda function with API Gateway and Kubernetes deployment with LoadBalancer service are both managing infrastructure. Things like policy as code, service mesh, and mounted volumes bring security, networking, and storage into application management. It abstracts them into smaller, sane defaults the application code can use without requiring application developers to understand how they’re implemented.

This is a good thing! Application teams can move faster and become customers of the services provided by the other components. Flexibility is available on a per application basis instead of per account or data center.

What goes around comes around and for people familiar with Heroku this is what many people call a platform, but application native is more than that. It’s more flexible than platform guard rails at the cost of learning the additional configuration.

Application developers don’t want to learn infrastructure, but when infrastructure speaks their language, hides the global complexity, and allows them to coordinate less with external teams the benefits are huge.

People

I met a ton of amazing people from all over the world last week in Dubai at Gitex. People from places I’ve never been and with backgrounds I know nothing about. The amount they’ve overcome, the skills they acquired, and the things they’ve accomplished left me humbled.

Every conversation opened my mind to the amount privilege I have for being born who I am, where I was, when I was. I’ve had my own challenges, but they seemed miniscule in comparison.

The people in Ukraine are overcoming more than I can imagine right now. They’ve been fighting to keep their freedom while others around the world fight for their freedom for the first time.

I’ve never been more thankful for all of the freedoms and privilege I’ve been given and hope I can use them to give others more.

Links

My talk last week was all about how to manage infrastructure with control loops—what I call “Infrastructure as Software.” In my demo I used this example of a Crossplane package to condense the complexity of an EKS cluster into ~15 lines of configuration. Worth a read if you’re on a platform engineering team or manage infrastructure from your application.

Production-Ready EKS Cluster With Crossplane Crossplane allows you to manage cloud resources with the Kubernetes API using the kubectl.

Charity Majors has seen a lot of stuff in her career and I generally agree with her observations about what’s happening with “ops” jobs. I like to describe the shift as “Application Native Ops” because more of the traditional ops tasks are shifting into the application layer. Scheduling, network, security, availability are all things that are handled by application teams via a platform or service (e.g. Lambda, Kubernetes, fly.io). The platforms have different default settings and some have more knobs than others, but it’s almost always the application that sets the desired configuration.

The Future of Ops Jobs | A Cloud Guru — acloudguru.com The role of operations engineers is changing fast, and the role is bifurcating along the question of infrastructure.

Since this issue is infrastructure focused I’ll add this great breakdown of what the backend of Netflix looks like at a high level. I worked at Disney+ for a while and one of my favorite things was trying to piece together all the components that users never see—there’s a lot.

A Design Analysis of Cloud-based Microservices Architecture at Netflix | by Cao Duc Nguyen | The Startup | Medium — medium.com A comprehensive system design analysis of the cloud-based microservices architecture implemented at Netflix to power its global video streaming services