A common question developers bring up when wanting to develop serverless and cloud native applications is: what will the developer experience be like?
It is an important question as a good developer experience with a quick feedback loop results in happier and more productive developers who are able to ship features rapidly.

Since we’re building Plain to be intentionally small, an outstanding developer experience is a must.
We need to make sure that the few engineers that we do hire can make the most impact by delivering product features quickly while maintaining a high quality.

We had the opportunity to think about how to solve this problem in 2021 as Plain is built from scratch.
When we were deciding our general tech stack, the experience of making changes on a daily basis played a large role in the decision-making, as well as how we’ll be able to build a successful business in the next 5 to 10 years without needing to do foundational replatforms.
This meant being able to run our services at scale at a low cost without needing to have a whole department just looking after a homegrown infrastructure.

The rationale behind these decisions definitely requires its own separate post, but we ended up deciding to go all-in on serverless and cloud native, full-stack TypeScript, and using AWS as our cloud provider, due to its maturity and popularity. We decided that using proprietary AWS services is an acceptable lock-in as there’s high value gained compared to the likelihood of switching cloud providers.
I’ve definitely seen companies spend huge amounts of effort trying to be cloud-agnostic, without actually realizing any tangible benefit from it.

What’s unique about serverless development

There are some unique aspects to developing and testing serverless applications.
One of the main differences is that you end up using a lot of cloud services and aim to offload as much responsibility to serverless solutions as possible.

In the case of AWS Lambda this means that you typically end up using API Gateway, DynamoDB, SQS, SNS, S3, EventBridge, ElastiCache, etc. to build your application.
Using so many services involves a lot of configuration, permissions, and infrastructure that needs to be developed, tested, and deployed.
If you only focus on testing your lambda code then you’re skipping a large part of your feature.
Examples that you might encounter if you don’t verify your infrastructure:

• missing an S3 trigger to SQS or a Lambda function
• missing an EventBridge rule to route events to the right targets
• missing a Lambda IAM role update to use a new AWS service
• incorrect CORS or authorizer configuration in API Gateway

One of the most important questions to answer is: when do you want to find out about these mistakes?

1. While writing and running your tests?
2. While working on a feature and the developer manually trying out their feature?
3. In your Continuous Integration run via some E2E integration test suite?
4. In a shared deployed environment, such as dev or staging?
5. Or in the worst case scenario: in production?

We decided to do it as soon as possible: while writing and running tests.
What this means is that the debate of “should you mock cloud dependencies or embrace the cloud” was not really a question.
Having our Lambdas use AWS mocks or some localhost emulation still leaves a lot to be desired in terms of “will it actually work” when deployed.
Gareth McCumskey’s Why local development for serverless is an anti-pattern blog post captures the “emulate vs. use the cloud” debate quite well, and I’d highly recommend reading it.

The largest implication of developing against the cloud is the need for internet access to effectively write code. While this might be an unacceptable trade-off to some companies or people, for us as a remote-first company, we require internet access to communicate with our colleagues therefore there would be very few times when people didn’t have network connectivity.

With the general principle that we want to be developing against the cloud and not trying to build a local developer experience, we set out evaluating various tools and technologies to find out what works for us.

The 🌈 magical 🌈 stack

So what does our magical AWS serverless developer experience look like? At a high level the following make up the key components:

• Every developer has their own personal AWS account
• AWS CDK to develop our infrastructure and Serverless Stack (SST) to get a very quick feedback loop
• Writing significantly more integration tests than unit tests
• Full-stack TypeScript

Adopting these technologies and practices yields a pretty brilliant developer experience.

Personal AWS accounts

When going full serverless, each developer having their own personal sandbox AWS account is a must.
As previously mentioned, to build most features it’s not enough to write the code, there’s a lot of infrastructure that needs to be developed, changed, and tested.
Having personal AWS accounts allows each developer to experiment and develop without impacting any other engineer or a shared environment like development or staging.
Combined with our strong infrastructure as code use, this allows everyone to have their clone of the production environment.

You might be thinking: isn’t that expensive?
Won’t we be paying hundreds of dollars to AWS? Nope—not with serverless solutions!
The genuinely serverless solutions are all pay per usage, so if your AWS account has zero activity, for example through the night and weekend when engineers aren’t working, then you won’t pay a dime.
There are a few exceptions to this, such as S3 storage, DynamoDB storage, RDS storage, Route53 hosted zone, etc. costs, but they tend to be minimal.

For example, Plain’s January bill for our 7 developer accounts was a total of $150—pennies compared to the developer velocity we gain by everyone having their clone of production.
Typically, the largest cost for each developer is our relational database: Amazon Aurora Serverless v1 PostgreSQL.
It automatically scales up when it receives requests during development and down to zero after 30 minutes of inactivity.