Your Lead Architect Doesn’t Really Understand Microservices
A few months ago, I was asked to review an infrastructure diagram for an application that would use a “microservices” architecture.
“I only see one database labeled here,” I said. “What are the datastores for all of the separate microservices?”
“Oh, that makes things too difficult — it’s much better with only one database,” the lead architect replied. “But we have an API layer microservice that connects all the other microservices to it.”
“What happens if the database goes down? Don’t the microservices go down? And if you need to upgrade the database server, doesn’t that impact the whole system?”
“Of course. That’s how you have to do it.”
The lead architect here had gone to several conferences that featured speakers on microservices architectures, and had read many blog posts on the topic. But fundamentally, he had no clue what the fundamental “raison d’être” of a microservices architecture is.
Along the same lines, I frequently run into agile development shops that always run two-week sprints, no matter what the project (so much for people over processes, eh?); lean startups that “plan” to run out of seed money shortly after the first time a customer gets to interact with the product; and cloud architectures that rely upon manual software updates and building AWS AMIs to “save configurations.”
This isn’t just garden-variety ignorance. These are all cases where someone has taken the time to learn something new, and brand what they are doing as that thing, while completely failing to understand what that thing is. Our current culture of scanning blog posts and trying to consume information via tweets has made us all much more ignorant than we think we are. And so, in large part, we are much worse off than if we learned nothing new at all. After all, it would be much better to not know what lean startup was, so you could learn about it with an open mind, than to think incorrectly that you knew it already.
The overall issue here is this: we have a significant problem in IT of tech professionals building things that they think are best practices (e.g., cloud, microservices), but they really aren’t. This is because those professionals don’t properly understand what the best practices actually are. And that is for three main reasons:
1. Lack of Continuing Education
There is no good continuing education in IT. We have conferences, but most of them are vendor-run, vendor-censored, and vendor-revenue-increasing. Try to learn about the cloud by only going to Oracle OpenWorld, VMworld, and Microsoft conferences! It was almost five years after AWS released, in 2013, that Microsoft and Oracle embraced an AWS-like vision of the cloud, and VMware has gradually added more and more cloudy features, but with an incredibly high focus on lock-in (and high prices).
2. Non-Practitioner Journalists
The only way to consistently learn about new technologies in IT is through tech journalism, which is sadly almost never conducted by practitioners. Generally speaking, tech journalism is just professional journalists pitched by PR agencies and talking to C-level executives — CMOs, far too frequently. You can see how starved people are for actual real practitioner reviews of technology by looking at how many personal blog posts get circulated around when someone actually spends four to five hours testing some new technology (for example, the recent evaluation of AWS API Gateway by John Titus and Hiram Software).
3. Technology Innovators Uninterested in Helping Newbies
Finally, the practitioners who do much of the great work in coming up with new methods and new technology just aren’t interested in writing newbie how-to guides on those. That’s not any knock on them, per se, but it does mean that as an industry, we don’t have an effective way of disseminating new and innovative methods except if it benefits a vendor financially. (This is probably one of the reasons why we have so much shitty PHP code — those were the easiest tutorials out there!)
So let me take a stab at giving a few high-level guidelines so you can figure out whether you’re actually doing the following things properly:
What Does a Proper Cloud Application Architecture/Deployment Look Like?
- All data is stored in data stores, and no data is stored on application servers. Logs may be stored on application servers temporarily, but are shipped to a central data store on a regular basis.
- Application servers are in an array and set to autoscale; you should never know or care how many application servers are running — except there should be at least two running in separate data centers with separate power and separate Internet connections.
- Data stores should either be cloud-managed (e.g., RDS, Google SQL); datastores as a service (e.g., S3, DynamoDB, Firebase); or backends as a service (e.g., Parse). These make failover, backups, and restores possible with high availability.
- All repeated server tasks — especially launching servers and failover — should be scripted, preferably automatically in response to various events.
What Does a Proper Microservices Software Architecture Look Like?
- Each microservice has a lean connection to every other microservice, usually through a RESTful API.
- Microservice boundaries are drawn around organizational capabilities — perhaps around particular development teams.
- Each microservice is deployed and updated independently of other microservices.
How Do You Know if You’re Doing it Correctly?
I don’t expect to see any significant improvements in continuing education, conferences, or helping the uninformed, so I don’t expect this problem to go away. However, I do think there is one strategy you can employ to help your own understanding of new trends: follow the leader. It’s almost certain that some reasonably well-known company has done what you’re thinking about doing, and for a particular strategic (meaning top-line, revenue-related) reason. Make sure that you are going to get the same or similar benefit, and that what you’re doing isn’t just being done for technological purity. And if you’re supposedly getting a different benefit, then you should be very skeptical! And remember, even your genius techie has vast gaping holes of ignorance.
