Agile Architecture

What's should an Architect in an Agile environment do?

Irrespective whether we have a single person as "assigned architect" or have architecture as a shared responsibility of (one or more) teams - there's a massive difference between "Emergent Architecture" and "Architecture by Chance". 
The one thing worse than having the wrong goal is not having a goal at all - and that's true for architecture as well as for the product from a customer's perspective, so let's discuss a bit of what the scope of agile architecture is.

Note: This article is my current, personal view on the topic and I sincerely invite feedback and criticism if I'm off.

Architectural Intent

Architecture may or may not have an intent. Such an intent is usually to create an overall system that minimizes cost or effort for one, more or all involved parties.
This intent can be formulated either in the short, mid or long term. If it isn't formulated at all, the probability quickly diminishes that the optimization goals of the organization are indeed met.

Optimization goals

Typical optimization goals for architecture could be, for example:

• Minimize Cost of Change
• Minimize Time to Market
• Minimize working hours
• Organizational autonomy

You're free to pick your own, based on your specific circumstance.

Organizational Constraints

Sometimes, we also operate under constraints, which may need to be considered as well. 

Examples might be:

• Regulatory compliance (e.g. privacy laws)
• Technological limitations (e.g. company policy)
• Security constraints (e.g. contractual demands)

These restrict the solution space and should be handled with caution - too many suffocate innovation, while missing any essential ones can kill the product! (as seen in the recent Sixt vs. Accenture case)

Not Strategic Architecture

Although the following activities are indeed part of architecture, these topics are not strategically relevant from an architecture perspective: The more fluently they can shift throughout a product's lifespan, the better other optimization goals can be met.

1. Formulating, planning and forecasting system capabilities
2. Defining interfaces, processes, data models and interfaces
3. Component delivery strategy and component responsibility
4. Deciding on specific techonologies to solve certain problems

While these things are relevant, agile teams can decide those things on a per-item basis and can indeed evolve and dynamically change when knowledge and skills to do these things are available in the team.

Architectural Direction

Some core questions may be worthwhile answering upfront. These are directional rather than content-wise and can be answered with "Yes" or "No":

1. Do we want to have Loose Coupling?
2. Do we want to enable Micro-Releases?
3. Will we go xAAS?
4. Do we need scalability?

Note that I'm not talking about big architecture upfront here - even for my company homepage, I have answered these questions by taking a couple minutes to orient myself what I want to achieve before writing the first line of code. 

Architectural vision

A vision that nobody can remember is worthless, as is a vision that doesn't help in orienting activities. Using the above questions, a vision statement could be formulated like:

"To develop a system where changes can be done (and undone) at any time within seconds, we create loosely coupled components that can be independently released as autonomous services which can be flexibly scaled."

Then comes the difficult part: getting there.
And that's more difficult on a "brown field" than it is on a green field.
In order to get there, we need to have some mechanisms in place, which may or may not be helpful depending on where we want to be.

Core paradigms

Where we may need to actively invest effort into enabling and establishing some core paradigms for an equally robust and flexible architecture. 

These paradigms could be, for example:

• Continuous Integration/Delivery/Deployment
• Automate Everything
• Measure Everything
• Test-Driven Design
• Impermanent Components
• You Build it, you run it (“true DevOps”)

This means that every architectural decision made by the teams can be benchmarked by whether (or not) an action, decision or choice brings us closer or distracts us from these paradigms.

As such, we may add to our architectural vision further statements like the following:

Every component uses a Continuous Delivery process, and where possible, even Continuous Deployment. 

Every activity within the pipeline and on the production environment is automated. 

We have real time data on every aspect of the system and therefore full transparency on its behaviour. 

Code that hasn't been designed based on executable tests doesn't go live. 

Replacing components is a low-effort task. 

Developers have both responsibility and authority regarding production.

All of these paradigms are still outside the specific solution space and do not answer the "How".

Architectural roadmap

After we have agreed on where we want to go, we need to actually go and do it. Since our customers don't pay us to do architecture, but to deliver something of value, these actions should be seamlessly integrated with our development efforts, ideally experimenting with practices and introducing tools while we're delivering tangible value.

If we're in a Green Field, setting up a CD pipeline and delivering an automatically tested "Hello World" into a measured, containerized production environment may bloat up the first user story by a day or two, and then speed up everything that comes afterwards.

On a Brown Field, that's an entirely different story - we can't just do this so easily. What if we have 16 million lines of unsustainable code riddled with defects? CD would be insanity. There's no data from Production, nobody knows the current state of the deployment environment and extracting a single feature from the existing monolith could take months.

We then get into an architectural action roadmap that might look like this:

1. Short term goals
1. Automate build process & set up CI pipeline.
2. Agree with stakeholders to accept current "status quo" as baseline quality for CI/CD.
3. Every new line of code gets a unit test.
4. Build metric hooks and set up a measurement system.
5. Build service API for core monolith.
2. Mid term goals
1. Introduce Coding Conventions and add measurement to CI pipeline
2. Stop the current branching process and move towards genuine CD process
3. Automated deployment to Test Stage
4. Automate e2e tests for critical user journeys
5. Decouple frontend and backend.
3. Long term goals
1. Build new capabilities as autonomous services.
2. Containerize services.
3. Extract service components from existing monolith.
4. Move towards polyglot development.
5. Move individual services to Cloud.
6. Build Security Services (IAM, Secrets-as-Service, etc.)

These items can then be added into the Product Backlog and prioritized as needed. The authority and autonomy on how to do this specifically still rests within the development teams.

Any architectural activity that doesn't trace back to a goal may be questionable. Any architectural goal that doesn't trace back to a paradigm invites scrutiny - and if anything doesn't match intent, another discussion may be in order.

Architectural responsibility

None of the above items mandate having a specific "architect" role, although in larger systems, it may indeed be worthwhile to have at least one person take responsibility to see that there is indeed clarity on the architectural:

1. Vision
2. Intent
3. Direction
4. Goals
5. Paradigms
6. Ongoing Activity