Coding Next-Generation Enterprise (Application) Architecture Systems using Architecture as Code

Architecture is code, which is an idea of ​​architecture 
design and governance. It revolves around a series of 
architecture patterns, combines and presents architecture
elements and features, and closely integrates architecture 
decisions and design principles with the system.

For example, in my last article, ” Modeling “Architecture”: How to Describe Software Architecture in Code? As mentioned in the book, it is quite difficult to accurately describe the architecture of the software. At the implementation level alone, it’s hard to get everyone to agree on a standard model, “Oh, this is the architecture”. Therefore, when the architecture cannot be defined, it is difficult to give an architecture governance model that convinces everyone. After all: Models are only right, never right.

However, we (ArchGuard Team) will still build an architecture model and an architecture governance model in ArchGuard as a recommended ” best practice “. In addition to this, we should also provide a possibility to customize the enterprise application architecture, which is architecture as code. Junior architects, only need to implement ArchGuard’s best practices; for intermediate and senior architects, they can build their own architecture system based on the plug-in capabilities + DSL provided by ArchGuard.

So, as you’ve seen in other systems, to provide this capability requires a certain amount of coding, configuration, etc. Therefore, we need to build an architecture-as-code system. So, what the hell is code, exactly?

What is Architecture as Code?

In a previous series of coding, I described how to fully code software development, including converting documentation, requirements, design, code, build, deployment, operations, etc. into Coding. Designing and implementing a domain-specific language is not difficult, as the process described in the article ” Domain-Specific Language Design Techniques “, in this context is:

1. Defines the rendering mode. Look for a suitable way to present the architecture, such as UML diagrams, dependency diagrams, sequence diagrams, etc.
2. Refine domain-specific nouns. A series of architecture-related elements, such as architectural style: microkernel, etc., architectural layering: MVC, etc.
3. Design associations and syntax. How to relate these architectural elements in a natural way, such as keywords, parsing placeholders, etc.
4. Implement syntax parsing. In addition to implementation, another consideration is: how to provide more flexible expansion capabilities?
5. The design of evolutionary languages. version iteration

Therefore, we define architecture as code as:

Architecture is code, which is an idea of ​​
architecture design and governance. It 
revolves around a series of architecture 
patterns, combines and presents architecture
elements and features, and closely integrates 
architecture decisions and design principles with the system.

The next question is, how to organically combine this concept with the system? And friendly to provide such an API interface (DSL)?

To put it into the current PoC of ArchGuard, the presentation method of architecture as code is “ArchDoc”, interactive code analysis and governance method based on Markdown. That is, all “code” is managed by markdown, which has a lot of advantages:

• Express the schema using an inline DSL (managed with syntactic blocks)
• It can record the architectural documents of the system, such as architectural decisions, business architecture, etc.
• Has an extensive parsing library for more flexible customization inspiration (Ctrl + C, Ctrl + V).
• Custom Render
• Extensive editing tool support

The only downside is that implementing such a tool is not trivial.

Architecture as Code Features

However, we have implemented a simple PoC (Proof of Concept) version which it features:

• Explicitly describe and present the architecture.
• Architecture documents are rules
• Design, documentation, and implementation are consistent

Of course, there are all kinds of scalability (this is an all-too-common feature).

Explicitly describe and present the architecture

Back in our day-to-day life, we often hear architects say, “Our services use a standard DDD layered architecture.” However, is this layering a series of dependencies such as “Interface layer depends on application, domain, infrastructure layer”? Are the developers aware of these rules? These are questions. Therefore, an architecture-as-code system should be able to explicitly present that tacit knowledge in the system.

For example, we should explicitly declare the dependencies in the hierarchy:

layered {
prefixId("org.archguard")
component("interface") dependentOn component("application")
component("interface") depends on component("domain")
component("interface") dependentOn component("infrastructure")

component("application") depends on component("domain")
Component("application") depends on Component("infrastructure")

Component("domain") depends on Component("infrastructure")
}

PS: Please ignore the Chinese elements in the Kotlin code above, it is only used to illustrate the possibility of using Chinese descriptions. After all, if you are happy, you can also use classical Chinese.

Combining the DSL in ArchGuard and the visualization tool (Mermaid.js here), we can present the layered architecture we designed:

Combined with the designed layered Linter tool (in progress):

linter('Backend').layer()

Once the dependencies in the hierarchy are wrong, commits of those codes can be blocked in continuous integration – a mechanism similar to that of ArchUnit. The slight difference is that instead of putting tests and code in the codebase, you can manage them in a unified way.

For other series of more complex rules, we can customize them and combine them with the document.

Architecture documents are rules

In this mode, we can also combine documentation with code — provided that we have written a set of rules. For example, in ArchGuard, we have written a series of rules for different scenarios:

• SQL, if not allowed select *etc.
• Test Code, for detecting bad smells in code
• Web API, analyze whether the design of the API is RESTful
• Layer (to be implemented), analyze the layered implementation in the code
• Arch (to be implemented), similar to ArchUnit or Guarding to develop finer dependency rules
• Change (to be implemented), write custom change impact scope rules, such as a class should not be affected by other changes

Once we have the basic schema document specification, we can regularize them and combine them. Here is an example of a list combining Checklist and rules:

- [x] There should be no ignored (Ignore, Disabled) test cases (#no-ignore-test)
- [ ] Allow duplicate assertions (#redundant-assertion)

#no-ignore-testCorresponding to the rules in ArchGuard being implemented, and in the Checklist of GFM, if it is checked, it can be expressed as an open rule; if there is no check, it is not open. The previous text part is the corresponding rule description, which is slightly more flexible than the traditional linter.

And whether it’s the person writing the document or the person reading the document, they can easily construct the corresponding context.

Design, Documentation, and Code Consistency

Once you have the design and documentation, it needs to be integrated into the existing code to keep the three consistent and accurate. In our case, it is the existing API of ArchGuard, which includes:

• Create an analysis of the code repository
• Analyze code for syntax and build tools, change history, etc.
• Analyze code for compliance with rules, etc.

The following is the DSL for repo design in ArchGuard (based on Kotlin), which is used to create code repository analysis:

repos {

repo(name = "Backend", language = "Kotlin", scmUrl = "https://github.com/archguard/archguard")

repo(name = "Frontend", language = "TypeScript", scmUrl = "https://github.com/archguard/archguard-frontend")

repo(name = "Scanner", language = "Kotlin", scmUrl = "https://github.com/archguard/scanner")

}

Only when the three are consistent can we ensure that the design and implementation of the architecture are consistent.

What is Architecture as Code?

At the implementation level, an architecture-as-code system is a data system that supports orchestration. The reason is that we don’t want to care about the process of processing the data, but we want to get the results of the data and gain insights from the results. As we have seen, each big data system has built a visualization capability one by one, and we can pray to get insights from it.

However, slightly different from prayer, we conjecture with N% probability, so it is called exploration.

An Exploratory Architecture Governance

The traditional software development model is edit-compile- run (edit-compile-run). The premise of this development model is that we have enough business insights. For a business system with rich domain knowledge, it is not difficult to build such a system. But how do we go down when we lack enough domain experts? For complex problems, you can only explore (Probe) -> Sense (Sense) -> Respond.

And since we ourselves, like many architects of the new generation, also need to explore and analyze, and then come to a conclusion. So, let’s try switching the mode again. Just like, we build ArchGuard’s software development model, which is also executed -explore. First, we analyze the system (publish an analysis function), they cooperate with the existing model, and finally, get a “conclusion” or rule (publish another one). linter function).

In the data field, this method is quite popular. In the past, people used IPython, but now they use Jupyter; the other type is a report-like idea similar to that provided by RMarkdown.

• IPython is a multi-programming language command shell for interactive computing.
• Jupyter Notebook is an interactive computing tool that runs on the web.
• R Markdown。 Turn your analyses into high-quality documents, reports, presentations, and dashboards with R Markdown.
• Observable for the D3.js community. Explore, analyze, and explain data. As a team.

In terms of mode, ArchGuard is more inclined to the RStudio mode, but in terms of community resources, there are more implementations related to Jupyter.

A “big data system” that often OOM

In our (ArchGuard core team) “several discussions”, it was finally concluded that ArchGuard is a big data analysis, not a simple data analysis. The reason is that there are a lot of bugs and big data-related (dog heads) in the system:

• There is a certain amount of Out of Memory.
• Visualization optimization in the case of large data volumes.

Also known as “bug-driven architectural design”.

In addition to this, another interesting point is that larger systems, it has a large number of new commits or new branches. We need to consider: dealing with continuously submitted code and building incremental analysis functions.

When we try to use big data ideas, such as MapReduce, Streaming Analysis related patterns to solve related problems, we find that it can work well – after all, it is data analysis.

How is it implemented in ArchGuard?

ArchGuard builds an interactive architecture analysis and governance platform around DSL + Kotlin REPL + data visualization. Because it is still being implemented, it is called the next generation.

1. Refine architectural elements

The above (www.ascode.ink) series also includes two architecture-related tools, one is the code generation DSL: Forming and the other is the architecture guarding DSL: Guarding. What the two DSLs do is group together architectural elements around specific rules, here architectural elements.

If you haven’t done it, this process seems to be quite troublesome. There are some fairly simple things for reference (copy) in the implementation:

• Architecture Description Language Paper (ADL). ADL is already a very mature field. In the era when design patterns were popular, architectural patterns (“Pattern-Oriented Software Architecture”) were also very popular.
• A directory of architecture-related books. A good architecture book only needs to look at the table of contents to have an index, so it also has the basic architectural elements.
• A schema language for architecture. Pattern languages ​​represent relationships between patterns
• ……

It’s boring to just copy, if you can do abstraction yourself, it’s also a very fun thing.

2. Build plug-in and rule analysis

As mentioned above, in ArchGuard, we try to build the rules of the system as a series of rules, and these rules are exposed in the form of plugins.

This means that such a system should support custom plug-in capabilities that allow you to:

• Access a new language
• write new rules
• Build a pipeline of new data

What also needs to be improved in ArchGuard is the ability to provide a kind of metadata.

3. Abstract DSL as glue

From the implementation level, in order to support the ability of bonding, we currently plan to design three types of DSLs: back-end architecture query DSL, architecture DSL, and feature DSL.

Backend Architecture Query DSL

Similar to LINQ (Language Integrated Query, language integrated query) encapsulates CRUD interfaces to provide compile-time type checking or IntelliSense support, in Kotlin there are forms such as: KtOrm . Such as:

database

.from(Employees)

.select(Employees.name)

.where { (Employees.departmentId eq 1) and (Employees.name like "%vince%") }

.forEach { row ->

println(row[Employees.name])

}

Written like a programming language, it can provide more friendly syntax support.

Architecture DSL

That is, the Architecture Description Language (Architecture Description Language), provides an efficient way to describe the software architecture.

Feature DSL: Analysis, Scanning, and Linter

That is to encapsulate ArchGuard Scanner, Analyser, Linter, etc., to build the basic architectural features required by the system.

4. Build an interactive environment

Two years ago, when building Ledge with many Thoughtworkers, we have been emphasizing document coding and providing an interactive document environment. In Ledge, you can use Markdown to draw all kinds of charts, just by declaring the chart type, see example: https://devops.phodal.com/helper.

In terms of mode, ArchGuard is more like a combined version of RStudio + Jupyter, that is, a REPL that provides a large number of custom graphics + component capabilities.

On the REPL, since we plan to use Kotlin to build the DSL, what we need to look for is the Kotlin REPL. The kotlin-jupyter officially created by Kotlin has become a good reference, but unfortunately, it has not been used yet. At the same time, Kotlin has the scene of Kotlin Scripting in the early stage of design, so it kotlin-scripting-compiler-embeddable meets the needs. So, in the PoC, we introduced the Kotlin REPL with reference to Apache Zeppelin and created a WebSocket as a service.

In terms of visualization, it is a little more complicated, and you need to build a Markdown parser, Block editor, etc. We have temporarily adopted Mermaid.js as one of the visual graphics libraries, and D3.js and Echarts are also one of them. The remaining question is how to integrate them through the DSL? Is building a front-end and back-end data model a temporary solution?

See screenshot for PoC example:

5. Still a PoC

Here, the interactive analysis of ArchGuard is still just a PoC (proof of concept), but in the near future, you can use it in ArchGuard.

Other

Building such a complex tool is not an easy task. Welcome to join ArchGuard, learn architecture and architecture governance together, and develop a pure technology-driven open-source software.

If you want to practice the following techniques, I will teach you how to:

• Compiler front end. Interested in designing and implementing DSLs
• Compiler peripherals. Kotlin’s compiler uses
• ……
• Of course, if you are also interested in:

Summary

Improve a legacy system. Refactoring and designing ArchGuard’s front-end and back-end.

Although we can’t provide you with code, maybe we can in the future — as an open-source self-research project, what if we can build an open business model? In addition, you can show your code in public and recognize more architects in the industry who know you.