In the previous posts, we modeled and tested domain behavior and created a trigger to start this behavior via a command. We mentioned the concept of Notifiers a couple of times with the remark don't worry about it now. We continue the Modeling in Tracepaper series by focusing on what are Notifiers and what can we do with them. The concept overview visualizes the scope of this post.

Notifiers are comparable to aggregate behavior because they both represent logic flows, but they do differ drastically.

As you can see in the visualization both execute a flow of logic after receiving a trigger. A trigger is always an event, for the execution it is irrelevant what kind of event.

• Domain-Event is emitted by an Aggregate.
• Actor-Event is published by the API (it is modeled using a command)

There are two key differences:

1. State: An Aggregate has state and the flow of logic has read access to this state. It can manipulate this state by emitting domain events. A Notifier is stateless.
2. Connectivity: An Aggregate has no internet access*. A notifier can connect to the internet and access any API you like. Even the project's API. A notifier doesn't emit domain events, but it could emit actor events with the aid of an API call.

*You could bypass this limitation by introducing custom code, but you really shouldn't. An aggregate should not be dependent on anything other than triggers and their state. In a future release, we may even enforce this.

Endless Possibilities

Accessing any API you like opens a world of possibilities. For example, sending an e-mail via AWS SES. But you are not bound to use AWS services, for example, we send our emails via the MailGun API.

One of the constructs we use a lot in Tracepaper is a read-loop-write. For example, refactoring a subdomain name entails updating the functional keys of the "child" aggregates (remember the canonical key concept?)

So we fetch all child keys with a GraphQL query, loop these keys, and trigger a GraphQL mutation on these elements

The before-mentioned construct is used for the refactoring and cleanup notifiers. The other notifiers are used to control identity and access management e.g. creating groups when a new workspace is created or adding or removing users from said groups. Sending emails or flushing the content delivery network (CDN) we use for caching the front end.

Send an Email

To get back to the modeling example. Let's say we want to send a welcome email when somebody opens a new account.

As a core concept, you can find the notifiers tab on the main page. But just like commands, there is a more convenient way to "attach" a notifier to a domain event using the autofill feature.  

I'll break it down for you.

We only covered the basics in this post, but it gives you a pretty good idea what you can do with notifiers.

Tracepaper backlog regarding Notifiers.

Thank you for reading. Next time we will look into Views.

In the previous posts, we modeled domain behavior and created a trigger to start this behavior via a command. We continue the Modeling in Tracepaper series by focusing on the quality aspect. Some of it is already available in Tracepaper. Additionally, we will present some plans for the future. The concept overview visualizes the scope of this post.

Modeling is all fun and games until you break production. We need tests to prevent introducing breaking changes. As an architect, I'm interested in correct test coverage. As an engineer, writing tests is not an energizer for me.

Behavior Flows

If we look at our behavior flows from a testing perspective, they are fairly simple.

A message triggers behavior, it may interact with the internal state for validations or extrapolations. And it may result in domain events and state changes.

And that is how we model our tests, where the flow is considered a black box. We model the intention of the test case by declaring the state and the trigger and asserting that the expected messages are emitted. Optionally we check if the end state meets expectations. Every behavior flow should have at least one test case. This is enforced by the engine, it will reject a flow without test cases.

How many cases you need depends on the complexity of the flow. In Tracepaper, for example, we have 154 flows and 216 test cases. Most flows have just one test case because the complexity is low. Some flows have up to 5 test cases.

We have not yet figured out how to do proper test coverage calculation. We could use simple code-line coverage, but it feels lazy. It is more valuable to base coverage on the model:

• did we use all defined triggers?
• did we see all defined events emitted?
• did we see all modeled validation errors?

We know what we want, but we didn't take the time yet to implement it. It is on our backlog somewhere.

Notifiers

Compared to behavior flows, notifiers are a harder concept to test. I did not explain notifiers yet, we will take a look next week. For now, you have to take my word for it. Notifiers interact with (external) APIs, so testing them means we have to figure out how to mock these APIs.

We don't support tests for notifiers yet, but we use them a lot in Tracepaper (19 of them) and they do important things. Bluntly said, if we break the modeled notifiers Tracepaper is effectively worthless. How do we deal with this? We use a more expensive test method, end-to-end testing.

Test Scenarios

We have a concept called test scenarios, essentially a scenario is a sequence of API calls against the generated API. The API is generated and extracted from modeling concepts. So when modeling a scenario, we don't think about the API, we think about domain concepts: commands/behavior/notifier/view. In the background, it leans heavily on track and trace.

The scenario must be executed against a deployed version of the application, running these test scenarios is a pipeline responsibility.

We are alpha testing this feature at the moment, when we are happy about it we start adding scenario modeling to Tracepaper and making the extended pipeline available for general use.

A lot of plans, but what can you do at this moment in Tracepaper?

Truth be told, it is fairly limited. Sending in a trigger message and asserting that the expected domain events are emitted. It is very basic but good enough for our beta.

Here is a filter on our backlog so you know what to expect in the coming months.

Thank you for reading!

We continue the Modeling in Tracepaper series by focusing on the interaction with our write domain. Last week we left off with the wish to trigger our behavior flow from the GraphQL API. This write interaction is modeled using commands. The scope of this post is visualized in our concept overview.

In IT, a command is a message just like an event. The two differ subtly.
1. An event is a message that represents that "something" has happened.
2. A command is a message signaling that we want "something" to happen.

A command is a wish, and an event is a fact.

We made a conscious choice to use events only to trigger behavior-flows or notifiers. The reason is quite crude, if we only have to handle events we can keep the cognitive complexity of our modeling tool (Tracepaper) in check.

If you are unfamiliar with GraphQL, this post describes the key concepts in a nutshell. It also features a 5 minute video if you prefere an audio visual medium.

We gave the write side of the API (GraphQL Mutations) 2 responsibilities:
1. Convert a command into an event. "OpenAccount" becomes "OpenAccountRequested", it does sound silly but it does effectively convert a wish into a fact. Of course, it may happen that the domain is not capable of processing the event. But that does not change the fact that somebody requested it.
2. Convert the synchronous API call into an asynchronous message.

The client issued a command however, this means they are probably interested in knowing if the processing was successful or not. To facilitate this, the API will return a correlation ID in case the command is accepted. The client can use this identifier to query or subscribe to the track and trace functionality. You may have noticed in our previous post that after every form submission we see a stream of events that have happened in the Tracepaper domain. This is a subscription to track & trace.

It is not necessary perse to show these trace events in our user interface, we do show them however and the reason is two-fold.
1. It gives insight into how our generated applications cope with asynchronicity and how we model our domain.
2. I like to see that something is happening after I press a button.

Modeling a Command

We need to model two things when modeling a command.
1. The message structure.
2. The API structure and access management.

Commands are a core concept and therefore you can navigate to them from the main project screen. A command is thus not bound to another domain concept, and it is perfectly possible to let multiple aggregates and notifiers react to a single command. Remember we convert the command to an event as soon as possible.

You can see that the first fields are meant for modeling the API structure. The GraphQL Namespaces are used to logically group commands together to increase the readability of the API contract. Namespaces can be nested by separating them with dots.
The GraphQL method name is the actual method in that namespace, those values should be unique together. We generate the command name from those two fields. And in the backend we generate the Event name from the command name:
GraphQL Namespace: Subdomain.Aggregate
GraphQL Method name: create
Command Name: CreateSubdomainAggregate
Event Name: CreateSubdomainAggregateRequested

We can select an authentication method, the default is authenticated, meaning that the user just needs to be signed in. We have 2 other options (there is secret option 3, anonymous access, but this is not yet enabled):
Role-based requires the user to have a specific role, this one is used a lot in Tracepaper.
User-based: requires a specific message field to be equal to the signed-in user name. For example, to make sure that only the user itself can mutate, let's say, their profile aggregate instance. The engine has another mechanism to ensure this, but unfortunately, this is not yet supported by Tracepaper (at the moment of writing, check this refinement task for the actual state).

The command input fields are used to model the message structure, they represent a JSON structure. We do allow 1 level nesting (Hence the 'NestedString' type), and they are always used for collections. The message structure for this command (naturally, this form is backed by a command that uses the same semantics) looks something like this:

If we would submit this form, the actual JSON structure that is sent to the API looks like this:

This is simplified actually because I don't want to bore you with the nitty-gritty details. But here is a screenshot from the actual API structure:

We now know how commands are used and modeled, and we know that they are not necessarily bound to one behavior flow or notifier. But you can imagine that most of the time we model specific commands to trigger specific behavior. With that in mind, there is a more convenient way for modeling commands using the best-effort autofill feature. And to utilize this we initialize the command modeling from the context of modeling a trigger for the behavior flow.

Let's look at it step by step:

We are done, we created a trigger for our behavior flow that is accessible via the generated API. Before we can show the API, we have to build the code, and we can't build the code before we have tests, a repository, and a pipeline.

One of the curses of writing is I'm bound to provide a linear flow of information while the subject is more of a graph. So for now, to protect my linear flow, let's focus on the modeling. We will look at pipelines and delivery after this series is done.

Just like the Aggregate, the Command Concept has more features than are available in Tracepaper at this moment. Here is a filter on our backlog to get a feeling of what to expect in the coming months.

This is a good time for a break, next time we will model a test for an Aggregate Behavior-Flow. This post was particularly hard to write, I think because the Command concept is a pretty hefty abstraction, it does quite a lot. A piece of advice I give to all interns and fresh architects.

Just because it was hard to write doesn't mean it should be hard to read. If your audience doesn't understand it, you failed.

And with that in mind, looking at this blog post, I may have failed this time. Thank you for reading, if you have feedback let me know.

We continue the Modeling in Tracepaper series by focusing on the hearth of our application. The Domain logic is encapsulated in aggregates. The scope of this post is visualized in our concept overview.

What is an Aggregate? An aggregate is a concept borrowed from Domain Driven Design. We used 3 core concepts regarding the aggregate:
1. An aggregate instance contains an identity and holds an internal state.
2. An aggregate defines a transactional boundary. This means any changes to the aggregate will either all succeed or none will succeed.
3. An aggregate represents a domain concept.
It is quite literally an aggregation of stateful behavior to represent a domain concept.  

A commonly used tactic is to model an aggregate as a cluster of associated entities and value objects treated as a unit for the purpose of data changes. Data changes are dictated by rich domain behavior, unlike CRUD-style applications.
At Draftsman, we use an alternative tactic for state persistence and manipulation, Event Sourcing.

In short, an aggregate in our modeling tool Tracepaper will contain:
1. a definition for the state, modeled as aggregate attributes.
2. domain event definitions, a domain event may mutate aggregate attributes (among other things.)
3. Behavior Flows, a behavior flow is a sequence of logic functions, triggered by an event, and may result in the publication of domain events.

An aggregate can only interact with the outside world by receiving and publishing events. For example, an aggregate can't use API calls to fetch data or send a mutation to another aggregate. All data needed for the behavior flow must be encapsulated in the trigger event, subsequently, other aggregates can subscribe to the event stream.

We do have a concept of Notifiers that are able to interact with the outside world. Just as an aggregate it is triggered by an event, whereafter it can orchestrate API calls to other systems or the application API itself. Unlike an aggregate, a notifier does not hold any state. But this is a subject for another post.

The Subdomain

An aggregate is not free-floating in our domain, in Tracepaper we support a concept called Subdomain that acts as a cluster of related aggregates. A subdomain is a bounded context within our domain. So before we can start modeling an aggregate, we have to initialize a subdomain.

The Aggregate

Let's say we have a subdomain accounting and we want to model a concept Account that is used to store the account balance. It has behavior for:
1. Opening an account.
2. Deposit into the account.
3. Withdrawal from the account.

We start with an empty aggregate. The first thing to do is to add a state definition.

Our state definition is very simple, but could definitely be more complex. The next thing to do is start modeling behavior, let's start with the behavior to "Open an account". Let's say we have an additional requirement when opening an account that we will reward everyone with a 10-credit addition to their balance as a welcome gift. With this in mind, it is important that the mentioned behavior is only applicable on non-existing accounts, read, and can only run during aggregate-instance creation.

We have initialized a behavior flow. A behavior flow is a chain of processors that together define the behavior. We have access to the internal state of the aggregate, although this is limited to read-only. So we can't manipulate the internal state, so let's create some flow variables that we can use as temporary storage.

Tracepaper provides "best effort" auto-filling fields. It saves you some typing, but you are not absolved from thinking... double check everything.

A behavior flow is always started from a trigger, a trigger is always an event. We have 2 types of events in our domain.
1. Domain-Events, emitted by aggregates. These will be introduced in this post.
2. Actor-Events, these are triggered by commands and can be triggered via an API call. An actor event always follows the "{CommandName}Requested" naming semantic.

For this flow we want to use the second type of event, this means we have to model a command. Modeling the command is a subject for another post. So for now, let us just assume that the flow variables emailAdress and balance are filled at the start of the flow with aid of the trigger mapping.

We skip ahead to the processors, here we can model the requirement to add the 10-credit gift. We do this by adding a "set-variable" processor.

We selected "balance" as a target and provided the expression "flow.balance + 10". The expression is evaluated as python, because that is the language we selected for our applications. When you generate source code you will find a line:

flow.balance = flow.balance + 10

We stated that you have access to the internal state of the aggregate, but issuing the next statement is not gonna do it.

flow.entity.balance = flow.balance

This statement will execute successfully, but it won't be persisted. To update the internal-state we have to emit a domain-event.

A domain event is an attribute of the aggregate and could be emitted from multiple behavior flows, it is not bounded to a specific behavior flow. This means we could create one from the aggregate screen. It is, however, more convenient to create one from within our flow. This does not change the positioning of the event, it is still bounded to the aggregate, but we will be supported by the "best effort" autofill functionality.

A little side step, maybe you noticed that I had to refresh the page during the demo. The thing is, we have an event-based architecture. Even the communication between the front-end and back-end is a mix of synchronous and asynchronous communication. This means everything is eventually consistent.
In the backend, we have something called the transactional outbox and various retries to protect us from missing events. The front-end, however, may sometimes miss an event, meaning that your projection is not consistent with the application state. Luckily we can just refresh the page with F5. Most of the time this will fix the issue. If not, you could click  "Tracepaper" in the upper left corner this will ignore any cached data.

A lot happening on the screen, let's break it down.

The first thing we did is model a domain event. A domain event has an internal name (Opened) that is unique within the aggregate and an external name (AccountingAccountOpened) that is unique within the domain.
It has fields, and those are the attributes that are persisted in the event log. There is also a handler-mapping. The handler-mapping defines how the event is applied to the aggregate... It is a setter.

The event is added to the aggregate view, and from here you could edit it.

You can choose to edit only the mappings.

Or you could edit the fields and the mappings. Let's say we change the balance from Int to Float.

This will result in a new event version to ensure backward compatibility, events are persisted after all and you may already have events in your log when you decide an additional field is needed in the event. We don't want our software to crash because we have trouble replaying the event log to determine the state.

Emiting the Domain Event

The second thing we did was model the emit-event processor. It is pretty straightforward. We select the event we want to emit (AccountinAccountOpened) and we map flow variables to event fields. This is best effort auto-filled, but at this point, my browser became out of sync and needed a page refresh.

Tracepaper backlog regarding the Aggregate.

A good time for a break. The only 2 things left for this behavior flow are a trigger and a test. As mentioned, we want to trigger from a command and that deserves a separate post. Modeling the test without a trigger is not possible, so this has to wait. Thnx for reading, until next time.

In June 2021 I wrote a post proclaiming that we don't have a roadmap and that we don't need it. And the argument we build what we need only when we need it still holds... Only for the engine though.

If we take a look at an abstract design.

1. Tracepaper is our modeling studio, a form-based visual tool to map out your application domain.
2. Tracepaper will export a technical model, this model is XML based in what we call Draftsman Modeling Language.
3. The Draftsman Engine converts this technical model into code.
4. The generated code is stored in a GitHub repository.
5. An AWS CodePipeline will build and deploy the application.

The engine has one function, convert the technical model into code. It is literally a stack of enablers on top of enablers. Mapping concepts to implementations. For example, we need the concept of state. We decided to use Event Sourcing for maximum flexibility. We design XML structures to model events and event handlers (we sometimes call them glorified setters). And we implement code that converts the designed XML structure into Python code and Cloudformation magic.

That's it, it doesn't do anything unless you feed it some specific XML. And that is what we did, we started modeling the Tracepaper application by writing the XML by hand. And that is how the engine grew. During the modeling of Tracepaper, we encounter missing concepts (querying, integrations to external APIs, event-chain tracing, and many more.) And that is how the stack of enablers grew. We built what we need when we need it.

In the generator report you can see the concepts mentioned, how many lines XML are used to describe the model and what is generated. In the screenshot you can see the generator report for the application that is modelled during the tutorial.

If you are curious, I also put a report online after converting the Tracepaper model into code. If you compare the two reports (and ignore the fact that Tracepaper has a bigger domain than the tutorial) you will notice that Tracepaper uses concepts that are not available to our clients. And there is our backlog...

The engine can do many fancy things and we even started to forget some of the features. So we decided to write down all the things that we need to model into Tracepaper, and this became some sort of a roadmap. So there you have it, the engine are enablers and we make it up as we go. But Tracepaper should make those enablers understandable to humans, and that is something we can plan.

Our velocity varies, but at least we have everything on a Trello board to keep track of the things we should do.

First of all, it's been a while for multiple reasons. Work, life, and the need to spend free time with friends and family.
The most joyous reason to not spend time alone with a computer screen is the birth of my son :)

But my son likes to sleep a lot, and preferably he cuddles up inside a baby pouch close to mom or dad. So sitting here as a marsupial and listening to soothing white noise, maybe this is a good time to pick up writing again.

Efforts on Tracepaper did not stop altogether, we just entered eco-mode, nonetheless, we can announce a new feature:

You can now rename the resources you previously modeled in Tracepaper!

I'm fully aware this sounds lame AF, but hear me out it gets better. To understand why renaming stuff is not trivial you need insight into the segregation between read and write models that we use inside Draftsman Generated Applications.

Let's start with the read-model (the green stuff ), the read-model is data-driven. Clients query this model for their data needs. We have to look at implementation details to understand some limitations. One of our guiding principles is serverless first to minimalize cost, and operational burden while guaranteeing high performance. One of the choices that reflects this is the selection of DynamoDB.
DynamoDB is a high-performant key-value store with document support, it provides filtering capabilities but to guarantee performance (and low cost) you need to be smart with your keys. Limit the set to be filtered so to say.  
Keys in our read-model need meaning, these keys are used as identity so the client can fetch an object based on its identity.
The first reason why our keys need meaning: clients need to know what to ask for.

We use the identity as the primary key in the view-store (actually as an element of the primary key, but more on that later), you can't change your primary key (I did not work with all databases known to man, but all DB's that I worked with have this "limitation"). To tie back in with the renaming, a rename probably means changing its identity and therefore the need for updating the primary key. From the view-store perspective, this means removing the old record and inserting a new one.

The view-store does not contain the truth, it is a mere projection of the truth.

This means we consider the view store as volatile, meaning, we could drop this table and repopulate by replaying snapshot events. This feature has proven to be valuable during concept experimentation in our development environment.

We use a single-table design for the view store to reduce infrastructural complexity, mainly regarding system access management but also to support operations. Suppose your application receives a high load (congratulations) if that happens you could opt to switch from on-demand to provisioned capacity for this specific table. As stated in the DynamoDB documentation:

You should maintain as few tables as possible in a DynamoDB application. Having fewer tables keeps things more scalable, requires less permissions management, and reduces overhead for your DynamoDB application. It can also help keep backup costs lower overall.

How did we implement the single-table design, let's take a look at the view table in our staging environment:

DynamoDB is a key-value store, but it does actually support the use of a 2-part composite key. This mechanism is used for the event stores and the view store, but let us focus on the latter.
We have a column named type, this is the partition key. A partition is a sub-collection within the view table e.g. if I would (I'll use pseudo queries for readability) select document where type == 'Project' and key == 'D1562559209:personal:Demo' I would receive a specific document.
The key field is actually a range-key, the unique identifier within this sub-collection.

If I fetch documents where type == 'Project' I will get the whole collection. Then I could filter this set to get all Projects that belong to a certain workspace.
But we can do it smarter, we model our views as nodes in a graph. The entities have relations with one another. For example, a workspace contains projects and projects have subdomains, etc. If we take a look at a GraphQL query:

We get a workspace and within this document, we expect a collection of projects. as it is a GraphQL API we only retrieve the information we ask for. The result shown on the right side is not 1 document but the aggregation and filtering of 3 documents 1 Workspace instance and 2 Project instances. The GraphQL server "knows" it needs to perform a second query to fetch the projects (don't worry, that is one of the things we can generate). It would be wasteful to first fetch the whole Project collection to only filter it afterward. Luckily DynamoDB supports an alternative:
fetch documents where type == 'Project' and key begins with 'D1562559209:personal' providing faster response times and lower cost.
The second reason our keys need meaning is because we use them to model hierarchy and relations.

We call this a canonical-key (data architecture is not my field of expertise if you recognize this concept and know what it is called please send me a message at papers@draftsman.io). This canonical-key plays also an important role in our access management and the way we modeled multi-tenancy, but that is a post on its own.

The way this is set up means you don't have to worry so much about cyclic dependencies in your view model. This is demonstrated in the completely nonsensical next query.

In this query, I traverse the relation workspace <-> project a couple of times in a cyclic manner. Not useful in this case, but it does work nonetheless.

So a lot of reasoning has to do with modeling relational data in a non-relational database. If the data is relational why use a document based database?
AWS does provide a serverless relational Database AWS Aurora. Two reasons why we didn't select it.
1. V2 was not generally available at the time we needed a decision (and V1 was not sufficient for what we wanted to do with it).
2. It was not serverless enough for us. It requires you to think about e.g. instance size. Secondly the pricing was not granular enough.
But Aurora sparks interest, so we do keep an eye on it for future updates.

Enough about querying and relations let us look at the write-model (the red stuff), the write-model is very much behavior driven. It receives events, executes some logic, and appends additional events. Determining state is the act of replaying all past events from a log.  The aforementioned event log is long-lived and therefore we can't use the canonical-key from the view model. The canonical-key has meaning and therefore it can't be inmutable. When we change the meaning of things e.g. a rename because I am human and made a typo the first time it may cause the identity to be affected as well. So no canonical-key as primary in de event stores. If we ignore this we can't change the meaning of this key without corrupting the event log.

So the write model needs a technical key, so we chose a UUID-String. And if we take a look at the event store you can see clearly see that the primary key is a UUID, this is in fact the partition key (the collection) and for the range key, we use a version number. This way the program can retrieve the whole collection and replay the events to determine the current state. Because the pair must be unique we can avoid collisions with the aid of optimistic locking.

From the client's perspective, however, I don't want t know about these keys. They are technical, better word would be internal keys. From the GraphQL API perspective I wan't to use the meaningful canonical key:

If we would take a look at Track and Trace:

You will see that the update was successful, and in the project event store an event is added:

The application is perfectly capable to figure out how to translate the external key (canonical key, in our case the Draftsman Resource Name drn) to the internal key (UUID). To do this we introduced the key chain.

The best part, we compressed this into the Draftsman runtime. So you only have to worry about the functional keys (You don't have to design a canonical key, but it proved to be very beneficial for us.)

Why did I ramble on for 1000+ words on key design, query concepts and whatnot if it is abstracted away? Well, I had to explain why this feature is not as lame as it sounds right? ;)

But it hasn't all been champagne and caviar. Not everything is magically fixed with the addition of the key-chain, some things needed modeling e.g. the cleanup of deprecated view instances. After a rename, an event is thrown that causes a new insert into the view-store instead of the usual overwrite. So we needed to model view updaters to get rid of the deprecated view instance.
Secondly aggregates might reference each other, e.g. a behavior-flow references a command (because it is a trigger). This reference needs to be updated, and therefore we modeled notifiers (if-this-than-that) to "fix" this. I will not go into detail, for now, I should first finish the Modelling in Tracepaper series.

Comments and questions are welcome at papers@draftsman.io

Cheers!
Bo

The title says it all. But before we dive in, let's recap what we wanted to build 2 years ago, what we learned, and how it changed our goals.

TL;DR; Tracepaper is a High Productivity Framework with a designer studio. It is not done yet. But there is enough for show and tell. Open up the tutorial and start modeling in Tracepaper!

Our guiding design principles

1. Code transparency
2. Serverless first approach
3. Visual coding concepts

The pivot

We started building a no-code platform, but as it turns out, making everything a visual concept may cause the cognitive complexity to increase. And our goal is to decrease complexity. So fairly early on during the design of Tracepaper, we settled for a low-code platform to reach our goals.

But, low-code is kind of a misnomer, Gregor Hohpe wrote a comprehensive article regarding this topic. So we changed one of our guiding design principles: visual coding concepts became reduce cognitive complexity! We still use visualizations, but that is just a means to the goal. Are you still with me?
At this point, we pivoted to a High-Productivity Platform.

But we are still not there, if you remember one of our key constraints is the fact that we are a 2 person team building this product in our spare time. So supporting a complete platform was not an option and therefore we offloaded the heavy lifting to AWS. So we are not a platform (PaaS) we settled for a Framework and a Designer Studio (SaaS).

Draftsman engine is a High-Productivity Framework supported by a Designer Studio called Tracepaper.

What did we build

Let's take a look at what we have created and how the different components fit together.

The Studio

In Tracepaper you can model your domain with the aid of forms, wizards, visual models, and custom code.

The engine

The Draftsman engine converts the created model into source code. With the aid of an AWS pipeline this code gets converted into a working application.

One of our guiding principles was Serverless First approach, therefore the entire application is constructed with AWS building blocks like:

• Python code in Lambda's (compute)
• Dynamodb as Database
• Cognito for user management
• EventBridge as a smart eventbus
• AppSync as a smart API (GraphQL)
• Secrets manager... well, to manage secrets
• Cloud Formation (Infrastructure as Code) to stitch it all together

The Infrastructure as Code (IaC) makes it posible to use a single repo for the entire generated application.

Our aim is to reduce cost and cognitive complexity. To manage cost we use serverless, to manage the added complexity that is were te modeling comes in. The modeling tools in Tracepaper are mainly ment to handle "the grey stuff" wiring everything together. "The orange stuff" aka the business logic, the things you realy care about, those can partialy be modelled or are created by adding custom code.

Full Code transparency

You build the app, we only generate some code. That code belongs on your GitHub Repo and should be running in your AWS environment. Thats how we guarantee transparency, at any moment in time you can cut us out, no problem. We aim to keep te generated code semantically clean, we improve with every itteration.

We are not done yet

Okay, projects are never done, but set asside all nice to have features we could come up with. There are still a bunch of features we used to model Tracepaper but that are not yet available in Tracepaper itself. We used dogfooding to test the Draftsman engine. By generating Tracepaper with our own engine we found a lot of missing features (and naturally bugs). The bugs are fixed, and the features are added to the engine. Unfortunatly not all of them are in Tracepaper yet.

For example, the engine can cope with: input validation, scheduled events, notifier loops, projections (running code @ query time), custom view updaters, Time To Live on data attributes (e.g. for GDPR), aggregate backups to S3, and generating GUI screens. Tracepaper, can not.

So, a lot more is to come! We will announce new features here on Papers by Draftsman. Why did we publish the beta if it is not done yet you ask? Well, the bare minimum is there, and we always promised ourselves to show it to the world once we had a working product. You have to protect yourself against overthinking and over-refining right?

Thank you for reading, open up our tutorial and let's start modeling. Let us know what you think, the likes and the dislikes, all feedback is valuable. Use our forum for feedback and questions.

Cheers,
Bo & Delano

In my last post, I mentioned that we are designing the visual client application. And as it turns out, we both dislike building Single Page Applications (SPA). Yes, "dislike" because "hate"  sounds a bit harsh.
Okay, I'm exaggerating a bit to get my point across. A SPA architecture is a very useful one... But only when needed, and most of the time the benefits don't pay of the added complexity.
And as it turns out, for Tracepaper building a SPA would be overkill. How did we choose a SPA in the first place? Well, we were just dazzled by the fancy GraphQL API in thinking we needed a fancy frontend framework.

How did we come to the conclusion to ditch the SPA? We were building the frontend in vuejs and even though we like this framework it just felt wrong, simply put, it was not fun building the frontend. And in my honest opinion, building software should always be fun, if it is not fun there is something wrong. And most of the time it has to do with Technical Debt.

Technical Debt has the tendency to suck all the fun right out of your development project. In this case, we had encountered some Architectural Debt. We chose a front-end architecture that didn't fit with our vision. Like hammering a square peg into a round hole, you are going to have a bad time.

What to do... We are getting fairly good in code generation, so let's do that. In the past 3 weeks, we expanded our generator and the draftsman modeling language to support what we call "screens & widgets".  What kind of models you ask, we'll get there eventually with the Modeling in Tracepaper series.

On a closing note, you assumed correctly that we don't generate a SPA. Inspired by a post from Aaron Swartz called "Bake, don't fry" and the principles behind the JAMstack methodology we prerender as much as we can and host it from a CDN. It is simple and fast, and that brings the fun back into our front-end design.

At the moment we are designing the visual client application, and now questions arise, how do concepts fit together? Not in a technical sense, but in a conceptual. At Draftsman we use two different models to describe applications, let's sum them up so we are on the same page.

• Conceptual model: This model describes the application and its behavior in ubiquitous therms. This means you will encounter concepts like Command, Behavior, Domain Event, and View. The stuff you could map out with sticky notes during an event storming workshop. This is the kind of modeling we should enable with aid of the Tracepaper GraphQL API and subsequently our user interface.
• Transfer model: an XML representation of the above model with a slight difference, the optimization goal for the conceptual model is to be human-readable or at least human-reasonable. While the Transfer Model is optimized to be machine-readable.

Our transfer model is XML based which we dubbed "Draftsman Modelling Language" it is nearly complete in the context of the MVP due to the fact we use this language to model the Tracepaper backend itself. And to be honest, this way you find the most urgent missing features in a timely manner. Features we probably wouldn't have envisioned if we tried to design it upfront.
Now for the API, we modeled a lot but we are not there yet... we have to figure out what data views we need, the best way to figure out what you need is to explain what you want to achieve. So here we go.

As stated in the introduction we want to optimize our modeling tool to fit with event storming and customer journeys and therefore we condensed our problem space to 5 core concepts.

A command is the starting point of a... let's call it a call stack to keep it simple. A command is considered a fat trigger composed of input fields. The command will trigger behavior in the domain, in this context behavior is the application of business rules on stateful data. The behavior may result in a domain event or also called a change event, the change events are persistent and used to reconstruct the state. You can see that we composed behavior and change-event right next to each other in this image, this is not a coincidence these concepts are strongly coupled, you could have multiple behavior-flows acting on a single event-log that is our implementation of an aggregate. But that is a topic for another post, because, the event is not only persisted it is also published within our domain. And then interesting things can happen, other components could react to the event "if this then that". We have a concept called notifier designed to do exactly that, notify another system, maybe you want to trigger an email using the AWS Simple Email Service, or store a document in AWS Simple Storage Service. A notifier is simply said, an event-to-command converter it invokes APIs from AWS, third party, your on-premise application, or even its own.
Another thing that could happen after an event, is triggering a state change in a view. You may have noticed the use of event sourcing for our persistence strategy, very powerful for storing aggregates, horrible for querying. Therefore we use another pattern called Command Query Responsibility Segragation to mitigate this hurdle. We literally use a different model for our query model, there are other reasons why we use this strict segregation, but this topic deserves a dedicated post.

Knowing the core concepts it is not difficult to see how a GraphQL API correlates to our model. Thanks for reading.

Here is a sneak preview, our generator will convert your model into something like this:

Holidays are over and kids are going back to school, now is a good time to share what we've been up to last month. We did take a break from writing posts to focus on implementing ideas, and that we did.

We came a long way and I'm proud to announce that last week we performed a successful integration test on the Tracepaper API. In a previous post, we presented a component view of what we intended to build. With the MVP in mind, rough estimate, 85% of our API is implemented. The other 15% is fine-tuning, the stuff we postponed during our event storming sessions, you can't focus on everything right?

The biggest challenge for September is implementing a comprehensive front-end that aligns with the customer journeys we envisioned while harnessing the power of our API. Why did we postpone the front end you ask? Well, from a value perspective it is the make-or-break component of our product. To do it well is the biggest challenge we will face, this challenge however is from a UX design perspective. It is a big mental investment and we were not ready to put in the effort before we proved our idea is feasible.  

Therefore, we build a test tool, a fairly simple script that invokes the different API functions to model a simple "Single Command, Single View" application. I bet you can imagine our euphoria when after a mere evening of bug fixing running our test script resulted in a deployed and functional generated serverless application.

So what's next? Dotting the I's and crossing the T's as they say, in our case this will be: Implementing front-end features > Refining Backend > Refining Generator  > Repeat.

And for the next post? I think it is time to write a series about the concepts we use in our Draftsman Modeling Language, stay tuned.

School's out for summer, a great time to step back, relax, and recharge. Leisure time with friends and family and hopefully some sunny weather. So for the next couple of weeks, it will be quiet on Papers by Draftsman.

Draftsman is still a hobby project, which means work is not paused altogether. Work on Tracepaper will continue as is, which is when we feel like it 😉

We did arrive at an exciting milestone in the development of Tracepaper, version 4 of our generator is operational and deployed to the cloud. This means we now can incorporate the whole product in our iterative process instead of two detached islands of technology. Delano and I collaborated during two event-storming workshops in June where we had discussions regarding our domain, customer journeys, and the relevant events around those journeys. Service Design was a key aspect of those sessions and we mapped part of our domain with paper and tape. Which part? Well, the minimal set of functional concepts we need to get the MVP up and running. At the moment we are converting the paper and tape model into Draftsman Modeling Language (XML). A rough estimate, I think we have 65% for the Tracepaper back-end done right now and we are excited to start integrating the visual layer.

You will hear from us in September. For now, have a great summer and stay healthy!

Cheers,
Delano & Bo

We use serverless techniques to handle unpredictable loads while keeping operational costs to a minimum. Nothing in life is free, and we do pay in complexity.

We tackle this complexity by making sure that the technology supports the design and not the other way around, my previous post mentioned we use majestic monolithic thinking to design our applications and merge this with serverless concepts by using an event-driven architecture. Our latest version of the generator uses a monolithic model as input, generates code, packages this code as microservices, and deploys it as function as a service. This approach compresses the conceptual complexity away from our business domain, but there is another disadvantage in event-based systems that needs solving.

Event-based means goodbye call stack! The architect's dream regarding loose coupling may as well be the developer's nightmare to debug. It introduces a lot of uncertainty regarding failure. Can we pinpoint the component that failed? Where are the logs? How do we set up proactive monitoring, and then what? Amazon Web Services (AWS) does provide a managed service to set up monitoring, it is called X-ray. But that is where we drew the line so to speak.

The thing we try to do is quite ambitious, venturing out into another realm of complexity does not sound like a good idea. Truth be told, analytics and monitoring is not our core competence. We decided we didn't want to pay the opportunity cost to learn how to do it ourselves... We "buy" it.

Finding a vendor that sells Observability

Everyone is giving out free samples, and that is how our quest began. The first platform we tried was bluematador.com, a good solution but not the right fit for us. Not a good fit for the same reason that Data Dog didn't feel right, the "problem" with these first two solutions is that you could measure practically anything you want, especially with Data Dog. Very impressive stuff, the thing is, more bells and whistles mean you have a steeper learning curve. This is fine if you're into big data analyses. But adding an extra bubble of complexity is something we want to avoid, we are looking for intuitive distributed tracing with minimal setup. Intuitive is a broad spectrum, so let me clarify. We are looking for a product that delivers distributed tracing for serverless applications on AWS that has minimal setup and is intuitive to use for two developers with a Java background in a Service-Oriented environment. So, we are looking for something more opinionated, and that is when we landed on Dashbird.io and Lumigo.io.

Both products are easy to set up and use, but we felt that Lumigo was more opinionated in how you should build a serverless application on AWS, which drew us over.

We are happy with our choice and found some nasty bugs in our code with the provided tooling, and even re-architected our GraphQL API. Are we ever going to use AWS's X-Ray? Well, I don't know, but for the foreseeable future, Lumigo is our monitor.

As I sign off for this weeks blogpost, I will leave you with some screenshots from Lumigo so you can see why we are so enthusiastic about this tool. Until next time.

Many companies, and even individuals, send us many emails. Some of them because we are a customer and others because we have subscribed to their newsletter. Either way, most of these companies are trying to contact us with new information, or even exclusive newsletter deals, convert us to buy their products or services time and time again. We might even be genuinely interested in what these companies or individuals have to say. But with that comes unwanted invasion of privacy through tracking their readers.

There aren't many companies out there who don't (ab)use the usage of e-mail tracking. Potentially every e-mail you receive, that's not directly from an individual you know, has a high chance of containing some form of e-mail tracking. The tracking of e-mails can go so far that these companies not only know you have opened an e-mail. But also how long you've had it open, what links you clicked or how you generally interact with their e-mails. Their smart tracking systems can tell them about what smartphone or laptop you have, at what time you read their e-mail and even how often you check back on that e-mail they sent you. Haven't read their latest additions to your inbox? Don't worry, they won't mind reminding you every once in a while that you haven't opened their most recent e-mails if you have decided to take a little break.

Spy Pixels are slowly invading our e-mail box. And with that, these companies are invading our privacy. Most of us don't even realize these spy pixels exist or the amount of information they reveal about you. This has to stop.

Ever since Hey.com has been released in 2020, with blocking e-mail Spy Pixels as one of its main features, more and more companies are deciding to let go of their e-mail tracking. Even Apple has recently announced that they will be blocking e-mail tracking with the default Mail app starting with their next software release in the fall. This is a huge development when it comes to privacy as now Spy Pixels and e-mail tracking will finally have a chance to become obsolete.

At Draftsman.io we are very passionate about doing right by our readers and, in the future, our users. We value transparency in what we do and what we do with your information. From the start one of our drivers was to generate beautiful serverless applications based on your models, where the generated code is readable and fully transparent. You own the code we generated, wether you decide to stay with us or leave us. When we started our blog Papers by Draftsman.io, we wanted to have the same transperancy. And since this e-mail tracking endemic recently surfaced again in a much larger way than ever before, we decided that we would never include any tracking in our e-mails starting from day one.

Obviously we are still looking to grow our audience, and we would love to welcome you onto our newsletter. For us, this means that we use a specific tool to give us anonymous insight into how our blog is doing. Plausible.io gives us insight in the amount of unique visitors we have had and how often certain pages have been looked up. We also see how this visitor found their way to our blog, maybe through our newsletter, a search engine or a direct link. All we see is anonymous information. We don't see any information about our individuals viewers, and we have no idea who you are. The web is entirely free, as it should be. And for us, well we can't think of anything better than letting our readers decide if and when they wanna read. We don't need to be in the loop via tracking.

So, we don't know if you have read our posts. And we're totally fine with it. Of course, we would love to hear from you! If you have any questions or feedback, don't hesitate to shoot us a message at papers@draftsman.io.

For this particular post, I would like to take you back about one year and three months. It was the beginning of Spring 2020, right before the pandemic hit, and Bo started sharing his new idea and vision for complex transactional serverless applications. We went back and forth for a few weeks discussing his ideas. Some elements that always came back were modeling our software (Model Driven Design) so that a business analyst could create a complex application, by just clicking together what they needed. Naturally, we focused on Domain Driven Design (DDD) since it allows us to focus on the context of the business and in the language the business analysts, domain experts and developers already know.

During these sessions we started to notice that the work we do as software engineers and architects, especially when basing our work on Model Driven Development, had a lot of similarities with the work of an Architect. When we look at the construction of a building, the architect is responsible for creating a solid foundation to build upon and structural integrity, as well as designing the building in such a way that it appeals to its audience. The architect hands his designs off to a contractor who will then begin the building process. In software engineering, specifically at Draftsman, we are trying to do the same. We draw up the designs and architecture of a system, using tools such as DDD, and based on these drawings we generate a fully functioning serverless application. The business analysts, also known as domain experts, design the system (draw) and the Draftsman Engine will turn those designs into high quality functional code.

This analogy with Architects became a theme, so much so that we decided to look in this realm for a good name for our project. Through many searches in OneLook related to architects and the tools they use we landed on some promising names. One of these was Draftsman. A draftsman is, according to the definition on MacMillen, someone whose job it is to draw the plans for something that someone is going to make or build. In other sources a draftsman is considered a master builder, a craftsman or even an architect. We just knew that when we landed on this name that this was the one. I'm sure you can imagine the excitement we felt when Draftsman.io was still available.

We got the name of our project covered and we're very happy with it. But we also have a designer studio that we wanted to build. Like I mentioned earlier, we also searched for tools in an architects toolkit. We came across many different tools, such as a lead holder and tracepaper. Tracepaper is the paper that is used by architects, or draftsmans, to draw their technical designs on. And, if you're catching our drift, it seemed like the perfect fit to us. Without hesitation it just clicked and from that moment on our designer studio was labeled Tracepaper.

What made these two names stand-out to us? Well for one, in Search Engine Optimization it is recommended that your brand name is a unique one-word word. Which narrowed down our options significantly. And from there, we just fell in love with the relation to Architects. On top of that, these are great brand names to carry the message we want to share with the world. Even Papers by Draftsman is a carefully chosen name that fits inside this little ecosystem we created so far.

I hope you enjoyed this different take on our posts this week. For now we are going to continue working on our dream to build the most accessible designer studio that is capable of designing world-class transactional software, in the cloud. Until next time.

In a previous post, I mentioned we are over the hill, the complex technical preparations are completed. We started designing our application. Designing what exactly? We don't have a roadmap after all.

Our mission is, enabling business-minded people to model performant cloud-native applications while reducing operational burden. We selected Model-Driven Development to separate design from implementation details.

In Tracepaper (our design studio) we promote monolithic thinking because from a business perspective the system has to be viewed as a whole otherwise it would lose its fundamental properties.

A system is a whole that cannot be divided into independent parts. The defining properties of any system are properties of the whole, which none of its parts have.

The essential property of an automobile is that it can carry you from one place to another. No part of an automobile can do that.
The wheel can’t, the axle can’t, the seat can’t, the motor can’t.
The motor can’t even carry itself from one place to another.

You can write; your hand can’t write. That’s easy to demonstrate: cut it off, put it on a table, and watch what it does… nothing!

And, therefore: when a system is taken apart, it loses its essential properties. - Russ Ackoff

For the performant cloud-native application part we chose to use serverless concepts and selected AWS as a vendor to do so. This means that for now at least Tracepaper will only support AWS... sorry folks.

Stitching together monolithic design and serverless implementation means we have to look for a common denominator between those two perspectives. The answer is events. This means we have to treat events as first-class citizens and subsequently use event-driven architecture. I can not tell you how we visually support this in Tracepaper because this is one of the things we are designing at this moment. But we will address this in-depth once we have succeeded.

The important part is, from the business perspective we have a model. And with the aid of a generator, we convert this model into deployable code.

One of our core values is transparency. You may choose to use Tracepaper for your next back-end application, and although we generate code for you, it is your code. We built a modeling studio and a generator, that is ours. But the code that this generator spits out reflects your thoughts. And your thoughts are certainly not our property.

Furthermoe, running your own code in a managed serverless platform is an agreement between you and the cloud provider. You should be reticent to whom you give insight into your applications at runtime. A need to know basis as you will, and we at Draftsman just don't need to know, and thus we won't.

We do trust another company to help us monitor our serverless applications, but I will discuss why we selected lumigo.io in another post.

To sum up, we create an application to manage models that reflect your business domain, this is our design studio Tracepaper. We need a component that converts said models into code. And we want to give our clients full ownership over code and operations.

Component view and ownership perspective

In the high-level design depicted below, you can see a clear distinction between our cloud environment and the client cloud environment. The commit from the generator to GitHub is the handover in ownership.

This does mean, that you will need an AWS and GitHub account besides your Draftsman account. We are aware this may be an extra hurdle for using our platform but we accepted this drawback. Why? because it supports an excellent exit strategy. If at any point in time you decide you don't like Draftsman anymore, you can cut us out, no problem. The code is yours, in a repository that you own and running on machinery that you rent. We strive to make the generated code readable and as maintainable as possible, what you do with it after that, that's up to you.

What we are working on

At the moment we are designing the back-end for Tracepaper, we do this in the XML-based modeling language we invented as input for our generator. A very useful process, we already added some missing features in language and generator. We aim to conclude this stage by the end of this month. The next step is for Delano to integrate the Tracepaper front-end using the prepared building blocks. Meanwhile, I will focus on integrating the last iteration of our generator into the Draftsman Serverless landscape and start working on automation to aid the client in attaching Draftsman, GitHub and AWS together.