This post is an expanded version of a part that is going to be presented at the Enterprise Architecture Conference Europe 2014.
The datalogical world – the quest for data, information and applications – Application Architecture
The classic view of the information domain is driven towards the persistence layer, where the data is stored in databases and servers. This is a standard that everybody is used to look at in many concepts and models. Application Architecture, under Business Architecture, which is told to support operations, followed by Information architecture and at the bottom, the Infrastructure layer.
This approach can also be found in TOGAF, when we enter in phase C, information system architecture:
The design starts with the business architecture and then progresses to data (or application) architecture, then application (or data) architecture, before finally designing the technology architecture
When we reread the last sentence, what is data and what is application? Seems it is a interoperable concept when it’s not. Often said, that applications support operations and applications are supported by data, this is probably the reason applications are flushed not taking into consideration what kind of information is necessary to be processed by actors that intervene in process execution. Data is function of execution, of the experiences that are being lived. A customer’s order appears in (not a predefined) moment, because the customer expressed to buy something, nor by the fact that someone in the company predefined that there is a perfect moment for that (probably when that moment was conceived, the customer already bought something). To buy something, to create a new insurance policy, to open a new bank account is an act of will. Hence data appears first and applications latter.
What is interesting, is the fact that in TOGAF, after the semantic confusion of what is data and what is application, it’s states first:
The objective of the Data Architecture in Phase C is to define the types and sources of data needed to support the business, in a way that can be understood by stakeholders. The output should be complete, consistent, and stable
The objective of this phase [application architecture] is to define the kinds of application systems necessary to process the data and support the business
For concluding this short introduction into the information space, data appears first, as a result of operation, the kind of applications after (not the other way around), despite being modelled contrary to this approach. After the first interactions, when we enter into the details of expressing how data is structured in class diagrams and other type of abstract representations (which details are out of the scope of this article) there will be recursive feedback brought by applications that are taking shape driven how they handle particular data entities, hence, there will be some bias about the shape of data to come, but for the objective of defining what kind of applications an organization should have, forget about the miracle an application can foster, applications should be a mirror about the kind of experience the stakeholders want to feel.
As referred before, the operation of an enterprise is constituted by activities which are elementary chunks of authority, competence and responsibility fulfilled by actors (humans, machines, systems). The kind of activities performed, are according to the social involvement in the enterprise. A Call Centre Operator will most of its day answering phone calls, one after the other, providing solutions to customers, but it can also be involved in forums with peers to explore other possibilities how to best serve the customers, as also, try to recover angry or frustrated customers. A Call Centre Manager will spend most of its time monitoring if the service levels agreements are being accomplished, preventing bottle necks through levelling the workforce, trying that knowledge it’s externalized to best serve the customers and so forth.
Again, it’s worth to revisit these three pillars to execute an activity :
- Authority: The actor can perform a particular activity on behalf of an organization in a process. Authority can be delegated.
- Competence: The actor has the necessary knowledge to perform the role ? If it is a Call centre Representative, it have the knowledge about the products and services that he is providing support to the customers?
- Responsibility: By virtue of the values, culture and policies of the organization one represents, is expected to exert the granted authority in a responsible way.
This is under Viable System 3, that is considered to define as the act of management. The Call Centre Operator is responsible to manage its own job with the authority and autonomy that was defined by up in the hierarchy (taking calls, provide solutions, escalate to experts when it cannot help the customer).
Theoretically, all the acts produced across the organization are recorded (despite this is somewhat far from the truth). Hence data (that constitutes the information architecture) is the mirror of what occurs during process execution (independently if it’s automated, normalized, ad-hoc, adaptive, etc). Today, we are headed towards the digitalization of all data as a consequence of the new norm that is, organizations are becoming digital. Hence, the data explosion phenomena, despite, there is still data on paper as also data that it is deleted.
Using that amalgamation of data, actors should have the opportunity to infer, reason, make decisions, the expression of our intellect, that is the reason of being the “Information space” – (how knowledge is diffused) that ultimately supports the very existence of the corporation.
The information layer, called by Dietz  the datalogical world, is the “fossil” of what occurs in the organization as a result of coordination acts or business operations. Information is the fuel of business processes; their flow generates value to the user. Transform data into information, that is transformed into knowledge, allowing actors to make decisions and have a concept of what the organization is and the actor belongs to.
Analysing the nature of coordination acts (in other words, how processes are executed) will allow us to identify the informational entities that will build the master data dictionary and will shape the application landscape. It will define what kind of data is necessary for each actor to create and consume during activity execution. These are the information centric components of an organization and define its relationships regarding business process orchestration. This way, we continue to assure alignment between each of the layered enterprise architecture and more than that, describes the principles and guidelines that enable consistent implementation of technology applications, that will be then expanded thought its own development cycles; how data and information are both governed and shared across the enterprise and what needs to be done to gain relevant and trusted business insight.
An Informational Entity is:
Any person, place, concept, thing, or event in the context of the business, about which information is necessary to keep the informational attributes of the entity itself. For example, the informational entity Customer has the attributes: “Number”, “Name”, etc.
Hence, by identifying the informational entities (that can also be redefined as business entities or as Archimate puts it, a Business Object) we are defining what kind of information, actors (humans, machines, systems) will need to access to accomplish the job they have been authorised to by the use of enterprise applications that process such kind of information intake. For example: an invoice is used when making payments, but also when analysing the customer orders history. The invoice will be also used in multiples applications like Billing, Accounting, Recommendation Engines (by retrieving the products that belong to the lines of each invoice in order to cross sell). Ultimately, allows to define at the logical level, the kind of relationships that exist between data and can be externalised if the architect wish, depicted in entity relationship diagrams or others. I am not going to explore the logical dimension here, the scope is focused how to create the enterprise application landscape that belongs to the Application Architecture.
Privacy and security issues should also be analysed. In the context of the example we’ve been working, (the Engineering council) access to member’s certificate, proof of qualifications, should be made accessible to the member and to those that need to make a decision about attributing a new skill once a training was completed (the dipole Authority, Responsibility continues to valid for this purpose).
Building Information Architecture – identification of informational entities
The process to identify informational entities is bidirectional. Top-bottom, looking to what is the intake during process execution, and bottom-up, looking to the existing applications identifying what is attached to it. Luckily in this example we are working on the top of blank sheet of paper, meaning that most of the existing applications will be erased and we do not need to concentrate in the bottom-up approach. Still, a word of caution: neglecting what the current applications do regarding creating and updating informational entities, may cause extreme conflicts in data governance, data quality and confidence by the actors when accessing data. For example: a Controller cannot expect that a Customer balance report includes invoices are on due, but the billing application already processed payment.
The process is pretty straightforward. Start looking to each process that belongs to the value chain in a high level terms (don’t try to detail it, don’t go for BPMN diagrams, at maximum, keep it decomposed into high level steps or sub processes as you like), identify the main activities executed and question what informational entities actors should create, consume to succeed achieving the process results. Identify if during executing process, the informational entity is Created, Read, Updated or Deleted. The CRUD operations should be not seen as database operations but as Application Services that the an Actor invoke (that lately can be used to create Application Cooperation viewpoints as described in the Archimate standard).
You should have as a result of analysing a business process, like Admission and Qualification of Members, something like this picture. If you are an Archimate practitioner a Business Process View could be handy (the informational entities are painted in blue, because I don’t like everything in yellow):
This way it is possible to create a puzzle that intersects the processes against the informational entities. The first exercise will result in a very high cluttered board as bellow.
The end result through the application of a set of heuristics will create application clusters that will transform into the application architecture.
The approach can also be used in data centric processes (ad-hoc, dynamical, adaptive) instead of thinking about the steps (than can be any order, think of the informational entities that are necessary to use).
The heuristics, derived and adapted from  are:
The main heuristics to consider when checking alignment between Business Process and Application Architectures are:
- Each business process should be supported by a minimum number of applications. This simplifies user interfaces among applications, reduces the need for application integration, and also minimizes the number of applications that must be modified when the business process changes. But it is very unlikely that there will be an holistic application to support the value chain or a wide group of business processes.
- Each application’s service should support at least one business process activity. Otherwise, it plays no role in supporting the business.
The main heuristics to check alignment between Application and Information Architectures are:
- An information entity is managed by only one application. This means that entities are created by a single application.
- Information entities are created when identifiers are assigned to them, even if at that time no attributes are known. For example, if the Client information entity may be created before its name and address and other attributes are known. Even so, the application that manages Client information entity must be the application that manages their IDs.
- Exporting and distributing information entities across organization applications should be done under a interoperability paradigm, service oriented approach, rather than a point-to-point Application integration.
Finally the main heuristics to apply for Business and Information alignment are:
- All business processes activity create, update and/or delete at least one information entity.
- All information entities attributes are read at least by one business process activity.
- All information entities must have a mean of being transformed for presentation to appropriate stakeholders using enterprise-standard applications and tools.
After the cleanse as been done you should arrive to something like this:
This is the starting point to discover the applications that will support business processes.
As a result, this provides the following application landscape.
The end result through the application of a set of heuristics will create application clusters.
The application architecture have what I would call very straightforward, most of applications are very clear taking into consideration its purpose, process oriented by bicephalous BPMS / CMS running on top of the secondary “line of business” applications. One I would like to point out that its particular important is the WIKI.
The observatory of Engineering, a service that the Engineering council wanted to provide . That observatory had as main objective, understand the future trends in engineering and the impact that it will have in the future of engineering at large and particularly in the country, like for example what kind of renewable energy sources should the country adopt and other themes like smart grids, more or less like a think tank. This pure Viable system Model system four, but working on reverse, because the results direction were pointed to the society. The wiki that the critical mission of dealing with engineering variety. Here is a good example about how a viable system can deal with variety
Let me bring this scenario: how to understand the impact of drone use in agriculture? Questions rose like:
- How to operate in order to make effective crops;
- What about data collection, like images that rise privacy issues?
- And what about licensing to fly?
How to study this challenges and the corresponding society impact? How we can help agriculture?
The wiki have three main missions:
- Support Task forces: “everyday”, it is necessary to create a task force on a specific topic. It can be around creating standards or acting as a liaison with standards agencies, understanding the changes of legislation and alike and its impact, it can be a focus group that develops new engineering methodologies it can be applied on everyday engineering, it can be a collaborative environment about the thinking of engineering and it is applied for example in schools. This is truly necessary to support the observatory of engineering.
- Knowledge Repository: the externalization of tacit knowledge in order to become explicit and help members to find answers that are not documented. For example lawyers like to write books about the interpretation of the law to help other professionals to reason an make decisions on that, but that is not practice in Portugal due to the fact that there is no publishing industry that supports that outcome.
- Experts discovery. When it is necessary to find someone that participated in a project / task force or have a particular skill in a domain expertise is critical to make invitations or interact in order to resolve particular matter. Constructing a social network on top of an ontology is critical to put the people working to make the best teams.
 – Pedro Sousa – Enterprise Architecture Alignment Heuristics – http://msdn.microsoft.com/en-us/library/aa480042.aspx
 – Enterprise Ontology – Jan L.G. Dietz – Springer – ISBN-10 3-540-29169-5