Process Mining must embrace complex adaptive systems

On April this year I wrote a post about one of the challenges that Process Mining is not addressing properly: Big data. In the last process mining task force meeting, resulted that one the biggest concerns was about the effort of making XES a “de facto” standard in order event logs used in process mining could be system interoperable.

Despite the fact is important to assure that data can be exchanged across multiple systems and they can be interpreted the same manner (contrary what happens with bpmn diagrams), I tend to think that process mining is loosing ground regarding the near future reality:

  • Technology will be like electricity or if you prefer a commodity;
  • Cybernetics principles will emerge (finally) because society and technology is backing humans able to observe, sense, act, learn and adapt. Once organizations are made from humans, and despite humans belong to some kind of organizational unit, they are not anymore animals on a tree branch and are part of social networks that constitute the enterprise where everything happens.
  • Technology no longer shapes the way human perform. People do it. Change and adaptation encompasses clearly to think and find new business models. To think about innovation. If companies do not have humans capable to find access, understand and process information, how can companies evolve and survive?

Again what is the challenge of process mining?

Organizations live in a world where interdependence, self-organization and emergence are factors for agility, adaptability and flexibility plunged into networks. Software-based information systems go into a service oriented architecture direction and the same goes to Infrastructures where services are become structures available in networks. inspired into empirical studies of networked systems such as Internet, social networks, and biological networks, researchers have in recent years developed a variety of techniques and models to help us structurally understand or predict the behavior of these systems. Those findings are characterized by been supported on the “complex networks” concepts [1].

Last week I stumbled into a great find based on a conversation about creating an Adaptive Enterprise Architecture framework to support business processes  that Process Sphere is sponsoring, in order to apply in real world implementations. Universidade do Minho is leading research in the field of understanding existing information systems can be characterized adopting the complex adaptive systems principles.

The approach is quite obvious (and at the same time so simple). Is about a black box (supplied by Palo Alto networks) connected to the enterprise network (the pipes where information flow, where today everything flows) and literally sucks all the information from the pipe to be analyzed and interpreted.  With this approach its possible to understand Who interacts an with Whom interacts, What information is accessed and processed, What and When, interaction occurs, What are the information services requested and (in my point of view) particularly important evaluating how information systems can cope with social iteration and can keep the pace of adaptability inside of organizations (something I’m keen regarding other project about requirements engineering using process mining).

The innovation of the approach is based into:

  • Analysis is carried based on complex network principles;
  • Contrary of one of process mining principles, that the staring point is to select data to make the analysis in order to prevent “data collection paranoia”, this project collects and analyses ALL the available data and digs into the networks that were formed (each network have a different analysis dimension depending on the field of analysis). Layered.

On one hand the method approach to interpret data it’s arguable among the scientific community that the one here presented is inferior of others used by other researchers that develop process mining methods available in tools (researchers tend to think they are above God), thus I’m not going to involve in that kind of discussion (by the way this one comes from physics), on the other hand, what I like most is the design approach. Putting a box so suck all the data to be explored (into the company veins) what could be called inline real time process mining to deal with complexity and adaptability.

References: [1] Modeling Organizational Information System Architecture Using “Complex Networks” Concepts, José L.R. Sousa, Ricardo J. Machado, J.F.F. Mendes, 2012 Eighth International Conference on the Quality of Information and Communications Technology


A new challenge for Process Mining – Big Data

Process Mining is transforming the way we understand the reality of organization’s interactions and helping to accelerate adaptation efforts. When people contact for the first time how Process Mining operates, discovering automatically process models, showing analysis dimensions accurately, jumping immediately to the act of change and adapt the process they realize the power of process mining.

Here comes Social and Big Data is attached

Social business is transforming the way we work. Internally it unlocks silos. Outside puts the customer in control of the business processes. This shift brings additional challenges regarding how we manage communication channels and drive process execution. Additionally it introduces an exponential increase of the amount of data available, some or most of it, is not interpreted by the organization. Imagine your company invested in a Business Intelligence mobile solution that does not support Blackberry devices and your customer base uses this device type . Basically your BB customers don’t exist.

This exponential data increase is in different forms and flavors: classic relational / transactional data, documents, blog posts, tweets, Facebook posts, video, podcasts, images. The last five data types are used widely in social interactions. You can post a photo on Facebook, with content that relates with your company on a particular complain management scenario. TAP airline handles complains/information requests on Facebook. Sometimes customers upload photos taken with mobile phones showing a copy of the boarding pass where the airline can get data to help the customer. Football clubs get data from fan’s seats, tweets, facebook’s posts, merchandise that was bought in the fan’s shop, to measure fan’s engagement and cross sell services or football’s club partners products or services. All this data blend is called Big Data.

The emergence of Big Data is putting a lot of analytical pressure. This kind of data is dispersed is not inside of your company systems like the products your customers buy from you, (it’s stored inside Twitter’s servers or similar) making difficult to relate and analyze (natural language interpretation capabilities for example).

Event logs challenge

As we are headed towards more and more to the socialization of business processes, this means there is leak in the foundation of process mining: event logs. Event logs are data about the nature of your business process, meaning, what was done, when it was done, who did it and extra meaningful data necessary to replay process reality. Events logs are extracted from the company systems or from the companies systems if the process being analyzed is executed by more than one company (imagine a bank that processes some activities to issue a credit card and other part of the process is executed by the credit card manufacturer and others by the a logistic company that delivers the credit card for you).

Process Mining relies on the prerequisite that you have access and control on the data source. But with the socialization of business processes you loose that ability. You are inside a dark hole.

As someone was saying in a enterprise 2.0 conference, Facebook is a threat to the internet: information search and discovery due it’s protection / privacy mechanisms meaning you don’t have access to data. Keep in mind that you have customers talking about your company and you are not able to engage with, but you still have a customer interaction process, but is broken. If you made a reset how processes where analyzed before (running inside company borders) when a customer submit a complain and the complain has not handled it was considered as “in progress” state. Today, if you have a customer complaining using social media you still have a “in progress” process instance, but unfortunately the data is not stored in your company system, thus it cannot be recorded and it cannot be mined.  If you take a snapshot of the processes with events logs with data that is missing, how can you analyze reality?

Processing power is always increasing and this was one of the factors that enabled hyperactivity and new information systems, but one thing is processing an event log with 800.000  lines with standard data, other is processing the very same 800.000 lines with documents and video because the enterprise system that is providing support to process execution allows you to use video chat and inside that video chat is import information you need to analyze.

Have you say?