1. Peter Schooff
  2. Sherlock Holmes
  3. BPM Discussions
  4. Tuesday, 12 July 2016
  5.  Subscribe via email

Another point that came up at the BPM & Case Management Summit: Do you think data has not been prioritized enough in BPM?

Patrick Lujan
Blog Writer
Accepted Answer Pending Moderation

It's like security, should be accounted for from the ground up in any implementation, but most either unnecessarily concentrate on it solely or too much (as opposed to incorporating into the overall picture) or don't account for it at all until it's too late. But data, particularly in regards to integration, will make or break you if not accounted for.

There's a reason why data and process are the first two columns in Zachman.

Just my tuppence.
what i find interesting is that BPM-focused folks (who we run into a lot) tend to de-emphasize data too much. People with no bpm background tend to try to define everything about data before discussing process. The trick is that data and process provide context to each other so you need to start somewhere and cycle back and forth keeping flexible about what the final data/process definitions will be.
  1. more than a month ago
  2. BPM Discussions
  3. # 1
Accepted Answer Pending Moderation

In the old days of "boxes and arrows" that was probably the case, I think. But as more and more processes are turning data into data (of more value, hopefully) it has become important of course.

When talking with customers about processes I prefer starting from the end, which can still be a physical product, but also a collection of data. Of course always to make clear if that collection of data or product really solves a customers problem and if a process is really needed for it.

I think it is getting more and more prioritized (or should), but don't forget that, when you talk about BPM, there is data on more levels:

- Execution of a Case level

- Monitoring of cases level

- Process (improvement) level

- Data that flows between processes

Wrote some words about that [url=http://procesje.blogspot.nl/2016/05/does-new-gold-work-for-your-processes.html]here[/url]

Sharing my adventures in Process World via Procesje.nl
  1. more than a month ago
  2. BPM Discussions
  3. # 2
Accepted Answer Pending Moderation
In some cases, the heart of the process is a complex data object with it's own lifecycle phases, rules, etc. In these types of processes, the data object's needs establish the boundaries of what process optimizations are possible - requiring a data-centric approach to BPM.
  1. http://www.bpmnext.com/bpmnext-2013-presentations/data-centric-bpm/
Founder at John Reynolds' Venture LLC - Creator of ¿?Trules™ for drama free decisions
True story, first class with the current #1 Fartner quadrant for BPM vendors back in 2009, instructor's purveying the Kool-Aid saying "you've never seen another animal like this, it's totally new," then he starts talking about work objects. ;)
lol Patrick. You've never seen this before... THE SPOON! :)
  1. more than a month ago
  2. BPM Discussions
  3. # 3
Accepted Answer Pending Moderation

Actually... Data is incredibly important.

However and IMO the issue is more: Data in
. I see many times efforts where there is so much focus on data (only), that people completely forget in which wood they're cutting the trees... In other words: Scoping, the big (system thinking, holistic etc.) picture is missing. So we end up with not doing things inefficient, but especially with
[i]ineffectiveness [/i]
(not doing the right things). Therefore, I believe it needs priority but with the effectiveness in mind...

Right, what data is needed at a given point in time, for what purpose, within what context. That be the process part.
Sure . . . too much data decreases effectiveness.

Organizations struggle with time/money saved by collecting less data only to find out, later, that it costs them a lot of money because of failure to collect data they figured was not, at the time, going to be needed in the future. (i.e. you cannot analyze data you did not collect).
nailed it. Data in context (or providing context) to the process. I see those data only efforts too - lots of wasted effort. And then very rigid data definitions... and then the process is defined and you realize the data model is wrong. constraints that can't be there, data that won't be known for a long time yet but data model assumes it is the index column, etc :)
  1. more than a month ago
  2. BPM Discussions
  3. # 4
Accepted Answer Pending Moderation

You can buy a car and spend all of your time reading the brochure or you can read the brochure and then get in the car and drive.

Clearly a high level concept-only process map can be posted to a wall/screen for people to look at/comment but once you have a map with multiple parallel sub-pathways and branching decision boxes, some of which are likely to have auto-execute rule sets, you quickly get to where modeling in a run-time environment, with data, is necessary to test the logic and the branching.

If some of your process steps are algorithms that take data and enrich that data, unless you piano-play your process you will never easily know if you are collecting, upstream, the data the algorithm needs - you won't be able to validate the algorithm.

Once you are in production mode, any step that requires more than a "done" declaration will need a form at the step to record observations/measurements and oftentimes you will need/want to attach documents to steps.

Are there any BPMs out there that only maintain a "log" ? (i.e. the system appends a record of the step name, date/timestamp, a user "signature", but no data beyond this).
  1. more than a month ago
  2. BPM Discussions
  3. # 5
Accepted Answer Pending Moderation

Nobody's going to suggest, I imagine, that data is unimportant (nor, for that matter, that data

That said, BPM isn't about trading C for SQL. If you have to understand third normal form and the difference between "GROUP BY" and "ORDER BY" in order to implement your application, you may not be taking advantage of the power that BPM technology offers.

How many product demos have I seen that begin with,
[i]start by using our simple data model generator[/i]
... Gee, thanks for the wafer-thin veneer on SQL Server Management Studio, but is that really why I invested in a low-code development platform? I'd venture not.

Scott's opinions only. Logo provided for identification purposes only.
There is a huge difference in terms of the data management needs of ordinary users running instances of templates at aCase versus a Data Mining specialist trying to detect run-time patterns across multiple Cases after the fact (i.e looking to see the frequency of steps being skipped by users, the frequency of ad hoc insertions by users).

Better of course to eliminate the Data Mining specialist and have software do the analysis (some of us are not there yet).

I think it would be nice to be able to overlay a plan-side process map with run-time variations, with declarations along the lines of:

1. this step was skipped x% of the time across this subset of users.
2. this step was inserted y% of the time across the subset of users.

As for implementers of applications where BPM is central, the customers we work with only have to deposit nodes on a canvas, link them with directional arrows, assign routings to steps and reference data collection forms at nodes.

The forms they park at nodes are designed via a drag and drop process, there is no access given or needed to database tables/fields (implementers never have to add/edit database tables).

The problem lies with interfacing disparate systems and applications where each system may have data import/export facilities but expects specific data element naming conventions and message formats or data transport format.

Each set of trading partners needs to write parsers/formatters until they tire of requiring custom formats and agree on some standard.
Well, it's true that we didn't zoom in much to what type of "data" it is to which the question applies.

I'd agree in general it can be hard to pull out ex post facto data for process owners to use in reviewing and improving their processes. (Our product does a good job of that but if you want to know more you'll have to email me. :))

I was looking at it from an implementer's standpoint. I want to dive right in and build moving parts. I don't want to have to learn SQL or understand DDL in order to do so.
  1. more than a month ago
  2. BPM Discussions
  3. # 6
Accepted Answer Pending Moderation

Even if we all agree that "data is important", still data is never "prioritized enough" because
[u]data is infrastructure and the business case[/u]
for infrastructure is always a challenge.

Data itself is the
[b]alphabet of business semantics[/b]
which are the content of our business systems.
[u]Without this alphabet, we cannot "say" the things we want to say[/u]
. And that includes the statements we want to "say in BPM". If we haven't defined that data elements that can be used to
[u]define a field service return visit[/u]
then we won't build a return visit very easily in our BPM field service application. (A complete inventory or ontology of
[u]irreduceable business semantics[/u]
starts with data, but also includes rules and process.)

But the big challenge around data is not technology, or even governance, but it's an
[b]economic challenge[/b]

Cheap sensors and
[u]cheap data combined with expensive business analysis inevitably lead to information overload[/u]
(or alarm fatigue). We end up drowning our users and
[u]overwhelming their decision capacity[/u]
because we didn't do the hard work defining the states, interactions and life-cycles of our assets, processes and rules. (And "hard work" translates into "budgeted for business analysis".)

So to build an application, you need to start with data and data models and then the whole business semantics kit.
[u]Will you build that kit yourself[/u]
? Or buy it as part of a package? Maybe with some after-the-fact customization? Can you afford to maintain and evolve it to support your evolving business model?

It's something to consider that
[u]business semantics[/u]
, starting with data, 
[u]should be an explicit concern[/u]
of executives responsible for business automation outcomes.
if data isn't prioritized enough you haven't met some of the IT shops I have :)
  1. John Morris
  2. 4 years ago
  3. #1928
Scott -- I can imagine some of the shops you've dealt with, although I suspect that the ratio of data analysts as opposed to DBA's doing data modeling has been evolving downwards for a long time.

But to acknowledge your point, I should have said "usually not" rather than "never". Your earlier point about "data and process together" is well said. On this basis we can begin to address "business semantics", comprising the language of work automation as the outcome we need.
  1. more than a month ago
  2. BPM Discussions
  3. # 7
Accepted Answer Pending Moderation

And this is done for a very good reason. The primary job of BPM is coordination – grab data from some places, make new ones (with some help from some people) and deposit them in the same & other places. And save the audit trail records, preferable in blockchain, before removing them from BPM. Like a perfect secretary or coordinator – light, scalable, agile, completely data-driven (thus transparent), and, of course, with some predictive analytics, if accepted.


  1. more than a month ago
  2. BPM Discussions
  3. # 8
Accepted Answer Pending Moderation

Yes but the question should put emphasis on quality of data......? Frankly without data outcomes and presentation in any shape or form there is no BPM. The emphasis should be on creation of one version of the truth from source and no duplication? In order to give that vital assurance to both business colleagues and external interested parties such as compliance audit etc. Such transparency on how data is created and used is vital. So yes this data message needs to be emphasized which shoud be one of the priorities in BPM thinking.
  1. more than a month ago
  2. BPM Discussions
  3. # 9
Accepted Answer Pending Moderation

BPMN has clearly excluded data from its scope. This may be technically correct, but it's still wrong for the business.

For business people, flows are easier to understand than states. But it is actually the states that need most architecting. The data model is extremely tricky to get right (and to scale right). And that is because, from an architectural standpoint, a database is one of the most serious (and dangerous) technological coupling points (since it's about consistency and integrity). Get it wrong and you're cooked for a long time.

So a proper working business solution will always have to include both data and processes. In a language, you cannot speak only in
[b]verbs and adverbs[/b]
(actions and rules in flows), you also need
[b]nouns and adjectives[/b]
(data objects and their attributes). And to really drive your point home you need
as well (UI).

Maybe that is one of the reasons of relatively low-traction of BPM technologies - it is never a pure workflow play. Never. Having data as second citizen of process tech was, strategy-wise, wrong for the process standards.

And it doesn't show many signs of improvement, as an industry. Look at all the cool no-code kids how far away they are from even mentioning data in their cute demos. Also, CMMN and DMN (much more persistent-data-centric than BPMN) are still making no inroads into proper business data modelling.
CEO, Co-founder, Profluo
Good points..

The way out IMO is to host BPM in Case. It takes nothing away from BPM. Case + BPM is better than either on their own.

Case puts a primary focus on data/decision support for meeting Case objectives.

You load process templates/process fragments into Cases, the templates provide background guidance, the Case environment provides governance (rules, global to the case) allowing Case to accommodate deviations away from BPM logic (skip steps, insert steps, re-visit steps, record data at not-yet-current steps)

One step up from Case you have workload management (i.e. because resources are typically shared across many Cases). There is a huge difference between workflow management and workload management.

Once you put in place policy that says the organization undertakes no work that does not contribute either directly or indirectly to strategy, you are well along the road to bridging the gap between operations and strategy.
  1. John Morris
  2. 4 years ago
  3. #1915
Bogdan you are describing the sales challenge of the century!

We rush forward for ever greater automation and escalating infusion of technology to the everyday. On one hand the entire edifice stands on data. And on the other hand the technology class is as little interested as ever in the culture and governance of data. There will be problems.

So what is to be done?

The usual market answer to a persistent gap is to define a need for which "there is a sales opportunity".

Let's look at market answers to the data model infrastructure challenge.

On one hand, you note that data may in fact drag down BPM as a "pure workflow play". So could process and data technologies be better separated in the marketplace? Data on it's own though is "very infrastructure". And as I have pointed out here, infrastructure funding is always a challenge. How many organizations enthusiastically buy MDM products or maintain data modeling cadres?

Today the solution most of the time is that data models and business semantics come embedded in COTS, whether local or cloud. But then of course you're "on a level playing field with the competition".

The problem remains that "we still need our own unique and differentiating business semantics" on top of the standard platform.

My suggestion on a possible way forward is contained in the use of the term "business semantics". Data is too far down the stack for business attention. An argument can be made however that "business semantics" can be a first-class citizen of business discourse. Business should "care" about data models -- "er, business semantics".

Hey, there's already a precedent! Accounting is a first-class citizen of business discourse -- and it's all about business semantics! Sure, it's "data" -- but don't tell that to managers and executives.
Welll... yes and no. In BPMN we do have data object and data stores. In v1.2 data objects were just artifacts with no semantics or rules, but in 2.0 the data object was upgraded to a first-class semantic element and the data store added. However, BPMN mostly treats data from the perspective of a developer doing executable process design. Like most other things BPM, BPMN and data both have a time, place and usage, and most people don't do them, much less do them well.

I've taken lately to using BPMN + DMN, still haven't much use for CMMN even though my primary platform is IBM Case Manager (more on that later
RE “Having data as second citizen of process tech was, strategy-wise, wrong for the process standards.” I don’t think that “the process standards” are to be blamed but the usual approach of BPM vendors’ consultants.

Let us defined data in our BPM-suite tool, keep them here, deliver a “quick win” solution and leave the local IT team (after getting all the certifications, of course) to refactor sometimes this solution to be a good enterprise “citizen” with enterprise data model, master repositories in use, etc.

And if somebody points that this approach is creating a huge technological debt, then this person is black-marked being “anti-agilist”.

Hmm, in three areas I am familiar with (healthcare, manufacturing, law enforcement), data collection is a key part of any sales pitch..

In healthcare the presumption is that all patient interventions have been rendered using "best practices" and data extracted from these interventions is core to all decisions relating to the next intervention. Government legislation (USA) requires the collection of data that will be of use for long term outcomes assessments. Next, the industry is transitioning to performance-based reimbursement which will require stats on the content/context of services rendered. Accordingly, any sales pitches of Case Management/BPM to healthcare agencies will have data as core to all conversations.

In military equipment inspections carried out by 3rd parties, the presumption is that the inspections will be done according to strict protocol - the end deliverable (proof of performance) is data.

The sales pitches for workflow/workload management in both industry areas focus on 1) guidance from BPM and 2) 100% data capture/storage and subsequent ease of access.

In law enforcement, evidence (data collection plus connect-the-dots) is key. Here, there usually is an overabundance of data and what counts in a sales pitch is how the software will facilitate connect-the-dots exercises.
  1. John Morris
  2. 4 years ago
  3. #1920
Bravo Dr. Samarin: "And if somebody points out that this approach is creating a huge technological debt, then this person is black-marked as being “anti-agilist”.
+1 on the props to Dr. S. BTDT, as recently as last fall. Sucked swampwater. It's the higher-priced consultants (think "Big Four") that usually overrule the lower priced grunts who are in the trenches and know better, but senior management paying for the hoity-toity ones know no better. At least not 'til they've blown 2 or 3 years and several tens of millions of dollars.
  1. more than a month ago
  2. BPM Discussions
  3. # 10
Accepted Answer Pending Moderation

@Patrick - I kinda disagree that the data objects in BPMN 2.0 are first-class citizens. The 30-page section in the standard is a bit quaint, covering a bit of i/o specs in how they relate strictly to other process artifacts. A business app needs business data, from business objects. That is not addressed in the standard. So I do not use the data objects in BPMN 2.0, ever. They are way too sparse for the purpose of real-life execution.

@Alex, I agree that vendors are master at eluding the data moose on their table. You are saying exactly that vendors treat the data problem as second citizen. I am saying the standards should address this at least in underlying how important it is. (as for the anti-agilist, I remember your pain from our discussion :-) )

I would expect that the standards people work together in bringing a common taxonomy for business data objects too. OMG could cooperate more with OASIS (UBL, anyone?) or with vertical EDI dudes, or other standards bodies. After all, having a commonly agreed upon framework (and I am not talking only XML formats) should benefit both process standards and data standards.
CEO, Co-founder, Profluo
Thus... BPMN + DMN. :D
I am a bit disappointed by DMN too, from that standpoint.

I wrote extensively about that, thank God no one gives a rat :-)
  1. more than a month ago
  2. BPM Discussions
  3. # 11
  • Page :
  • 1

There are no replies made for this post yet.
However, you are not allowed to reply to this post.