Interaction Profiling is a new paradigm that provides a fresh, object-oriented approach to simplify Business Process Management (BPM). The concept focuses on the different types (profiles) of business interactions (transactions) in an enterprise business processing application and associates the corresponding, table-driven validation rules and BPM tasks (and more) with each profile. This metadata is managed by the Users directly with simple maintenance forms.
As an application development approach, Interaction Profiling does not involve hard-coding of any validation or business processing logic. Once the first, comprehensive Interaction Profiling framework is constructed, it can be re-used with minimal changes for all future applications. Based on the first prototype that embodies Interaction Profiling, it is estimated that the new paradigm will halve development times and costs for all subsequent systems.
On-going maintenance is estimated to be reduced at least one-third and possibly more, because changes to the validation and BPM tasks and rules really can be managed and controlled by the users themselves, with much less involvement from IT, based on empirical evidence to date .
This represents massive savings in development and maintenance and provides truly functional, fully-reusable components for any future application system. In addition, organizations benefit greatly because the Users and managers are empowered with the tools to control their own business growth and changing business requirements to a degree never before envisaged.
The profile level adds a new, User-definable dimension of marketing and commercial program possibilities to the existing Client and Product levels. This is a truly powerful, business-enabling feature. The re-usable framework in the first prototype application is almost three-quarters of the full implementation of the hypothetical order processing system. This confirms the order of magnitude of the estimates and substantiates the significance of the savings.
Existing business transaction processing application systems typically include complex criteria for validating transactions. A significant number of these systems (including all case tracking systems) also include a varying number of associated manual business processes that are required, in conjunction with the automated system, to complete the transaction life cycle.
The validation logic and workflow aspects of these systems are almost always hard-coded, either directly in an executable program, or using designer-assisted specification facilities that produce declarative XML or require compilation in some form and then change management through the ubiquitous develop - test - user acceptance - production staging. The business process management (BPM) of the workflow operations is often provided by relatively complex, runtime, server-based, proprietary software that is usually expensive and requires considerable expertise to implement and manage.
Interaction Profiling simplifies the specification, management and control of the validation and BPM logic by identifying the transaction profiles - those types of transactions that require unique data content in some way and / or a unique sequence of BPM Tasks - and associating with each profile the corresponding validation rules and BPM tasks and activities. The user comprehensibility of this approach is considerably increased because the individual profiles exactly model the differentiation that the users themselves apply in their daily operations.
Scope of an Interaction Profiling Framework
To enable an application system to apply these profiles automatically, the criteria that define each profile needs to be determined and stored in the application database. The set of profiles that match a specific transaction can and will change throughout the lifetime of the transaction. The set of BPM tasks and validation requirements will vary accordingly, which is encapsulating within the system the business intelligence needed to ensure all transactions are automatically processed correctly throughout their lifetimes.
We also include our Users, Organizational structure, the Roles that are required for security purposes to perform our business processes and which Users belong in which roles. If we tag our BPM data in the database with the roles required to perform our BPM tasks, we have now enabled our Interaction Profiling empowered framework with the possibility of automatically routing transactions to those users who can perform the specific tasks.
If we use Service Level Agreements with our Clients to promise completion times based on some kind of priority scheme, it makes sense to include the target duration for each BPM task in the metadata. Now the system can tell exactly how to validate each transaction, the specific BPM Tasks required, their sequence, how long they should take and precisely which set of Users can perform them.
This extends the capabilities to include workload balancing to optimize operations and spread workload evenly across the User community. Now we have enabled productivity reports, trends, gap analyses, bottleneck detection, more accurate completion estimation and more, which will seriously improve our operational management awareness and effectiveness.
All of this can be achieved just by applying Interaction Profiling and some minor, sensible design additions. For details on the Interaction Profiling framework design, please see Barrie's blog, here. The framework presented in the 12-post blog series generalizes the Interaction Profiling concepts and can be re-used for all future business transaction (interaction) processing information system development projects.
Level of Simplification Achieved
As a graphical illustration of the simplification that Interaction Profiling brings to a BPM application, the following diagram, depicts Microsoft's BizTalk Orchestration Designer (the BizTalk workflow designer) with a partial schematic of a sequential timesheet approval process workflow, taken from MSDN on line. To the right is the Interaction Profiling, equivalent, BPMN 2.0,complete workflow model.
The logic on the right really is all that is required conceptually with Interaction Profiling, to run through all of the tasks for every type of transaction in an application. The completely generic profile matching logic, all in one, re-usable piece of code, applies all of the varying criteria, as defined in the individual, user-managed profiles.
I admit that this is somewhat oversimplified, as it is essentially comparing apples and oranges. To clarify the intent of the graphic, the Use Case is timesheet approval requiring both project and resource manager approvals. The BizTalk approach requires all the logical possibilities diagrammed as indicated. To switch the approval rule from AND (both Project Manager and Resource manager approvals required), to OR (either one is sufficient), BizTalk requires the process flow diagram to be modified by a specialist, recompiled, tested and staged into production.
Interaction Profiling can achieve the same switch using an "Approved Timesheet" profile for the workflow implementation. The input data could contain an "Approval Status" enumeration (application or calculated field) with values of 0 - Pending, 1 - Rejected, 2 - Approved By Project Manager, 3 - Approved by Resource Manager and 4 - Approved by Both. This field would be used as a Profile Attribute for this profile and a User simply changing the profile attribute definition metadata from 4 to the from/to values, 2 - 3, would achieve the same result.
With Interaction Profiling, it would also be as easy for the Users to create a profile for overdue timesheet approvals and select that profile for reporting on overdue approvals without ever involving IT. This is just one of several possible, equally simple approaches using profiles. The key thing to understand here is that the Interaction Profile metadata controls the processes that are required, their sequence and which profiles can perform what processes.
There is no need for complex process logic diagrams for operational processing. The BPM diagrams are still required during the initial investigations though, to derive the Interaction Profile metadata in the first place. BizTalk can still be used for all of the other benefits it provides, but the process diagram(s) would be much simpler, as all decisions are based on the profile metadata.
Determining Interaction Profiles for an Application
It is worth making a few comments on the impact of this new approach on the work of the Business Analyst (BA) during the investigation and analysis phase of a project. In today's agile, iterative-prototyping application development world, there is a reduction in emphasis on initial investigations and analysis when compared with the older, more traditional approaches.
Agile approaches are generally recognized as an improvement over the older methods. However, it remains important to the overall success of a project to ensure that the real business needs are understood and addressed. With the intent of triggering some thought and discussion as to how Interaction Profiling could be integrated with the initial investigations, one logical, methodology-neutral approach might be:
- Investigate user problems and information requirements - interviews, Joint Application Development (JAD) sessions, research, collation of problems and requirements by subject / issue.
- Gather and label samples of all documents and files used in the existing system(s) - during task 1 typically.
- Document business processes, paying attention to the existence of any divergence in the task sequencing for business processes related to the Interaction(s).
- Identify the exact conditions that determine each divergence in the sequencing of tasks for the interaction instances.
- Analyze the information gathered and perform a top-down, iterative design to conceptualize a solution to the problems that also addresses as many of the information requirements as possible.
- Derive the data model for the conceptual designs.
- Iteratively refine this data and functional model, preferably with a prototype and plenty of user involvement, to finalize the new system design (data and functionality).
- Now re-visit the analyzed material and derive / finalize the production set of Interaction Profiles needed for the system.
- Determine and agree with the Users the detailed Profile Attribute values (criteria) required for the Profile set.
- Also formally document and agree the security Roles required, keeping these to as small a number as possible.
- Define and agree / finalize the detailed, field-by-field Validation Rules for all the interaction types and their Profiles. Last but not least, (probably in parallel with 10 and 11), determine and agree the master list of distinct BPM Tasks (and optionally subordinate Activities) and the individual lists for the interaction(s) and each of the Profiles associated with the interaction(s).
Yes, this is a lot of work, but there is really no shortcut to the work required to derive high quality user requirements and a good conceptual design. Anything less will dangerously increase the risk of pathological damage to the design right from the very beginning. Placing a more formal and definitive emphasis on the up-front investigations, analysis and design will mitigate the risk of failure, which occurs throughout the industry all too frequently.
Sales Order Processing Prototype
The first, real-world implementation of generic Interaction Profiling currently exists in a hypothetical, prototype Sales Order Processing System, SOPS (no irreverence intended, ahem). This section presents some of the more pertinent aspects of the prototype, focusing on the functionality and benefits implicit in the Interaction Profiling approach. The following graphic presents the simplified SOPS Sales Order initial data entry screen.
The screen data is fairly self-explanatory and is basic, given the prototypical nature of the application. The highlighted "Profiles" list box warrants serious consideration, however. It contains the current list of profiles that match the transaction and is actually updated by this form when data is changed and the list box refreshed. The list of Profiles completely classifies this transaction in every characteristic relevant to the validation rules and the BPM task requirements. No User action was required during data entry. SOPS performs the Profile matching behind the scenes and this list of definitive characteristics is the immediate result.
What is most significant, though, is the effective encapsulation, within the profile metadata, of all the complex criteria testing, which typically is performed in numerous places throughout flow diagrams or application code or some combination of the two. The profile attribute criteria is specified in one place and that controls the validation rules and the BPM logic throughout the complete application. Thus, all redundancy in logic specification is eliminated.
Another big advantage of the Interaction Profiling approach is that there is NO redundancy in the data. The profile level provides a user-specifiable classification for any business requirement. This is dynamic classification and re-classification, as required, by the Users themselves of their Client or Product base for any purpose whatsoever- a very powerful concept!
From the flexibility perspective, Interaction Profiling provides a huge simplification for any kind of new program implementation. Just create a single new profile with the Attributes and defining criteria, run a global update of all profile metadata (SOPS includes this capability on the Administration menu) and immediately you see a new classification for the new program on every relevant transaction within the system, with no IT involvement.
This flexibility to tailor Interaction Profile(s) to any combination of defining criteria is extremely powerful and eliminates a huge amount of maintenance that would be required with less flexible design patterns. Serendipitously, this also provides an easy-to-use, ad hoc reporting capability at no extra cost.
All of the profile metadata resides in the database. Maintenance of the metadata is performed by the Users (see Blog Post #9) with almost no IT involvement. This gives the Users full control at their fingertips. The prototype metadata maintenance forms contain minimal data. In fact, all 23 non-reporting profiles are implemented in the prototype with just one Profile Attribute and only the 2 demonstration ad hoc reporting profiles require two attributes.
There is no doubt that this minimal degree of complexity is easily manageable by User supervisory personnel. Considering the advantages cited previously, the benefits clearly outweigh the one-off overhead of implementing the first Interaction Profiling framework. The development and maintenance cost savings (which are tentatively estimated to be as much as 50% or more reduction in all future development effort and 33% less annual maintenance going forward) certainly outweigh the initial overhead.
Prototype Design Framework Scope and Metrics
Full detail on the SOPS Interaction Profiling, re-usable framework is available in Barrie's blog series, so we will simply summarize the functional scope in bullet form:
- Input Validation
- Business Process Tasks and optionally Activities
- Automatic Task Allocation and Routing
- Workload Balancing
- Service Level Agreements
- Target Dates and ETAs for Clients
- Bring Forward Reminders for Staff
- Accurate Completion Estimates
- Bottleneck Detection
- Productivity Metrics (Target versus Actual, Trends, Gap Analyses)
- User and Role Administration
- Application and Business Process Role-Based Security
- Optimal Lookup Handling
- Report Filtering
- Simplified Ad Hoc Reporting
- Audit Trail / Activity Log
Not many (if any) existing BPM products provide all of the above items (inherent in the prototype Interaction Profiling framework) in a relatively simple, re-usable and truly User-maintainable manner.
The following graphic quantifies the relevant statistics for the SOPS implementation.
The SOPS hypothetical order processing prototype is a relatively small application system. The order processing designs are rudimentary, but the overall application contains over 50 tables. The most highly significant statistic, however, is the determination that a full 74% of the developed objects (client GUI forms, pivot charts, queries, application code, reports, SQL Server tables, views, stored procedures and user-defined functions) are completely re-usable.
This implies that subsequent, similar - sized applications would only require approximately one quarter of the development effort that would otherwise be required. A small number (around 9 estimated) of database tables in the framework would need to be modified very slightly between applications. The conceptual purpose of all of the framework tables, however, remains the same, so modification would be simple and quick.
The following conclusions are proposed for consideration.
- Interaction Profiling provides a quantum leap forward, simplifying the approach to BPM within an Organization. The re-usable framework provides considerable functionality to subsequent applications once the initial framework is constructed or acquired.
- The encapsulation of the validation rules and BPM logic in a table-driven manner empowers the Users to take control of their business processes in a more comprehensive and comprehensible manner.
- Interaction Profiling effectively provides business-relevant control at user-definable profile levels.
- Much code and data redundancy is eliminated as information can be recorded at the profile level.
- Operations such as transaction routing, workload balancing and monitoring, task sequencing, bring forward reminders are fully automated, eliminating manual delays, errors and omissions.
- Service to Clients can be improved with improved efficiency and accurate completion estimates.
- Managerial control is considerably enhanced, with workload monitoring, bottleneck detection, productivity reports, trends, gap analyses and annual reports, graphs and pivot tables and charts, plus marketing, communications and new programs that are easy to implement at user-definable profile levels.
- Reporting is facilitated by intelligent filtering, including profile-based breakdowns.
- A truly simple ad hoc reporting capability is inherent in the Interaction Profiling concept.
- Future application development effort will be reduced significantly, possibly as much as 50% or more.
- On-going maintenance costs will also be considerably reduced because of the flexibility and User-maintainability inherent in the Interaction Profiling concept. This is estimated at 33% on-going, after the production of the first framework.
- The concept is implementation neutral. Desktop, browser and portable device applications can all benefit. It is also transparent to the design paradigm whether a system is cloud-based or not.
Interaction Profiling is a brand new concept. It is in its infancy, yet the first prototype clearly demonstrates that the potential benefits are significant, not just to BPM but also to other, common application system features. For more in-depth information on the prototype Interaction Profiling design framework, follow the link to Barrie's blog on Interaction Profiling below. Questions, comments, criticisms, suggestions and any other feedback is welcome.