BPM and the Great Data Consolidation

I consider myself a student of trends…not a techno-junky who leaps on every new product, but someone who watches the marketplace and decides when to enter.  I was late to Facebook, and didn’t blog until two months ago…about two years after the mainstream.  As you can see, once I’m in, I give it my energy.  I seem to usually think it out before I try it out.

As I watch the movement of technology from the early 70’s to now, I see a movement that can be described in simple terms as the increasing consolidation of data, regardless of what we called it in the moment.  In the beginning, the low hanging fruit was the core data of all business, accounting.  Then came a progression of the rest…production data, customer data, transaction data, communication data (i.e. email).  All along, it has continued to be a grand consolidation exercise as the tools to move, store, filter and disseminate get stronger and faster.

The slow start

While the internet was the ‘great leap forward’, finally connecting everyone, it took another ten years for people to grasp how that could be used to generate revenue.  From “when will Amazon be profitable?” we are seeing business move beyond the baby steps into world-affecting applications being launched and adopted within mere months.  But it still continues to be essentially the same march.

Revolution of ideas

This is a simple but profound movement to consolidate data in new and cool ways.  Nothing much more complicated that that.  It isn’t the technologies that lead this, but instead the ideas for how the technology can be used.  Imagination is leading the revolution, not technology, which is just along for the ride.  That’s a significant break from the past.

Once upon a time, television came out, and people saw technology that has never existed before and they adopted it.  Now, a new idea comes out and it has to be comprehended before it can be even valued.  I’ll admit that I didn’t comprehend Twitter for a while.  It wasn’t new technology at all, but simply a new idea, and I was stuck behind the new idea curve for a while.  It was when I understood hash tags as categorization and filtering that it dawned on me that it was yet another step along the way of the Great Data Consolidation.

Behind the curve

Is your enterprise behind the curve?

The business process community is in this same place.  Stuck behind the curve.  The vast majority of companies have not centralized process data in any meaningful way beyond the data that could be entered, moved and used in automated ways.  Quite typically, if it can’t be automated, it isn’t a candidate for centralization, but why not?  Are we missing the next enormous opportunity?  There’s no reason the Great Data Consolidation should skip over the central aspect of what we do every day…true business process.  Secondly, while companies implement social media, it is only a business version of what people are doing in their personal lives, and unstructured conversations only create more chaos.

Game changer

Enter social BPM.  A welcome opportunity to show that data, in this case…conversation, can be at last the urgent reason to put business process data into its rightful, central place.  The end of Visio as we know it.  No more data locked in files, stuck in the upper tributaries of the network.  Oh, and when you centralize it, you’d better govern it sensibly or it will lack authority and flexibility.  There is an enormous amount of conversation that has always taken place around business process that has just been done in low-tech ways with no central management.  With Social BPM, the conversations must be managed alongside business processes that they support.  This obviously that means deployment needs to be added to the centralize and govern steps, otherwise the conversations stay locked away from those who need to participate.  These aren’t new ideas for those of us who’ve been in the collaborative BPM space for years.  But they do, hopefully, provide the urgency that has been missing in the broad market for a centralization of process data that is long overdue.

The Great Data Consolidation is soon to pull social BPM into its orbit, and that will a game changer for BPM as a whole. 


Categories: Future of work, Social / Collaboration

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

11 Comments on “BPM and the Great Data Consolidation”

  1. June 2, 2011 at 7:50 am #

    Chris, sorry to be a bit blunt about this. I spent a good deal of time on reading through the APQC document yesterday and each page I was shaking my head more about the humongous bureaucracy behind this approach. Has anyone accounted for the TRUE cost to not only set this up but run this? Anyway, stacking ITIL, SCOR, eTOM and others on top of each other sounds like a nightmare to me. But I do admit that my focus is not manufacturing but service industries. You would need to be much more clear what types of business you are targeting with this.

    And now you write about process data consolidation and centralization and then you throw Social into this as another means to centralize and control the busines top-down. Yes, there are always these good will messages in your descriptions that mention the user and the customer but it always ends at standardization, optimization and centralization – all great indicators for communist-like, plan-economy bureaucracy structures.

    My take: Social BPM IS NOT about controlling onversations ‘alongside’ the processes. That is utterly useless. Social BPM would be about empowering business people to CREATE and ADAPT processes as a team and as needed. Anything else could be done by email or collaboration tools such as Notes as well and it hasn’t improved anything or reduced the bureaucracy connected with BPM. Ideally management can enter the Social interaction by defining and verifying objectives, targets and customer outcomes directly.

    But you are right also: Collaboration is nothing new at all. So why would it change something now under a new name? Well, it won’t. What you write sounds much like ‘jumping on the marketing bandwagon with a verbal commitment’ without changing anything.

    ‘Social’ is not about central control of conversations. It is about letting go of control by means of user empowerment and utilize the evolutionary powers or emergence. You say that Social needs a definition of authority and that is correct, but that authority can’t be central as otherwise it is not social! Empowerment is GIVEN through authority, goals and means! Allowing people to chat about something is not empowerment. Controlling that chat is even less social empowerment.

    Yes, BPM must allow unstructured, collaborative processes and Social needs be channeled to become part of the actual case/process and empower people to work with real-world process resources, entities and content. Only then they will meet in the middle, but that is not BPM with Social conversations tacked on. It is what I described years ago as adaptive processes.

    Finally, I don’t understand what kind of great data consolidation you are referring to. Is it the centralization of all process definitions? Mapping business data into the process? MDM? Business Architecture? What exactly do you mean by that?

  2. June 2, 2011 at 8:09 am #

    Max, there’s nothing top-down in what I’m describing and social BPM is actually bottom-up. It isn’t about controlling, either. I couldn’t personally see the purpose in Twitter’s streams of information until I understood how I could filter the things that were valuable to me from the noise. Unless you understand the paradigm, Twitter is like having all radio stations broadcasting on the same frequency. Companies implementing social networking without having a way of connecting it to what they do (process) are missing what I’m referring to as a great opportunity to consolidate data in both conversations and how work gets done. This is being done as I type, and I can have a conversation with you offline on this. Also, managing compliance and other frameworks is very difficult if you can’t make the links and manage the changes, which is something BPM does, when done well. We can talk about ThyssenKrupp, for example, when we talk offline.

  3. Tom
    June 2, 2011 at 8:20 am #

    The end of Visio as we know it – I like that.

    In terms of Social BPM gathering steam – there seems to be a tipping point that involves people “getting” a new idea as well as someone executing and communicating that good idea in an easy to understand way.

    Think Facebook – social networks had been around for a few years, but not that widely used. I had a Facebook account, but didn’t use it all that often because not too many friends used it. Then that all changed in the space of about 6 months. Suddenly, everyone I knew – across the country and across age ranges started friending me.

    This was due to the combination of people seeing the personal value of a social network as well as Facebook executing that idea well – both the ease of use, speed, – and communicating new ideas and actions like “friending”, “liking” needed to participate.

    The same is likely to happen soon for Social BPM – people are starting to “get” the value. I hear this a couple of times a week now from smart, forward thinking process leaders within companies. Execution is also quickly getting much better – centralized, fast, easy to use. I think we’ll see a takeoff in Social BPM too.

  4. Loriocarl
    June 3, 2011 at 4:09 am #

    I think you are on to something. A typical reaction to complicated situations is centralization, with the theory that experience and talent will rise to the central governing group. The Internet has taught us that talent is everywhere and traditional experience may not be necessary. A social BPM allows a richer quicker flow of ideas amd requires thought leaders rather than traditional governance. Very interesting.

  5. June 3, 2011 at 6:27 am #

    Have to agree with Max. “Empowerment is GIVEN through authority, goals and means!” Chatting about a process in a sidebar doesn’t materially change the process itself, though it might have some therapeutic value. That’s just BPM plus Social.

    I’m also not sure how a sidebar social conversation ‘consolidates data’, except for adding some unstructured text to history.

    Social BPM, means process itself has to be A) real-time; B) interaction/event-driven; and C) contextualized. Context is about the user co-creating the process within authority, goals, and means provided, which enables emergence.

    The ACM community missed the importance of ‘Joint Optimization’, I had originally felt that would have been a good place for such notions, but I’m glad to see it being picked up by the same folks under different banners.

    • June 3, 2011 at 6:44 am #

      There’s much more than therapeutic value about capturing and distributing the conversation around business process within the context of the visualized process. The idea of consolidation is that things that happen in many, distributed places can be brought together in one, rather than relying on people to manually connect the dots of dialogue and process data. It will happen either way, but is much more valuable when it happens in a single stream.

      I can see where social BPM allows for unstructured process to have a place to ‘live’ alongside the more structured types of process, like compliance or regulatory. I suspect we simply come from different worlds of requirements.

      • June 3, 2011 at 7:34 am #

        Hi Chris – Thanks for responding.

        Could be the title that threw me – ‘the great data consolidation’, which is what made me follow the link from LinkedIn. My expectation was that you’d address the challenges that BPM has with Data beyond its process variables and its dependence on Services or other integration to do anything meaningful at the enterprise level.

        There is certainly value in joining the dialog to the process, a lot of us do that in some fashion already, I just don’t know if it’s worthy of the title.

        If you are talking about joining 3rd party activity streams and chats, as an integration point, that is BPM plus SOCIAL as I defined it above.

        Our use case are probably related, but semantically speaking I think Social BPM should mean process that is intrinsically socially oriented (conversational/contextual).

        This term was first used by BPMN community to mean ‘collaborate re: design-time’, which also fell flat as it didn’t mean what it said.

  6. June 3, 2011 at 9:07 am #

    Title aside, there are a great number of non-automated activities that require structure and conversations. A classic example is what I see at Northrop Grumman, where there are many ways to go about ‘designing’ depending on the end product (ship, plane, satellite, SEAL Team prototype), but there are definite needs to contain the number of variations to ensure reusability and follow contractual obligations. The social side means there are many important conversations that need to be managed around a design method, and it has to occur within the context of specific, planned design activities. There also needs to be a historical record of all of this. Bringing the two together is a significant consolidation of two things that are as old as mankind…activity and conversation. The technology to do this isn’t new, but the ideas for its use are.

    Social + BPM (non-workflow BPM) means it takes far less time to get everyone on the same page, and the ‘getting on the same page’ is in one place, not many. Consolidation.

    I have many more examples of where this is happening and where people have a genuine desire for consolidation of the process and contextual conversations. This can be chaos if done by simply adding social technology to the mix. It needs to be done in a way that consolidates the conversation and process data and has some way of filtering and classification to avoid chaos.

    Thank you for your comments.

  7. Pearl Zhu
    June 4, 2011 at 11:12 am #

    Hi, Chris

    Always enjoy your BPM series of writing, when you articulate the great data consolidation, make me think about quite many articles discussing Big data: the opportunities and potential risks. Social/mobile computing is really transforming the traditional organization into the digitized economy, which means data management become the significant disciplines for any forward-look companies now: on the one hand, we need the open environment to inspire innovation, on the other hand, we need standard process to catch & measure the data in real-time, and transform them into business vision, I like your Social + BPM concept, that should be the future trend to manage the extended enterprise in the future. thanks.


  1. BPM Quotes of the week « Adam Deane - June 5, 2011

    […] BPM and Data Consolidation – Chris Taylor The business process community is in this same place. Stuck behind the curve. […]

  2. Why hasn’t process data been treated fairly? | BPM For Real - September 22, 2011

    […] I was recently asked, “Do you ‘fix’ processes before implementing a business process management system (BPMS)?” This is a very good question and something that sparks great debates whenever it comes up….how much time should be spend documenting processes that exist, especially if processes are seriously flawed? I’ve said many times that if you don’t start with the processes you have, good or bad, it is tough to create something better from scratch, have it make sense, and at the same time understand the path to get there. The reason people have this debate is a symptom of the bigger problem…that process data usually hasn’t been captured, owned, and change managed in a way that even allows for what I’m advocating. Those who argue for a clean sheet of paper don’t see process data in the right way. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: