In the previous article, concept storming to provide data platform value, I have outlined why I think it is important to understand business concepts in the context of building a data platform, and I proposed the general idea of a workshop based methodology that I called “Concept Storming”.
In this article I am trying to provide a bit more operational ideas on what are the business concepts we need to discover and how a Concept Storming workshop can be carried over to collect and organise these concepts to build the foundation of a data platform.
Mind this is “work in progress”, not a tried and tested recipe to follow, so I am sharing it with the hope to inspire you to try it out, even with your own variations, and provide suggestions and feedback to improve the overall idea.
Concepts for your data platform
What is a concept?
A concept is an abstract idea, like “customer” or “document”; it is the common idea behind the real customers or documents.
A concept generalises the common “features” of a set of real objects.
We concentrate on the features we know about / are interested in.
When we try to organise concepts it is important to consider that they can often be organised in a hierarchy; just think about “document”, “invoice”, “order”, “shipping bill” or “party”, “customer”, “prospect”, “user”…
What concepts does the business needs?
A business centric data platform needs to describe the concepts that have (enough) meaning for the business, i.e. the ones that business users want to “reason” about.
Reason here can be expanded into report, explore, analyse, build an app upon and so on depending on the intended use of the platform.
The main goal is to identify what concepts are important to business reasoning, so that we will integrate our source data and transform it into information expressed through those identified important concepts.
Business concepts can be seen as the desired output of a data platform.
Just because a concept exists in your domain and it is important for operations or other practical reasons, it does not mean that it is an important “business” concept.
Take “invoice” as an example. It is definitely a ubiquitous and important concept for the operation of almost any business. It is also found in almost all source systems and it provides a trove of important information, but it might be that the business does not really care about invoices, they care about “sales”, possibly “orders” and maybe “deliveries”.
What is important is also different by company and by the scope of your platform; the scope might also evolve, becoming more detailed in subsequent iterations. The good thing is that higher level, more important concepts are here to stay.
How can you figure what concepts are important?
Talking with the business people and asking simple questions like “are you interested in counting invoices per month?”, “are you interested in knowing how many invoices you sent to a customer?” or “are you interested in the average invoice line value?”.
The answers will tell you if “invoice” should be an output of your data platform or remain just one, possibly very important, input.
Concept Storming: uncovering relevant business concepts
Now that we know what we are looking for let’s explore one way to uncover the Business Concepts of interests for a domain, using Concept Storming.
One important key is to be clear about the domain to look at. You can look at all of a company, to a specific department or get started with a reporting area.
The setup: people and time
You should assemble the people for your Concept Storming workshop according to the domain you want to uncover.
The ideal is to have different kinds of future beneficiaries of your data platform, the ones that will use it directly and those who will receive the work derived out of it, plus a few power users or IT representatives.
Regarding the number of people I tend to prefer smaller groups: more or less two to four business users, one power user and/or one IT user, and one facilitator. With these numbers everybody gets well engaged and has a very good chance to express his ideas.
It is possible to scale up the format by splitting a bigger group in a few smaller ones and then aggregating the results of the individual groups for each phase, providing a more diverse input and a second level of discussion.
Regarding the time I would suggest about half an hour every two people in the room for each phase that you want to cover during the workshop.
The phases
I tend to divide the possible activities in the following four phases.
After you go through the first one and you have your list of concepts and relations it is up to you to decide which one to perform and in which order.
- Identifying and clarifying concepts and relations
This is the core activity to be carried over in a workshop style.
It provides the basic result and the material to discuss in subsequent phases. - Organising the concepts
Unless the domain is really small or simple you probably end up with very many concepts and your project would benefit by having them grouped by a few main business patterns and then organised in a hierarchy. - Discovering the details of concepts and relations
Once we have identified the concepts it is of paramount importance to agree on how business users establish the identity of individual instances of a concept (business key) and to double check relations by verifying how they put one concepts in relation with another (foreign keys).
It is also a good time to capture the important attributes of the concepts, i.e. the ones that business users are most interested into. And clarify what they mean. - Test and refine our discoveries
Now that you have used one domain / use case to discover, flesh out and organise the business concepts you might want to test if they describe well the business by throwing at them some current pain points and few key business processes.
1 – Identifying and clarifying concepts and relations
Start by stating clearly what is the scope that you want to tackle during this workshop, define what a business concept is (see above), but overall be clear with business users that this is the time to spell out what they think to be important for them to find in the data platform.
This is the creative phase of the workshop. Everybody should be energised and willing to propose ideas, as there are no bad ideas, and it is better to err on the side to have too much input.
Be sure that each participant has plenty of adhesive notes and markers and give them some five to ten minutes to start writing down each concept they can think of on a separate adhesive note. In the meantime, if you do not have a big whiteboard, prepare a huge piece of paper hanging on one wall.
You can also suggest participants that they can take notes of important relations they come thinking between these concepts. Whatever helps them to capture these relations works; “A customer places an order for a product through a salesman” or “an order has a customer, product, salesman and destination” are both fine.
After this individual warm up to the domain, it is time to start with the group work. To encourage feedback by all, you can start by asking the participant with less adhesive notes to come to the whiteboard and place one concept at a time, briefly explain it and its relations with the other concepts already on the board. Ask him to draw a line between two concepts, when he talks about a relation.
If the concepts and relation are not straightforward for all, it might be a good idea to capture at least the relation name and a collectively agreed upon definition of the concept. You can also add remarks and clarifications on the adhesive notes.
Go through the participants so that all identified concepts are on the whiteboard and you probably have a mess of lines going through them.
One important task of the moderator during this phase is to try to help the audience to distinguish between concepts and relations versus attributes and measures. As an example “discount” or “promo” can be valid concepts if the domain is marketing, as you probably give “a name” to a promo / discount and you want to report about its use, but most probably they are attributes / measures in other domains where you just care about the difference between full price and price actually paid. Only your business user can tell you what is their situation. That is why you have them there.
Once you have gone through all the identified concepts with their relations and you have clarified their definitions, it is the time to put together the effort from different groups, if you started with a bigger group and you split it up.
The final outcome of this first phase is a set of agreed upon concepts that are important to the business and the relations between them. Probably you have also discovered a few important attributes the business people care about.
Depending on your goals and the team composition the next step could be to reorder the concepts in a hierarchy or to dig deeper to define their identities.
2- Organising the concepts
The first phase will probably leave you with a mess of notes and lines on the wall, this is fine.
If the domain is wide enough and / or the mess is big enough it might be worth spending some time re-organising the findings.
One possible way to reduce the complexity is through the use of business patterns, like Resource/Asset or Party/Role or Document or Event, to group the many concepts into more homogeneous groups and possibly in a hierarchy, like Document > Contract > Sale contract VS Rent contract.
You can use a palette of business patterns, like the one used by John Giles in “The elephant in the fridge”, or any set of business patterns that make sense for your business and go through the following steps:
- assign a super-type to all concepts you have
- go through the set of patterns and use them to double check that you have not forgot anything important; e.g. do we have “Events” we want to report about?
- reorder each concept under its super-type and if it makes sense build a hierarchy inside each pattern
- most relations should be fulfilled once at a higher level for many concepts, while a few ones might make sense only between lower level concepts.
At the end you should have a more orderly layout that is easier to understand and offered you the opportunity to aggregate concepts and discover if there are multiple levels of abstraction.
3 –Discovering the details
Once you have a good set of concepts it is time to fill in some details that are important to manage them in a data platform.
Essentially we are talking of identification of concepts, connection between concepts and important information the business cares to know about each concept.
Let’s go through them:
- identify each concept’s Business Key (BK), i.e. how business users identify, find, communicate each other the identity of a specific instance of the concept.
E.g. what does a sale writes to admin to uniquely identify a customer? what is requested in forms to identify a customer? what is shown in report when showing a list of customers?
You might find more than one for a concept. While that is a bit of a pain it is better to know that there is a split world, so you can decide how to handle it. - identify how in each relation is built the connection to the concepts that participate in the relation. Where are and what are the Foreign Keys (FK)?
E.g. How is a sale represented? How the sale connects to the products, to the customer and to the salesman? - identify what are the important informations (attributes) that business people wants to know about a specific concept. And what is the meaning of them.
This is definitely a task that could easily degenerate and go into too much detail.
At this moment the focus should be only on the few attributes that are really essential and are not trivial or because are hard to get or are not immediately though of. A customer’s name or address is obvious, but its credit score might not be so much
4 – Test and refine
Once you have a clear(er) picture of the concepts, their relations and the important info you might want to have a little check to verify that what you have captured is not just a castle of cards and it is useful in the context of the business.
You can “stress test” your findings against these categories of realities and eventually go back and refine the areas you find not satisfactory:
- current pains VS goals / ideal future– what are the major pains with the current data platform / reporting system? what is overly difficult to know / manage / follow up with today’s information?
Does the identified concepts and relations provide for a satisfactory future? - business processes – pick one or two, definitely not many, important processes and check that the identified concepts, relations and key attributes allow to represent and reason about such processes.
- key reports / application / ML / AI – the business model expressed by the current set of concepts is adequate to fulfil the key reports , support the key applications or the desired ML / AI processes?
If there is something really important that will weight on the success of the new data platform it just makes sense to check that it can be achieved.
Conclusion
I hope that this discussion of an experimental method to produce a business model to build a data platform will be useful for you, as it has been for me trying to clarify and formalise it.
If you have limited time with the business users I suggest that during the workshop you concentrate on identifying, clarifying and organising a bit the business concepts, with identifying the business keys as the next best thing to do that benefits from having multiple business users in the same room.
All the other steps can be carried on also after the workshop and with direct communication with specific stakeholders and the support from internal tech people.
Consider that all this will just produce an initial model that will certainly evolve and integrate more knowledge as you go through the project, so do not try to build a perfect model, focus instead on building a network of people who understand what is going on and can contribute their knowledge when needed.
I would be very happy to hear your thoughts, suggestions and feedback to improve the overall idea !
Ciao, Roberto
Roberto "robmczag" Zagni 