Why is it so difficult to measure PLM success?

Everybody talks about measuring the result of their PLM initiative but nobody does anything! This is of course exaggerated, but in general, there are too few companies knowing what PLM has given them (or cost them). Many companies do not really know if
the PLM project is a success or not. The result is that management only cares about PLM once a year when they see the cost in the budget. Why is it so and what can be done about it?

There are few numbers available

In my 18 years in the PLM business, I have seen very few monetary values of PLM achievements. Of course, the PLM solution providers have their slide decks with generic numbers and maybe a few concrete examples they use over and over again. It is difficult to use those numbers directly in another company. The KPI’s can be reused, but you have to have your own numbers to be able to trust them.

This quote from Accenture is quite illustrating:

“The CEO came to the department and asked: I have spent 100 million euros on the implementation – what are the business results? They couldn't give him a proper answer.”

Unfortunately, this CEO is not alone.

PLM vendors have Excel sheets where you fill in a predefined list of possible improvements. By magic, you get a huge number that nobody (except the PLM sales guy) trusts.

We use the lack of numbers also as an excuse. The others do not have the numbers so I will manage without them as well.

The engineer’s flaws

I think that one reason for this lack of measures is that it is mostly engineers (as me) which care highly about PLM. In addition, engineers have two flaws in this context:

• We are satisfied when things work (then we move to the next task)
• We want answers to be un-ambiguous

We engineers want efficient and consistent workflows and a CAD integration that works. Yes, it would be nice to know what value that brings, but what we really care about is getting it to work. We are satisfied when it works and we have a feeling that this is good for the company. We do not see the importance of measuring what value it brings.

Engineers also like things to be accurate and without uncertainty. Calculating a business benefit and ROI is not an exact science, thus we do not trust it and do not do it unless we have to. If we are forced to, we tend to look only at the benefits that we can measure and get an accurate number. E.g. number of documents stored, ECO throughput time etc. Such numbers can show how PLM progresses, but it does not tell so much about actual business value. We miss the big picture and do not put areas that maybe are more important but harder to measure. E.g. global collaboration enabling two sites to share resources and work together or increased service margins due to better access to information.

Do not know how to do it

Given the engineering background, we feel it is hard identifying what to measure and defining calculation methods that we trust. Many PLM people agree that it would be good to have these values, but do not know where to begin and give up before starting. Many PLM people think they lack the financial tools and understanding to attack this beast in a proper way. They focus instead on the things they know: Getting a good configuration management process in place.

Hard to measure

Yes, it can be hard to measure business benefits and you cannot always trust the numbers 100%. It is easier to measure reduced volume of products on storage or faster production processes than improved product information quality. ERP focus on physical products and things you have while PLM focus on virtual products and things you do not have. This is still no excuse. We have to start showing the value of PLM or it will continue to be a product development support tool or even just a CAD management tool.

A typical excuse is also that we lack the numbers from before PLM came in and do not have anything to compare with. That is the same as saying that PLM is cemented and will not improve beyond the initial improvement. If that is true, the management is right in not investing more in PLM.

Why do you need to measure PLM success?

PLM people always complain that ERP gets all the attention and PLM at the best gets the leftovers when it comes to management attention and budget. The typical complaint is that management (thus the budget owners) do not understand PLM and its importance. We can only blame ourselves. We are unable to talk to the management in a language they understand. My experience is that some ERP business values also can be tough to calculate, but the ERP guys and girls are much better at talking a language the management understands – business benefits, calculated savings, earnings and cost. If PLM people were able to do the same, PLM would be in a much better position.

You cannot expect management attention and large investments or operations budgets for PLM (at least not anymore) if you cannot show what PLM brings and an ROI that management trusts. A trend is that PLM investment decisions are done on C-level.

Another thing is that measures in itself give improvements as you put attention on the area and follow the progress.

What can you do?

We also touched this in a previous blog where we looked at data from PLM related processes with a method improvement focus. Here is another interesting blog from Melanie Manning about how to create effective metrics.

First, you have to agree on what to measure. Tie PLM to the business strategy. What is important for management? Where will your business go in the next 2 to 5 years? Identify the areas where PLM contribute to the overall business strategy. PLM can be an enabler for changing things. Identify KPI’s related to the business strategy.

Start thinking like the financial guys. Act like a business analyst. Is there is a business improvement somewhere supported by PLM? How can you measure it? Do not only focus on the bottom-up simple stuff. Look at the big picture and involve the business and management. Think about short-term, long-term, tangibles, intangibles, cost reduction and revenue increase. As a common exercise, you can find improvement areas. Get business and management to give their thoughts on possible improvements. If they give the numbers, they are more likely to trust the result.

Some examples are: Product introductions per engineering hour, degree of collaboration, engineering efficiency, number of executed projects, project execution time, increased service revenue, reduced number of applications, product error costs, degree of CTO vs ETO, amount of manual entries of information. You have to agree on what makes sense for your company. The trick is turning such KPI’s into money values.

The most important is to start measuring. Agree on some KPI’s and follow them over time. And show what the KPI’s mean in money.

Tore Brathaug

www.infuseit.com

You don’t know what you don’t know about PLM

It is hard for non-PLM experts to know all the aspects and possibilities of PLM solutions. I have seen it over and over again. How can companies looking for a PLM solution make the right choices? Should they start specifying what they need or should they just trust their hunch and pick the one on top of their mind?

The topic is also valid for companies that have had PLM for a long time and are stuck. It is hard to know how and in which direction to go. In this blog I focus on first, or even second time buyers that don’t have large PLM organizations and a lot of internal PLM knowledge.

PLM can be so much

There are as many opinions about what PLM is as there are people. On an overall level you can probably agree that the core is:

• Part Management
• BOM Management
• Document Management
• Change Management

Throw in some CAD integrations and an ERP integration. Maybe variant management or project management. Or maybe PLM are all the tools and processes creating or managing product information?

An enterprise PLM solution like Teamcenter, Windchill, Agile and Enovia has an enormous breadth, but there is also a challenge to understand the details in a selected PLM area. For example CAD integration or the change management process.

The core functionality mentioned above is really not more than classical PDM and which what most companies still need the most. But it can be hard enough. There are so many details that you probably didn’t think you had to care about.

• What should the naming rules be? Running number? Any logic?
• What metadata do you want need, and can you agree on it? How does that match for instance ERP?
• What are the approval steps for different object types? What is an object type by the way?
• Easy revisioning in development and full blown change process in productionmanufacturing?
• How do you get a BOM from CAD? Can you use it directly?
• Etc.

There is so much detail in core PDM that you will not be able to cover it all during an evaluation phase. The devil is in the details and some of these differences can have significant impact on your way of working. Unless you really know what to look for you will not be able to identify the details that matters for you until you implement or even roll out the solution.

The evaluation process

Buying PLM is like buying a house. You visit the house one or maybe two times for half an hour. You read the prospect and then pay more than you planned. Then you get surprised a couple of months later when you get the house as there are so many details you see now that you could not possible see during the sales process. The good thing is that the human brain usually rationalizes bad decisions to look good. The good things about the house is exaggerated and the bad things are ignored. It can be the same with PLM. However, problems with PLM tends to be visible to others than the ones making the decisions, and that makes the bad things hard to ignore.

The problem with buying a PLM system, and a house, is that unless you are an expert you don’t know what to look for. You look at all and end up looking at nothing and make a decision based on who did the best demonstration and who you trusted most in the sales process. By the way: those people you trust the most are not likely to be part of the implementation anyway.

The sales presentations (and the information you find on the vendor’s websites) from the competitors can probably be switched without you noticing. On a sales level all solutions can do everything. You have to find the differences yourself. I like this quote from David Stewart at Zerowait-state: "..it’s hard to discern which is the better (PLM) choice. I still feel like the best way to know or choose is based on experience".

To compensate for not being able to see clear differences between the PLM solutions you typically describe what your requirements are and send out RFQs with a lot of details. If you are considering any of the enterprise tools the problem is that the solution providers probably can say yes to almost all and do the same things as the competitors. It is very hard to distinguish PLM solutions based on a list of requirements. See this blog for some insights to why traditional RFQ processes is a problem.

The typical tendencies when specifying needs/requirements as first time buyers are:

• Not to specify at all or on a so high level it does not give any value. The answers all look the same. You end up picking the one you know or the one you believe the most in.
• Or specifying in detail everything you can think of. You spend tremendous effort in detailing a very long list of requirements to be sure you cover all aspects. You end up with a complex solution.

The first one might be as good as the second. A big requirements phase is a danger in itself. You easily end up specifying today’s way of working. You don’t see all the changes a PLM solution should bring.

I suggest that you spend more time on reference visits and focus on the supplier, and not that much on the PLM solution itself. Find some of the references yourself. Don’t just take the ones that are picked by the suppliers. The key question: Is the supplier capable to be a long-term partner ensuring your success?

Hard to differentiate the PLM tools (even for experienced people)

I have been both on the specifying side as a customer, on the response side as an implementer and now on the customer side as an advisor in many PLM evaluations. And I see it again and again. The people evaluating PLM solutions are not able to distinguish clearly and rationally between the different options.

It is very hard to find facts about the differences. And the truth is that on a high level the differences among the enterprise PLM tools are quite small. You will see differences if you consider some lighter “PLM” solutions that are more CAD management oriented or ERP with PLM functionality. But even there it might be difficult. For CAD management oriented tools they might fall short on multi-CAD and integration to ERP. For ERP they might fall short on CAD integrations or on advanced PLM functionality like systems engineering. But for the core functionality it will seem that they all can do it.

The vendor’s tendency

The vendors tend to fall into one of two categories:

• The oversimplifyers – They oversimplify the message and the effort. This makes it easy to understand for management and other stakeholders that are not experts in this field.
• The scope-extenders – They want to extend the scope to show all the magnificent solutions they have that you didn’t ask for in the beginning

Be very aware of people in the extreme end of these two.

• The too good to be true is usually that. You will end up with a limited solution and realize you got less than you expected.
• The large scope might be too large. You end up with a very long and complex project which is more expensive than you thought and it is hard to get the organization on-board.

The sales guys anyway will paint a nice picture. The challenge for you is to see behind the fancy slide decks and prepared sales messages.

What should you do?

It is hard for first time PLM buyers to grasp the totality and at the same time identify the crucial details that can make or break your PLM project.

Don’t over-specify to compensate. You will not be able to specify up front how the solution and processes should be in 3 years. Even if you could the world will change in the meantime. It is more important to draw up the big lines and have a long term vision, strategy and plan. You should start with the most important areas and specify to such a level that you can differentiate. There are differences between the PLM solutions, you just have to know what to look for. If the differences is in an area that are of great importance to you it is better to find that out during the evaluation phase than in the implementation phase.

To be able to do this you could recruit or hire people who have done PLM evaluations and implementations before. It might seem expensive, but it will pay-off with a solution that is better fitted to your needs.

You might consider running a proof of concept (POC). But you risk going in the same trap. You have to know what to look for in the POC to get any value of it. Done properly a POC based on your data, products and your wanted way of working can be a really good approach.

You should look at other factors than just the feature list. The most important is the supplier itself: Have they done this many times before? Do they have understanding of your business? Do they act as real advisors guiding you in the right direction? Do they have references in your region?

See our How to select a PLM system blog post for more details about how to run a PLM evaluation.

Summary

PLM buyers: You don’t know what you don’t know about PLM. That means it is hard for you to distinguish the tools and the suppliers. Don’t try to specify everything. Have a clear vision of where you want to go and what you want to achieve and what is important for the business. Establish a commonly agreed strategy and a high level plan on how to get there.

Don’t go too much in detail, just where it matters. The challenge is to know which areas and details that are important for you and at the same time can differentiate the different tools and suppliers.

Focus instead on the suppliers and identify the one that is most likely to become a long-term partner ensuring your success. It is far more important to select the right supplier than selecting the right tool.

Tore Brathaug

www.infuseit.com