Project Managers

The Project Management Social Network | Project Manager Jobs

PMOs and projects regularly collect and report on data but you have to ask how often is the data converted into information the organisation can use?

A while ago I wasted 15 minutes trying to respond to a survey commissioned by our local city council. The structure of the survey was so bad there is no way the data collected would allow any useful information to be gathered. Two of the questions (from memory) were:

  • Overall how do you rate the “The Council’s” performance with a range of 5 options from bad to excellent. Sound like an insightful question but what does the answer tell anyone?? The Council delivers a wide range of totally separate services from refuse collection to maternal and child welfare. Any overall answer is meaningless; unless the Councilors are spending rate payer’s money on a ‘feel good’ ego trip and hope they will be rated ‘good’. It does not matter if the answer is bad, good or indifferent unless the data collected identifies what is good or bad so management action can be taken to lock in the good and improve the bad.
  • How do you rate the street cleaning service overall?  More specific but still useless data. The council cleans shopping strips on a daily basis, other roads on an occasional basis and numerous lanes and footpaths very rarely. My answer was shopping strips great, major roads OK, side roads like the one I live in fairly poor and the laneways are a total disaster….. But how would I rate the service overall??? At this point I terminated the interview!

Overall ratings should be complied from specific measurements weighted appropriately. In this case a weighted assessment of the relative importance of clean shopping strips -v- clean local streets -v- clean laneways to the local citizens. How do you assess the weightings? Ask a sample group of people which they feel is more important on a comparative basis before running the survey. Then collect specific data and build some useful information that can help direct improvements in the overall service.

When designing this sort of data gathering, you also need to filter out influences like staff culture (the Council’s staff are really great and helpful) from systemic issues such as the contract conditions the street cleaning contractor operates under.

Then by compiling all of the various rating for the specific services, an overall rating for the council could be compiled and more importantly the high performing areas noted and lessons from these areas transferred to the less well performing areas.

There’s a lot to learn from this example of bad surveying. Designing surveys, collecting data and using the data to create useful information is not a trivial exercise. Success starts by focusing the data collection on a limited range of key factors that can provide useful management information then converting the data into useful information but even this is not simple, correlation does not equate to causation – see:

As this post has hopefully demonstrated, there is a real skill needed to develop an effective survey; unfortunately it’s a skill that seems to be in very short supply. There is nothing simpler than bogging a PMO down collecting masses of data that can never be converted into useful information. Do this and the PMO will be seen as a useless bureaucracy and sooner or later it will be reorganized out of existence; conversely, if the PMO can develop the skills to become a source of really useful information, it will be seen as a real value-add to the business! The choice is yours!!

On a closing note –
The number of really bad surveys seem to be increasing exponentially – I think around 80% of the various project management surveys I look at, mostly from post graduate students, seem to be either designed to support a pre-determined answer or so badly designed the data could be interpreted to suit any answer the researcher chooses. To appreciate just how easy it is to bias a survey, watch this YouTube clip: Then answer the following survey question:

How do you rate your PMO overall?     Only kidding…..

Views: 782


You need to be a member of Project Managers to add comments!

Join Project Managers

© 2017   Created by Mike.   Powered by

Badges  |  Report an Issue  |  Terms of Service