PMOs and projects regularly collect and report on data but you have to ask how often is the data converted into information the organisation can use?
A while ago I wasted 15 minutes trying to respond to a survey commissioned by our local city council. The structure of the survey was so bad there is no way the data collected would allow any useful information to be gathered. Two of the questions (from memory) were:
Overall ratings should be complied from specific measurements weighted appropriately. In this case a weighted assessment of the relative importance of clean shopping strips -v- clean local streets -v- clean laneways to the local citizens. How do you assess the weightings? Ask a sample group of people which they feel is more important on a comparative basis before running the survey. Then collect specific data and build some useful information that can help direct improvements in the overall service.
When designing this sort of data gathering, you also need to filter out influences like staff culture (the Council’s staff are really great and helpful) from systemic issues such as the contract conditions the street cleaning contractor operates under.
Then by compiling all of the various rating for the specific services, an overall rating for the council could be compiled and more importantly the high performing areas noted and lessons from these areas transferred to the less well performing areas.
There’s a lot to learn from this example of bad surveying. Designing surveys, collecting data and using the data to create useful information is not a trivial exercise. Success starts by focusing the data collection on a limited range of key factors that can provide useful management information then converting the data into useful information but even this is not simple, correlation does not equate to causation – see: http://mosaicprojects.wordpress.com/2012/01/02/charting-for-effect/
As this post has hopefully demonstrated, there is a real skill needed to develop an effective survey; unfortunately it’s a skill that seems to be in very short supply. There is nothing simpler than bogging a PMO down collecting masses of data that can never be converted into useful information. Do this and the PMO will be seen as a useless bureaucracy and sooner or later it will be reorganized out of existence; conversely, if the PMO can develop the skills to become a source of really useful information, it will be seen as a real value-add to the business! The choice is yours!!
On a closing note –
The number of really bad surveys seem to be increasing exponentially – I think around 80% of the various project management surveys I look at, mostly from post graduate students, seem to be either designed to support a pre-determined answer or so badly designed the data could be interpreted to suit any answer the researcher chooses. To appreciate just how easy it is to bias a survey, watch this YouTube clip: http://www.youtube.com/watch?v=G0ZZJXw4MTA Then answer the following survey question:
How do you rate your PMO overall? Only kidding…..